Pro Football Reference Blog

It's been awhile since I utilized the Insane Ideas tag. This is one of those things that has zero chance of being even listened to, much less accepted, by any audience of football fans outside this board. I'm not sure it has much of a chance even here, but I'll give it a shot.

Ever since I can remember, the use of head-to-head as the first tiebreaker seemed a little funny to me. Not wrong, necessarily, just funny. And that's what I'm going to discuss here in this post.

Just to avoid unnecessary extraneous issues, let's fix the conversation on the cleanest possible case. Namely, we'll assume the conference in question plays a full round robin and no other games (that count in the standings). PAC 10 football is an example of this scheme, and a timely one, as they might be heading toward a potential USC / Oregon State tie at the top this season.

At least on a theoretical level, any tiebreaking rule involving only wins and losses --- including head-to-head --- has a flip-side that argues against itself. After all, these teams have the same record. That necessarily means that if one of them had a better record in a certain subset of games --- the one where the two played each other, for instance --- then the other one must necessarily have had a better record in the other subset. If Oregon State was 1-0 and USC 0-1 in games between Oregon State and USC, then that means that USC must have had a better record than Oregon State, against the exact same teams, in the other games. Why favor one set of games over the other?

I think the answer is that, to most people, it just seems morally right. The fact that USC beat Stanford and Oregon State didn't isn't as important as the fact that Oregon State beat USC. In other words, most people think that Oregon State is more deserving of the title than USC is. And I really can't argue with that.

But what I'm interested in is a somewhat more objective question: does the head-to-head tiebreaker do a good job of crowning the better team? Most people would agree that, if a tie should come to pass in the PAC 10 this year, the better team will not be the one wearing the crown.

I think most people are fine with that, and I'm not saying I'm not. I'm just wondering: is that typical? When two teams end up tied at the end of the year, which team is more likely to be the better team: the team that won the one-game subset of games between the two teams? Or the team that did better in the eight-game slate against the rest of the conference? The team that beat USC but lost to Stanford? Or the team that lost to Oregon State but beat Stanford?

Using real-life data is probably out of the question, as this kind of situation doesn't happen often enough to get a good sample, so I ran a simulation. I simulated a gajillion PAC 10 seasons using rules similar to (but not exactly the same as) those established in the Ten Thousand Seasons post from way back. Remember, in a simulation, we can be omniscient. In each of these gajillion fake seasons, unlike with real football seasons, we know who the better team is. The better team doesn't always win, of course (if it did, we wouldn't need tiebreakers at all), but at the end of the season, we can say definitively whether or not the best team won.

So every time there was a two-way tie for first, I gave the crown to the team with the head-to-head win, but I noted whether or not that team was the better team.

What do you think happened?

A. The conference champ (i.e. the head-to-head winner) was the better team more often than not

B. The conference champ was the worse team more often than not

C. The results were not statistically distinguishable from a 50/50 split.

ANSWER EDIT...

Before running the study, my thoughts were exactly the same as those spelled out by commenter "Dead Cat Bounce":

I’m guessing (B). The other subset is bigger, 8 games, than the head-to-head subset of 1 game. Statistically, it would be tougher for the “worse” team to maintain their advantage over a longer series than just springing an upset in a single game.

[NOTE: he actually wrote (A), but I am pretty sure, based on the rest of his comment, that he meant (B). If not, I apologize.]

While the one game is direct evidence and the other eight are indirect, the other eight are more evidence and the one is less. A lot less. That's why I thought, like DCB, that (B) would be the answer.

But it wasn't. The actual answer is some combination of (A) and (C). In particular, the technically correct answer is (A): the head-to-head winner was in fact the better team more often than not and the margin was statistically significant. But (C) might be the more practically correct answer. Though statistically significant because of huge sample size, the margin was about 51/49 or 52/48 or something that, for practical purposes, is essentially 50/50.

So where did my and DCB's reasoning go wrong?

Here's where: if all you know is that one team did better than another team in an N-game season, then it definitely matters how big N is. But if you know, as we do in this case, that one team was EXACTLY ONE GAME better than another team in an N-game season, I don't think it matters much how big N is. In this example, while USC did better than Oregon State in the bigger 8-game part of the schedule, we know by virtue of the fact that the teams are tied overall, that they really didn't do better in the 8-game part of the schedule. They only did better in one of those games, the Stanford game in this case. So it's not an 8-vs-1 situation. It's really a 1-vs-1 situation with the other seven necessarily canceling out.