Pro Football Reference Blog

Last week I posted some quick lists of bad teams that made the playoffs and good teams that didn't.

In the comments of the former appeared this:

2004 really was a bad year for the NFC! I can see at least 4 teams on the list [of below average teams that made the playoffs], and the Falcons are 16th, despite IMO being clearly the second best team in the conference that season.

Four of the six playoff teams in the NFC that year were indeed below average according to the simple rating system. In fact, according to that system, 14 of the 16 teams in the NFC were below average. The average rating of all AFC teams was +7.8, which means the average rating of all NFC teams was -7.8, which means that an average AFC team was 16 points better than an average NFC team in 2004. I'll do a full post (or more) on conference imbalance someday, but for now I'll just say that that differential is the highest since the merger. The NFL was an absurdly imbalanced league in 2004.

This is probably the place to remind everyone, self included, that the ratings are just rough estimates and we should be attaching some mental error bars to them. In particular here, I think the Eagles' rating is likely an understatement of their strength because they mailed in their last three games. This would have a ripple effect on the rest of the NFC, which might mean that, really and truly, only 11 of 16 teams being below average instead of the 14 we're estimating above. Or something like that. Anyway, it doesn't change the fact that the NFL was an absurdly imbalanced league in 2004.

Consider the Colts and the Falcons, for example. In order to reach the Super Bowl, the Colts would have had to first beat a Denver team that was arguably better than any team in the NFC. Then they would have had to beat a 14-2 team and a 15-1 team --- both of which compiled their records against tougher-than-average schedules, I might add --- on the road. That's rough. All the Falcons had to do was win two games, one of them against a below-average opponent. If you believe that teams who accomplished more in the regular season should be rewarded with an easier postseason road, something which is implicitly assumed in the postseason structure of every sports league I'm aware of, then you have to consider this unfair.

I decided to investigate just how unfair it was. The basic idea is this: estimate the Colts' chances of reaching and/or winning the Super Bowl, and compare it to what their chances would have been had they been in the other bracket.

The first thing we need to do is find a formula that relates two teams' ratings to their chances of winning a game between the two of them. I'll skip the details, but here is the formula I used:

Home team prob. of winning =~ 1 / (1 + e^(-.438 - .0826*diff))

where diff is the home team's rating minus the visiting team's rating. If the home team is 7 points better than the road team, this model gives the home team a 73% chance of winning. If the home team is 7 points worse, this model gives the home team a 46% chance of winning. I wouldn't go to war with any bookies using this alone, but it should serve our purpose here, which is to give us the rough estimates needed to simulate the playoff tournament a few bazillion times. That will then give us a rough estimate of each team's probability of winning the Super Bowl.

Here were each team's estimated chances of reaching and winning the Super Bowl at the beginning of the playoffs in 2004:

      ReachSB WinSB

===================

1. pit  35.4  22.1

2. nwe  35.7  24.6

3. ind  13.5   9.2

4. sdg   9.2   5.9

5. nyj   3.4   1.8

6. den   2.8   1.8

1. phi  56.3  22.4

2. atl  19.5   5.4

3. gnb  11.5   3.5

4. sea   6.4   1.6

5. stl   2.4   0.6

6. min   3.9   1.1

Anyway, let's see what happens if you switch the Colts and Falcons, giving the Colts the two seed in the NFC and the Falcons the three seed in the AFC:

      ReachSB WinSB

===================

1. pit  37.7  19.8

2. nwe  41.7  24.9

3. atl   2.0   0.6

4. sdg   9.9   5.3

5. nyj   3.8   1.8

6. den   5.0   2.6

1. phi  41.2  17.1

2. ind  47.1  24.8

3. gnb   5.2   1.6

4. sea   3.3   0.8

5. stl   1.2   0.2

6. min   2.1   0.5

The Colts' chances of reaching the Super Bowl would have been about three to four times greater had they been in the other league. The Falcons' chances would have decreased by a factor of 10 had they been in the other league. The Bills missed the playoffs in the AFC. Had they been the #6 seed in the NFC, they would have had a 15% chance of getting to the Super Bowl.

Finally, this comes from the comments of the "best non-playoff team" post:

Don’t forget, that 1991 San Francisco team lost to the Falcons on a Hail Mary pass (Tolliver to Haynes, I believe for 44 yards). If that pass is incomplete, SF goes 11-5 and wins the division, NO is a wildcard team and Atlanta misses the playoffs entirely.

Had things played out that way, San Francisco would have had an estimated 16% chance at reaching the Super Bowl and a 10% chance of winning it, and those numbers would be quite a bit higher had the 1991 Redskins not been such a juggernaut.

Yes, yes, I know. That's the way the ball bounces, that's why they play the games, great teams will find a way to overcome bad breaks, and so on and so forth. Anyone with the urge to post, "the Patriots won the 2004 title on the field and that's all that matters" will not be telling me anything I don't know. I get that. I am aware that it's meaningless to say that being in the AFC cost Indianapolis .156 Super Bowl titles in 2004.

For some reason, it's something I wanted to know anyway.