Posted by Doug on June 1, 2006
You'd better read yesterday's post if you haven't yet.
So the plan is to simulate an NFL season a bazillion times and observe what kind of wacky stuff happens. Here are the particulars.
For each simulated season, I will assign each team a true strength which is a random number from a normal distribution with mean 0 and standard deviation 6. This means that the teams' true strengths are mostly somewhat close to zero. In particular, roughly two-thirds of all teams will have true strengths between -6 and +6, about 95% of all teams will have true strengths between -12 and +12. As you probably guessed, these numbers were rigged so that they generally agree with the values that the simple rating system produces for real NFL seasons in this decade.
You'll note that, even though it will be true for a real NFL season, I am not requiring that the teams' strengths in a given year average zero. Even though we can't observe it (at least not easily), there must surely be years when the league is stronger and years when it's weaker. And in any case, since we are primarily interested in questions like "how often does the best team in football (for that year) win the Super Bowl," it doesn't matter much.
Each simulated season had the same league structure and schedule as the 2005 NFL. That is, there were 32 teams divided into eight divisions of four teams each, and the schedule is just like that of the 2005 NFL.
There is one potential complication here, but I think it's minor. In the simulated world, each season is independent of the previous one, so the two intra-conference games in each team's schedule that are determined by last season's finish are instead essentially against random teams. In the real NFL, the seasons are not independent and good teams probably end up playing very slightly stronger schedules in general than bad teams do. Fortunately, this effect isn't nearly as dramatic now as it was in the 80s and 90s.
Also, I was too lazy to program the tiebreakers. All ties were broken by coin flip. I don't think this will affect anything, but let me know if you think I'm wrong about that.
Finally, the individual games are played by using the same formula we used in this post:
Home team prob. of winning =~ 1 / (1 + e^(-.438 - .0826*diff))
where diff is the home team's true strength minus the visiting team's true strength.
OK, that's that. Let's get to the question of the day, which is: how often does the best team in the NFL win the Super Bowl?
The answer is roughly 24% of the time.
I simulated 10,000 seasons. The table below shows that the best team won the Super Bowl 2,399 times, the second-best team won it 1,448 times, and so.
[NOTE: if you thought this table looked slightly different earlier, you're not seeing things. I accidentally inlcuded the wrong table at first, so I updated it about an hour later.]
Very nearly 50% of the time, the Super Bowl champion was one of the best three teams in football. And let me reiterate that when I say "the best team," I am not necessarily talking about the team with the best record. I am talking about the best team. Remember, we're omniscient here. We know which team really was the best.
I'm sure what caught your eye was that the 32nd-best (i.e. the worst) team in the NFL won the title once. Let me tell you about that season.
It was simulated season #6605. The Seattle Seahawks were truly a great team (true strength +15.1) and they played up to their potential, posting a 15-1 regular season record. The Chicago Bears were the worst team in football, but with a true strength of -9.0, they really weren't that bad, at least by worst-team-in-football standards. The NFC North was relatively weak, and Chicago took the division with an 8-8 record.
The Bears' first round playoff opponent was the Carolina Panthers, who were not great (+2.8) but had posted a 10-6 record to finish second in the NFC South. The game was in Chicago, of course, and it was therefore only a mild upset when Chicago won it. Chicago then beat the Saints in New Orleans and the Seahawks in Seattle to reach the Super Bowl.
The AFC was weak in 6605. The best they had to offer was the Jets (+7.2) who had gone 12-4 in the regular season and had beaten the Colts on the road to reach the Super Bowl. The Bears beat the Jets to win the title.
As James points out in his article, there is no single event here that is too hard to believe. It's not unlikely that there wouldn't be any truly terrible teams in the NFL in a given year. It's not unlikely that an entire division would be weak, and it's not unlikely that the worst team in such a division could win the title with an 8-8 record. In their four playoff games, their probabilites of victory were 37%, 10%, 8%, and 21%. That they'd win those four games is certainly unlikely, but no more unlikely than, say, an NL team getting four straight hits at the bottom of their batting order, and I'll bet you've seen that.
No one of those things is terribly bizarre. Yet they all come together to create an almost-unbelievable occurrence. Almost unbelievable. Ten thousand years is a long time. Most of you have probably been watching NFL football for 20 or 30 years, and think of all the crazy stuff you've seen in that time. If you lived another 500 lifetimes, you'd see some even crazier stuff.
Do you think you'd ever see a team like the 2005 Jets win a Super Bowl? And I'm not talking about the Jets if Pennington and Curtis Martin had stayed healthy. I'm talking about the Brooks Bollinger Cedric Houston 2005 New York Jets. If you gave that team 10,000 tries, would they win a Super Bowl? Before you say no, think about all the times you've seen a really bad team rattle off three or four unexpected victories; think of the Craig Krenzel-led Bears during that stretch in 2004, for example. Such runs are unlikely, but you've seen lots of them. Don't you think that, in 10,000 years, some team could string a couple of those runs together, get some breaks from the schedule, and then fluke out in the playoffs?
It could happen.