SITE NEWS: We are moving all of our site and company news into a single blog for Sports-Reference.com. We'll tag all PFR content, so you can quickly and easily find the content you want.

Also, our existing PFR blog rss feed will be redirected to the new site's feed.

Pro-Football-Reference.com » Sports Reference

For more from Chase and Jason, check out their work at Football Perspective and The Big Lead.

How often does the best team win?

Posted by Doug on May 31, 2006

In the 1989 Baseball Abstract --- yes, there was a 1989 Baseball Abstract; I'll bet I am one of no more than ten people on the planet who has it on his bookshelf right now --- Bill James wrote an essay called How Often Does the Best Team Actually Win? Here is a passage from the introduction:

Yes, we know that the luck evens out in a 162-game schedule, but how consistently? Does the best team win the division, in a 162-game schedule, 90% of the time? 75%? How often? Does the best team in baseball win the World Championship nine years in ten, or two? Is it possible for a team which is in reality just average --- a .500 team --- to win its division (and therefore possibly even the World Series) by sheer luck?

Note that he was not asking how often the team with the best record wins the World Series, or how often a team with a .500 record would win. He was asking how often the team that really and truly was the best wins the World Series, and how often a team that was morally a .500 team would win the world series (most likely lucking into a better-than-.500 record in the process).

Questions like the former can't be answered by looking at real life results, but only because we don't have enough of a sample size. Questions like the latter, though, cannot be answered using real life results even if we live to see a million seasons. We don't know how often the best team wins the World Series or the Super Bowl because we don't know --- we can't know --- who the best team is. Pittsburgh may have been the best team in the NFL last year, or they may have been the 3rd best or the 14th best. We don't know how often a .500 team wins the Super Bowl because we don't know who the .500 teams are.

If you want to know how often the best team wins the title, you have to build a model. In that model, you can create teams whose strengths you know, because you defined them. James did just that, and he concluded that in Major League Baseball, structured as it was in the late 1980s, the best team wins the World Series 29% of the time. The best team in a division wins that division about 53% of the time. The best team in all of baseball missed the playoffs about 29% of the time.

These results seemed to make him a little uneasy. He closed the essay with this:

The belief that in a 162-game schedule the luck will even out is certainly unfounded --- but that unfounded belief may also be essential to the health of the game. Would people lose interest in baseball if they realized that the best team doesn't win nearly half the time? Would it damage the perception of the World Series if people realized that the best team in baseball only emerges with the crown about 30% of the time?

For me, no. It would not damage my interest, and for most of you also, I suspect. I am afraid that for some people, the answer would be the other one. I've learned a lot of surprising things in running these simulations, and I'm happy to have that knowledge....But I don't think it's something I'm going to talk about a whole lot.

I think he's got it backwards. I think it's the stat geeks who are concerned about the best team winning. The rest of the public, in my experience, doesn't give much thought at all to the notion of "the best team," or is content to define the best team to be the one that wins and/or to appreciate the unpredictability for unpredictability's sake. Furthermore, I don't think that, in a 26-team league, 29% is all that low. If the best team in baseball is morally a .600 team, say, then most years there are probably two or three more teams pretty close to that. If a third-best team that is within a few percentage points of the best team happens to win a title because of luck, I don't think anyone considers that a travesty.

In any event, I --- like James --- find the topic fascinating, and have for years been meaning to replicate this study for the NFL. Yesterday's post was not exactly like the James study, but was in some ways similar. And it prompted me to roll up the sleeves and get the simulator built. So I did. And I'm going to spend the next post or five discussing what kinds of things it spits out. Discussion will include, but not be limited to, the followng:


  • I'll answer the same questions James did. How often does the best team in football win the Super Bowl? How often does the best team in football fail to make the playoffs? How often does a sub-.500 team win the Super Bowl? It's not clear how the answers will differ from MLB circa 1989. On one hand, baseball plays ten times more games, which gives the luck more of a chance to even out. On the other hand, football simply doesn't have as much luck built into it as baseball does. If the worst team in baseball beats the best team, it barely raises an eyebrow. In football, that almost never happens.
  • I want to examine various playoff configurations and see how much the answers to the above questions change. For example, what if we eliminated the wildcard and simply let the eight division winners play a standard tournament? Would that increase or decrease the chances of the best team winning? It's not clear, not to me anyway. Sometimes the wildcard lets weak teams in, sometimes it lets strong teams in. What if we had four divisions of eight instead of eight divisions of four? How would that change things? What if, as a friend of mine advocates, we have two conferences of 16 teams each and no divisions at all?
  • I also want to briefly investigate questions along the lines of, how often does a sub-.500 team win its division? Unlike the first bullet, here I'm not talking about teams that were morally sub-.500. I'm talking about teams whose record was under .500. Similarly, we can investigate question like, how often should we see an undefeated team? How often should we see a winless team? What are the chances of a four-way tie in a division?
  • James didn't do this, but I think it will be fun to take a look at some specific teams in specific years. In the previous post, I talked about what would happen if we switched the 2004 Colts and Falcons prior to the playoffs. Now I'll talk about the what would have happened if we had switched them before the season started. This will require an extra step (i.e. leap of faith) which I'll explain when the time comes. As another example, I talked last week about the Chargers having a rough schedule last year. What if they had played the Panthers' schedule last year and the Panthers had played theirs?

Many of these ideas were touched upon in the comments to yesterday's post. If you have more suggestions of questions to ponder, bung them down in the comments.

This entry was posted on Wednesday, May 31st, 2006 at 5:11 am and is filed under Statgeekery. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.