How often does the best team win?
Posted by Doug on Wednesday, May 31, 2006
In the 1989 Baseball Abstract --- yes, there was a 1989 Baseball Abstract; I'll bet I am one of no more than ten people on the planet who has it on his bookshelf right now --- Bill James wrote an essay called How Often Does the Best Team Actually Win? Here is a passage from the introduction:
Yes, we know that the luck evens out in a 162-game schedule, but how consistently? Does the best team win the division, in a 162-game schedule, 90% of the time? 75%? How often? Does the best team in baseball win the World Championship nine years in ten, or two? Is it possible for a team which is in reality just average --- a .500 team --- to win its division (and therefore possibly even the World Series) by sheer luck?
Note that he was not asking how often the team with the best record wins the World Series, or how often a team with a .500 record would win. He was asking how often the team that really and truly was the best wins the World Series, and how often a team that was morally a .500 team would win the world series (most likely lucking into a better-than-.500 record in the process).
Questions like the former can't be answered by looking at real life results, but only because we don't have enough of a sample size. Questions like the latter, though, cannot be answered using real life results even if we live to see a million seasons. We don't know how often the best team wins the World Series or the Super Bowl because we don't know --- we can't know --- who the best team is. Pittsburgh may have been the best team in the NFL last year, or they may have been the 3rd best or the 14th best. We don't know how often a .500 team wins the Super Bowl because we don't know who the .500 teams are.
If you want to know how often the best team wins the title, you have to build a model. In that model, you can create teams whose strengths you know, because you defined them. James did just that, and he concluded that in Major League Baseball, structured as it was in the late 1980s, the best team wins the World Series 29% of the time. The best team in a division wins that division about 53% of the time. The best team in all of baseball missed the playoffs about 29% of the time.
These results seemed to make him a little uneasy. He closed the essay with this:
The belief that in a 162-game schedule the luck will even out is certainly unfounded --- but that unfounded belief may also be essential to the health of the game. Would people lose interest in baseball if they realized that the best team doesn't win nearly half the time? Would it damage the perception of the World Series if people realized that the best team in baseball only emerges with the crown about 30% of the time?
For me, no. It would not damage my interest, and for most of you also, I suspect. I am afraid that for some people, the answer would be the other one. I've learned a lot of surprising things in running these simulations, and I'm happy to have that knowledge....But I don't think it's something I'm going to talk about a whole lot.
I think he's got it backwards. I think it's the stat geeks who are concerned about the best team winning. The rest of the public, in my experience, doesn't give much thought at all to the notion of "the best team," or is content to define the best team to be the one that wins and/or to appreciate the unpredictability for unpredictability's sake. Furthermore, I don't think that, in a 26-team league, 29% is all that low. If the best team in baseball is morally a .600 team, say, then most years there are probably two or three more teams pretty close to that. If a third-best team that is within a few percentage points of the best team happens to win a title because of luck, I don't think anyone considers that a travesty.
In any event, I --- like James --- find the topic fascinating, and have for years been meaning to replicate this study for the NFL. Yesterday's post was not exactly like the James study, but was in some ways similar. And it prompted me to roll up the sleeves and get the simulator built. So I did. And I'm going to spend the next post or five discussing what kinds of things it spits out. Discussion will include, but not be limited to, the followng:
- I'll answer the same questions James did. How often does the best team in football win the Super Bowl? How often does the best team in football fail to make the playoffs? How often does a sub-.500 team win the Super Bowl? It's not clear how the answers will differ from MLB circa 1989. On one hand, baseball plays ten times more games, which gives the luck more of a chance to even out. On the other hand, football simply doesn't have as much luck built into it as baseball does. If the worst team in baseball beats the best team, it barely raises an eyebrow. In football, that almost never happens.
- I want to examine various playoff configurations and see how much the answers to the above questions change. For example, what if we eliminated the wildcard and simply let the eight division winners play a standard tournament? Would that increase or decrease the chances of the best team winning? It's not clear, not to me anyway. Sometimes the wildcard lets weak teams in, sometimes it lets strong teams in. What if we had four divisions of eight instead of eight divisions of four? How would that change things? What if, as a friend of mine advocates, we have two conferences of 16 teams each and no divisions at all?
- I also want to briefly investigate questions along the lines of, how often does a sub-.500 team win its division? Unlike the first bullet, here I'm not talking about teams that were morally sub-.500. I'm talking about teams whose record was under .500. Similarly, we can investigate question like, how often should we see an undefeated team? How often should we see a winless team? What are the chances of a four-way tie in a division?
- James didn't do this, but I think it will be fun to take a look at some specific teams in specific years. In the previous post, I talked about what would happen if we switched the 2004 Colts and Falcons prior to the playoffs. Now I'll talk about the what would have happened if we had switched them before the season started. This will require an extra step (i.e. leap of faith) which I'll explain when the time comes. As another example, I talked last week about the Chargers having a rough schedule last year. What if they had played the Panthers' schedule last year and the Panthers had played theirs?
Many of these ideas were touched upon in the comments to yesterday's post. If you have more suggestions of questions to ponder, bung them down in the comments.
This entry was posted on Wednesday, May 31st, 2006 at 5:11 AM and filed under Statgeekery. Follow comments here with the RSS 2.0 feed. Post a comment or leave a trackback.
"If the worst team in baseball beats the best team, it barely raises an eyebrow. In football, that almost never happens."
I don't know enough about baseball to say whether that's true, but I assume you have numbers behind it. What is the winning percentage of the worst team against the best team in each sport?
Posted on 31-May-06 at 5:51 am | PermalinkUh oh.
No, I've got no numbers, just intuition. But I went to baseball-reference real quick and looked at the 2005 NL. The Cardinals had the best record and the Rockies and Pirates were tied for the worst. The Cardinals were 4-4 against the Rockies and 8-4 against the Pirates. In the AL, the team with the worst record (KC) beat the team with the best record (Chicago) five times.
Posted on 31-May-06 at 6:15 am | PermalinkIt depends on how you figure out who the best team is. Using the rankings from http://www.footballoutsiders.com, I can tell you how often the "better team" from those rankings won in the 2005 regular season. At home, the better team won 73.1%. On the road, the better team won 61.2%. Of course, as you discuss, it is not simply black and white of who is better--there is a difference between how often Indianapolis would perform over a large sample size of games against Pittsburgh compared to Houston.
In games where the teams were relatively even (within about a FG on a neutral field), the better team won 55.8% at home and 46.8% on the road. In all other games, the better team won 84.1% at home and 70.3% on the road.
Here are my guesses to some of the questions you posed:
At least in the modern playoff format of multiple wildcards, I am guessing the best team has never failed to make the playoffs. However, teams in the top 5 will miss the playoffs, particularly if another team in their division is better.
As far as playoff formats, # of wildcards will increase the chances that the best team makes the playoffs, but at the same time, will decrease the chances that the best team wins the superbowl.
I dont think any team has won a division with a sub-.500 record. I think one did win with an 8-8 record in the AFC Central in the 1980's. However, the current format increases the chances that it will happen, because teams play fewer games in division now than they used to do.
In the current format, the schedule actually creates 4 divisions. the AFC West/AFC East played 10 of their games within that group, as did the AFC North/AFC South, NFC West/NFC East, and NFC North/NFC South. How about just giving the winner of each group of 8 an automatic place in the playoffs and a bye, and the remainder being wildcards?
Finally, rather than Chargers vs Panthers, how about Chiefs or Chargers vs Seahawks. After all, 4 years earlier, the Seahawks moved from the AFC West. If one of the other teams moved instead, do the Seahawks even make the playoffs? If they do, its at the expense of the other Super Bowl team, Pittsburgh. And the Chiefs or Chargers, playing in the NFC West, would have had a very good chance of going 6-0 in division like the Seahawks, and getting home field.
Posted on 31-May-06 at 6:54 am | PermalinkI love this blog! Being a math guy and a football fan, I couldn't ask for a better site!
This blog seems in ways to be football's answer to baseball's sabermetrics. I especially like pro football, and your blog here helps me to see it in a more complete light, and adds to the various aspects of the game that I already spend a lot of time thinking about. Thanks for the stellar effort... please keep it going!
Posted on 31-May-06 at 8:31 am | PermalinkNot sure this is relevant, but it is a question that touches on lesser teams beating better teams I think, so I'll pose it. In football there seem to be these "circles of dominance" like rock-paper-scissors, such as over the last few years, Denver owns New England, New England owns Indy, and yet Indy owns Denver. So in those cases who is better?
Posted on 31-May-06 at 8:42 am | PermalinkThis is one reason I'm not a big fan of a lot of the stats at footballoutsiders.com. Football is such a funny game, where a few big plays can make as much difference as consistency, where a few bounces of an wierd-shaped ball can determine the outcome of a game.
Posted on 31-May-06 at 9:06 am | PermalinkI can't really comment on this because I don't follow baseball but I have a few immediate thoughts...
Posted on 31-May-06 at 9:43 am | Permalink1. Football teams don't rotate QB's five times a week. Assuming five standard starters per team (ignoring any other starter), couldn't one argue that there are 150 "different teams" How many times did KC's 5th starter beat Chi's Ace?
2. The 1994 Cowboys didn't get a seven game series to best the 49ers. Frankly, all they needed was another quarter to make up for the unfortunate outcomes of the 1st quarter. The Law of Averages does not affect Football at all like it affects Baseball.
3. How do we measure baseball players? Averages. Football players? Totals.
4. I think the whole point is the best team over the course of the season might not be the best team on any given sunday for many reasons. Thats why we love it. Thank GOD the New York Giants have not won 27 superbowls (or however many the Yanks have won) - I think many would argue the fact that the Cowboys have five is virtually disgusting. Sure, the SB has not been around as long - but you get my point.
after re-reading my fourth point, let me clarify that I know that baseball is even more the case of "any given day"... - my point was that in baseball, you've got a lot more opportunities to make up for a single day's anamoly. And that, as I meant, is why I love football. A seven game series between the 1998 Cards and Cowboys would have resulted in the Cowboys winning - and what fun would that be. Answer: No fun.
Posted on 31-May-06 at 9:53 am | PermalinkI feel the need to explicitly make clear that I attach no value judgements to the answers to any of these questions.
Some people may want the best team to win 75% of the time, while others may want the best team to win 10% of the time. Some people --- like me --- don't really care but are just interested in knowing. Well, OK, knowing approximately.
Regardless of what comes out in the next few posts, I will be making no actual or implied statements about whether it's a Good Thing or a Bad Thing.
Posted on 31-May-06 at 10:21 am | Permalink"This is one reason I’m not a big fan of a lot of the stats at footballoutsiders.com. Football is such a funny game, where a few big plays can make as much difference as consistency, where a few bounces of an wierd-shaped ball can determine the outcome of a game."
Pacifist, I'm not sure what your point is. Isn't the idea of footballoutsiders to try and separate the skill from the flukes and put ratings on it? Yes, flukey plays are inherent in football, but better teams probably have more flukes land in their favor over a whole season.
Posted on 31-May-06 at 10:36 am | PermalinkI'm also intrigued by the comment that football doesn't have as much luck built into it than baseball does. I'd like to see a better argument for that. The worst beating the best discussion really opens up a whole new can or worms touching on competitive balance, parity, season length, labor restrictions, among others, and it's not very clear that we're talking entirely about luck in the general sense.
Posted on 31-May-06 at 10:38 am | PermalinkIf nothing else, I'd guess that injury risk inherent to football adds a significant luck factor not seen in the other major sports.
Richie, I understand that point. And I haven't read footballoutsiders enough to make a real credible critique. But when I have read it, I've been turned away by things like "estimated wins" based on factors other than simply who wins games. That goes against the nature of football as a unique sport. In basketball, points only come in 1s, 2s, or 3s. In baseball, the most points one play can get is 4--and that's rare. In football, you can get to 6 with two decent drives and field goals, with a consistent 12 play drive, or with a 90 yard pass, or even a fumble recovery. It's weird. Figuring out averages and projections in football, in the same way it might be done in other sports, seems odd. Football seems to be about who is able to come up with an important play in the big situations--not about what the projections and averages figure out to (and I don't say that to disparage the use of statistics to make sense of the game). That's why teams that do little things on defense and special teams can eke out wins and even championships against teams that seem to be dominant statistically on offense and/or defense. I understand the basic principle of what they're doing, and I do read it, but I don't put a lot of weight into the findings.
Furthermore, I think in football the clearly better team is going to win a high majority of the games against a clearly inferior team. For evidence, look at college football. But in the NFL, with 32 teams and rules built in for competitive balance, more of the games are played between teams that are slightly better or worse than each other, so the games could go either way on most Sundays (though I think, and the numbers over the last few seasons back it up, that parity is a myth).
This is a pretty incoherent and contradictory response--take it for what it's worth.
Posted on 31-May-06 at 11:27 am | PermalinkAt least right now, I'd probably be inclined to simply retract the statement than to attempt to give a better argument for it.
Maybe what I should have said was "the range in true strengths among NFL teams is greater than that of MLB teams." What I was getting at was this: if MLB played a 16-game season with a 12-team single-elimination playoff, the best team would win nowhere near as often as the best football team wins the Super Bowl.
Hence, there is something different with the competitive structure between the two sports. What is the best way to describe that difference? I don't know. But whatever it is, it causes fewer games to be necessary for an NFL season than an MLB one. My only point was: fewer yes, but is only 10% enough?
Posted on 31-May-06 at 11:38 am | PermalinkJust thinking out loud a bit:
If baseball played 1 game a week for 16 weeks, like football, they would probably totally change the way they used pitchers. A 12 man pitching staff would be unnecessary. Would an ace starter end up pitching the whole game, or would a team have 4 or 5 pitchers on the staff, with all of them pitching 1 or 2 innings every game (like the all-star game). Or, would the talent distribution just be totally different. Would fewer young players be pitchers, and the great pitcher athletes become hitters instead, increasing the overall talent for hitters? Would a team have more hitter on the roster so they could make more defensive substitutions and pitch hitters in every game? This is an interesting topic that I haven't given much thought to before. I suppose there are 2 ways to look at it. 1) using current baseball roster compositions and instantly switching to a 16 game schedule, or 2) assuming a 20+ year history of 16 game schedules and roster compositions based on such.
Either way, I think that the act of hitting a baseball is so difficult (the best hitters fail 70% of the time) that the difference between success and failure is so fine that there is a lot more chance brought into the equation.
While in football, so much of dominance is based on (in my opinion) the physical strength of offensive linemen. A strong offensive line is not going to fail 70% of the time. In fact, there are many degrees of failure. An offensive line may not block as well on one play as they did the other, but it was still good enough. Not so for a baseball hitter. He may have a great swing, almost as good as his last swing which was a home run, but this time it is just a pop up.
So - I'm thinking there is a lot more chance built in to the outcomes of baseball games.
Posted on 31-May-06 at 12:20 pm | PermalinkAnother quick point about Super Bowls. I don't follow other sports enough to know if this is the case in other championships, but as a Bills fan, I know a thing or 4 about Super Bowls
A team's confidence (any team) is very very fragile in Super Bowls. Take the Bills for example. They lost XXV to the Giants in a very close game. Their whole reason for being the entire next season was to make up for that. Unfortunately, a solid first half put the Redskins far enough ahead in XXVI to the point where, despite 24 second half points, the Bills were out of the game. Next, for a 3rd season in a row, the Bills mow over everybody else again and get back to the big game, this time to face the Cowboys. The Bills led 7-0, but early turnovers hurt them and they got killed. How confident could they possibly be? They lost the previous 2 such games. Then in their fourth Super Bowl, the Bills went into the locker room up 13-6 at half time, but they must have known the game was far from over. Then, bang, a fumble gets returned for a TD. In any other game, big deal, 13-13, but in a Super Bowl, confidence is fleeting. They went on to score no more and lose 30-13. The Bills would have had to play a perfect game in order for their confidence not to be shot having lived thru such big scale heartbreaks in recent times.
Consider another fact. No team in a Super Bowl has recovered from more than a 10 point deficit and gone on to win. In regular season games and playoff games (and pre-season games for that matter), a 10 point deficit is not a death knell. In a Super Bowl, it's curtains.
Probably because of the difficulty of getting there and because of the extreme stakes of the game for a team's psyche, confidence can be lost in a single play. My theory is that the team to win the Super Bowl is the one not to have their confidence shaken first. I believe the Bills were as good during that period as any team in any 4 year period, but their confidence was fragile in light of previous shortcomings.
Another example.... remember when the Patriots got up 32-29 over the Panthers in the Super Bowl and had to kick off with a few seconds left in the game. For some reason, Carolina made virtually no effort to set up a big return or a trick play or anything to try to win the game. You'd think there'd be a lateral or some sort of last ditch effort in hopes of getting lucky. They must have been psychologically defeated and not even tried to get lucky. A lot of pressure in Super Bowls.
Posted on 31-May-06 at 2:04 pm | PermalinkOn the topic of the relative amounts of luck involved in football and baseball, the answer lies in the moneylines. They are efficient enough to trust -- i.e., teams that are 3-2 favorites really do win about 60% of the time.
So if good football teams tend to be bigger favorites over bad football teams than good baseball teams are over bad baseball teams, it would seem that baseball involves more luck (per game) than football. (At least in the sense Doug seemed to mean -- i.e., that the worst teams upset the best teams more frequently in baseball. "Luck" in this sense, however, may just be a synonym for parity.)
Posted on 31-May-06 at 4:26 pm | PermalinkBTW, if we can come up with decent moneylines for individual games in a series, it is straightforward to compute the appropriate moneyline for the entire series. I'm sure readers of this blog can do it on their own, but here's an example.
This can tell us how much of the luck factor in baseball is negated by its use of best-of-seven series in the playoffs instead of using individual games like the NFL does.
Posted on 31-May-06 at 5:54 pm | Permalinkre: 6
One of the best examples:
Houston at Pittsburgh, 2002
first downs- Houston 3, Pittsburgh 24
plays- Houston 40, Pittsburgh 95
total yards- Houston 47, Pittsburgh 422
score- Houston 24, Pittsburgh 3
Judging solely by the final score, Houston thoroughly demolished Pittsburgh. However, they really didn't do much all day aside from three plays- a 40-yard fumble return by Kenny Wright, a 70-yard INT return by Aaron Glenn, and a 65-yard INT return by Glenn.
Posted on 31-May-06 at 6:09 pm | PermalinkMentos,
I watched that whole game with a good Steelers buddy of mine. Just a really whacky game. Always stands out in my head. David Carr was 3/10 for 33 yards, Jonathon Wells had 10 carries for 12 yards, and James Allen had 13 carries for 19 yards. One of the weirdest games I've ever seen. Pittsburgh did thoroughly dominate Houston that game (it's not like Houston was protecting a lead all game, they were just unsuccessfully trying to move the ball).
Posted on 31-May-06 at 6:54 pm | PermalinkLots of good points on the differences between baseball and football as relates to the likelihood of the "better team" losing any particular game. I'll throw in another factor. Fewer plays. And the fewer the plays that could affect the outcome of the game, the more likely the upset will occur. To put it another way, with more opportunities, the better team will tend to rise the top at a higher rate.
In a regulation baseball game, there are 27 at bats, plus additional at bats for hits, walks, etc. I would guess, on average, somewhere around 40 at bats a game for each offensive team. In football, teams probably average around 65-70 offensive plays from scrimmage, plus other plays that could affect the outcome, like field goal attempts, kickoffs, and punts. That probably puts those teams over 80 plays, or around double that of a baseball game.
What happens when we cut the football plays in half? (I know, teams would have used different strategies if the games really ended at half time, but here are some of the results):
Cleveland (8-8) finishes with a better record than Jacksonville (7-9). Houston (6-10) is not the worst team in football, and it is the Jets (4-12) who get to pass on Reggie Bush instead. Carolina (9-7) ekes into the playoffs in the final week, but has to go to Seattle (11-5), who loses out on a bye on tiebreakers to Tampa and Chicago. Indy (12-4) doesnt get a bye either, losing out to Denver (13-3) and Pittsburgh (13-3), and has to play Cincy (10-6), (and Palmer plays the whole half -er, game- and stays healthy). Buffalo (7-9) actually wins the AFC East in a 3-way tiebreaker with New England and Miami. San Diego (11-5) makes the playoffs and draws Buffalo. My Kansas City Chiefs (9-7), alas, come up one game short again. Even with my revisionist history, Carl Peterson is begging to add a 7th playoff team, and the Chiefs are without a playoff win since 1993.
Posted on 31-May-06 at 7:31 pm | Permalink[...] Support pro-football-reference.com « How often does the best team win? [...]
Posted on 01-Jun-06 at 4:32 am | Permalink[...] Following up on a 15-year-old idea of Bill James, I decided to simulate 10,000 NFL seasons and see what would happen. Well I’m going to milk that idea for several posts. So if the idea doesn’t intrigue you, you might want to check back in in a week or so. I’ll still be here when you get back. [...]
Posted on 02-Jun-06 at 4:16 am | Permalink[...] How often does the best team win? [...]
Posted on 06-Jun-06 at 5:57 am | Permalink[...] A few weeks ago, I did several posts (I, II, III, IV) on the idea of simulating a bunch of NFL seasons and seeing what kind of crazy stuff happens. [...]
Posted on 28-Jun-06 at 4:29 am | Permalink[...] In case you just stumbled in today, this post is the latest in a long string. Read these first: I, II, III, IV, V. [...]
Posted on 29-Jun-06 at 4:10 am | Permalink[...] Delving even deeper into the geeky side of things, over at Pro-Football-Reference.com they cranked up their silicon and actually ran 10,000 simulated football seasons with the goal of finding out how often the best team wins the Super Bowl. The project is spread across 5 articles (1, 2, 3, 4, 5). [...]
Posted on 21-Dec-06 at 3:03 am | Permalink