SITE NEWS: We are moving all of our site and company news into a single blog for Sports-Reference.com. We'll tag all PFR content, so you can quickly and easily find the content you want.
Also, our existing PFR blog rss feed will be redirected to the new site's feed.
Pro-Football-Reference.com » Sports Reference
For more from Chase and Jason, check out their work at Football Perspective and The Big Lead.
ShareThis#1 vs. #1 In the Super Bowl
At the Hall of Fame's website, I came across a post that reminded me of a little research project I had wanted to do but never got around to it... As you probably already know by now, the #1 seeds from each conference have not faced each other in the Super Bowl since 1993, when the Cowboys beat the Bills in SB XXVIII, and in the 19 full playoffs since 1990 (when the league expanded the playoffs to the current 12-team format), the 1-vs-1 matchup has only taken place twice -- 1991 & 1993. Much has been made about this phenomenon, almost to the point that certain people said it was actually a bad thing to be a #1 seed (that HoF page said "there was a slim chance of the Colts and Saints meeting in South Florida in February"), so I wondered, is there something about the current format that discourages #1 seeds from facing off in the Big Game (i.e., what we're seeing since 1990 is to be expected), or are we in the midst of a random stretch where the #1 seeds simply haven't happened to make it very often, and a correction is to be expected in the future?
To find an answer, I did what I usually do in situations like these: I built a Monte Carlo simulation. This time, I simmed the 2009 NFL schedule after assigning a random SRS-style strength rating to each team (just like Doug did here), and used the following equation to find the probability of the home team winning a game:
Home team prob. of winning =~ 1 / (1 + e^(-.438 - .0826*diff))
Where "diff" is the home team's true strength minus the visiting team's true strength. I ran through every game in the season, all the way through the Super Bowl (tracking how often the game featured two #1 seeds), in 19-year blocks, and repeated this process 2,500 times. Here's the distribution that resulted:
| 1-vs-1 in 19 Yrs | Count | Pct |
|---|---|---|
| 0 | 36 | 1.4% |
| 1 | 168 | 6.7% |
| 2 | 397 | 15.9% |
| 3 | 556 | 22.2% |
| 4 | 518 | 20.7% |
| 5 | 421 | 16.8% |
| 6 | 231 | 9.2% |
| 7 | 120 | 4.8% |
| 8 | 41 | 1.6% |
| 9 | 9 | 0.4% |
| 10 | 3 | 0.1% |
Overall, in the 47,500 simulated seasons, #1 took on #1 in the Super Bowl on 9,472 occasions, or 19.9% of the time. Compare that to the 10.5% observed rate we're looking at since 1990 and there's definitely been a drought in the real-life results -- but how improbable is the current 2-for-19 stretch? Well, in the 2,500 19-year blocks I simulated, a 2-for-19 run happened 397 times, or in 15.9% of the trials, making it the 4th-most-likely outcome behind 3-for-19, 4-for-19, and 5-for-19.
So the answer to the original question is that while what we're seeing now (at least before this weekend's games) is somewhat unlikely, it's certainly not as rare as some of the media coverage would have you believe. If these simulation results hold water, we should see an uptick in the rate of 1-vs-1 matchups at some point, but the simple truth is that the best team in a conference doesn't always make it to the Super Bowl, and the #1 seed isn't always the best team in the conference anyway. Besides, if one of the selling points of the NFL is its parity, is it even such a bad thing that the clear-cut most dominant teams in their respective conferences can only be expected to square off in the Super Bowl 20% of the time?
This entry was posted on Friday, January 22nd, 2010 at 12:08 pm and is filed under Statgeekery. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.
Good post, Neil. This empirical analysis matches up pretty well with what my gut says, so I'm highly inclined to believe in it.
As evidenced above, any # is unlikely; if #1 vs. #1 happened 4 times, that would be really unlikely, too. I suppose what people mean when they say "#1 vs. #1 rarely happens" they would only say that if they met 0, 1, 2 or probably 3 times since 1990. Well, there's a 46.2% chance over any 19 year period that the #1 seeds would meet 0, 1, 2 or 3 times.
The more likely "complaint" is that it's happened zero times since 1993, but it harkens back to Doug's post "What are the odds of that", which is a must-read, IMO.
So, expand things a bit. There are four major sports. What are the odds that, in one of the four major sports, the top seeds in each conference would fail to meet over a 15-year period? Probably not too unlikely, and odds are we'd only hear commentators in that sport remark about how crazy that is (and, of course, we'd never hear other commentators remark about how ordinary it is that #1 seeds have met 36% of the time). Considering the NFL has a single elimination playoff format, it surely would be the frontrunner for "most likely sport to have had this happen." And, of course, it's worth pointing out that the #1 seed is not usually the best team in the conference.
OK, I ran a sim of just 15-year stretches, too:
So when you just look at the drought since 1993, then it does become quite a bit less likely that you'd see 15 consecutive seasons without a 1-vs-1 matchup. But seeing zero in 15 years is still more than twice as likely as seeing a 1-vs-1 happen 47% of the time, which feels like a rate that's closer to what the commentators have been expecting.
I think the lack of #1 vs #1 Super Bowls shows a couple of things:
1) In a single-game playoff system (as opposed to a multi-game series), the chances of an upset increase. So, if either #1 seed happens to have a bad game, it kills the chalk matchup.
2) The #1 seed is probably not the best team in the league for multiple reasons (short season, fluctuating rosters due to injuries, schedule strength variations).
3) I think Bill Simmons' theories of the lessening value of home field holds some weight.
4) The morally "best" team in each conference probably isn't that much better than the morally "6th-best" team in each conference.
The NFL first started using seeds in 1975. From 1975-96, the team with the best overall record in the regular season won 14/22 (63.6%) Super Bowls. Since then, they've won just 1/12 (8.3%). Only the 2003 Patriots won it with a 14-2 record. The only other teams that even reached the SB were taken down in big upsets (2001 Rams, 2007 Patriots). The 2009 Colts are trying to become just the second SB winner since the 1997 season.
Mayb the lower seeds are just getting better than what they used to be, while the top seeds aren't as dominant as some of the older teams. And one & done does make increase the chances of a major upset, but the NFL has always used one & done.
I love this type of post. I have a feeling I'll be linking to this post if the Colts or Saints slip this weekend.
On a more theoretical note: How do you adjust for homefield advantage here? I mean, how big is the adjustment (if any) and is it in terms of points or SRS-points?
Re: 4, The salary cap, I would imagine, is responsible for a great deal of that change. Some of those old Packers, Steelers, Raiders, & Cowboys teams of the 60s/70s had almost 10 Hall of Famers on one roster, which is mind-boggling in and of itself, and something that would never, ever happen in the cap era.
Re: 5, HFA was built into Doug's original logit regression, and it's 60.8%, which is a little bit higher than the real-life % (it was like 58-59% this year IIRC), but not really a huge difference. I've found that logistic regression has a tendency to overvalue the home advantage in basketball as well, and I'm definitely open to suggestions as to why that's the case.
"2) The #1 seed is probably not the best team in the league for multiple reasons (short season, fluctuating rosters due to injuries, schedule strength variations)."
I think the simulation would probably catch this if I got the methodology right, if those things were random - however, scheduling could produce a systematic bias, especially in the short term. Teams' strengths aren't random year-to-year, and so if you get a situation like the NFC East or AFC South where you've got multiple good teams in a division.
One thing that would be interesting to look at is this:
1) Calculate the average SRS of the #1 seeds in the simulation.
2) Average the SRS of all #1 seeds over the past 15 years.
That's a great point. The way I ran it, it just resets each team's true strength every season, which is really not correct at all -- in real life, there's obviously a lot of carry-over between years.
Wouldn't the simulation still create #1 seeds that aren't necessarily the best team in the conference - just like happens in real life?
Yes, it's disturbingly frequent in fact. And now that I think about it, I'm not sure, in a 15- or 19-year span, would having some yearly carry-over in terms of true strength make any difference in terms of the likelihood of the #1 seed in a conference reaching the SB? It would just be the same team over and over trying to get there, instead of a new random team (with roughly the same strength) vying for the conference title each year. Shouldn't make a difference, right?
I think that it may have to do with a team winning the #1 seed on a tiebreaker in some of those years, meaning that they are probably equal to #2 team. Then you basically have a 1a & 1b team, but it won't show up for the data. If you have that in one or both conferences, then you have the situation where #1 seed vs. #1 seed is less likely.
I think if you looked back at the data, checked the years where the #1 seed was CLEARLY the best team in the conference (as the Colts & Saints are this year), then my guess is that that particular #1 seed probably got to the SB.
Personally, I think the Colts should win their game, and that (as a Saints fan) that the Saints will win their game. I think that this year will help balance the data set.
It would just be the same team over and over trying to get there, instead of a new random team (with roughly the same strength) vying for the conference title each year. Shouldn't make a difference, right?
It's the scheduling bias. Imagine if the best teams in football were the Eagles and Cowboys over a 5-year stretch: that's going to depress their record compared to a random schedule because they have to face their hardest opponent twice every year, as opposed to once every year if they're in their conference, or once every *four* years if they're in the other conference.
The other question is whether or not the distribution of SRSs you're using is actually representative of the past 15 years.
Remember Brian Burke's observation that a little bit more than 50% of all NFL games are determined by random chance.
In the playoffs the percentage probably is greater, because in the regular season you get a lot of "good team v really bad team" games, while in the playoffs the quality level of all the teams is in a much narrower band.
When was the last time you heard a commentator talk about this?
If anything they "hero-ize" the best teams and make it seem as like they should be more likely to win.
What's weirder is that from 1994 through 2007, one top seed made it in every year except 1997, but never both top seeds together. The conferences were extremely different in this regard: the NFC put nine #1 seeds into the Superbowl, to the AFC's four.
In the previous fourteen-year period (1980-93), the NFC also sent nine top seeds to the Superbowl -- as did the AFC. Of course, only one of those AFC top seeds actually won the game.
Why should the AFC top seed have failed so frequently in recent times? Is it a Schottenheimer-and-Cowher blip? Or just dumb luck?
Re: #14
Is it possible that the "random chance" factor in determining outcomes of games has increased the more the NFL has become a passing league?
After all, back when NFL offenses relied much more heavily on running games, and on their defenses being able to stop the run first and foremost, it would seem to me that the running game consists of less variance than does the passing game.
In other words, say you're a 1972 powerhouse NFL team, like the Dolphins, Raiders, or Steelers were that season. These were heavily run-oriented teams. If they would run the ball, say, 44 times, and half of the results had to do with their skill, and half had to do with "chance" (I'm following Burke's breakdown for sake of argument), it just seems like the variations that can happen on a running play are a lot LESS dramatic than the variations that can happen on a passing play.
In other words, let's say this great running 1972 team runs 44 times a game. On half of those carries, their skill reliably gets them 6 yards a carry. On the other half, chance determines the performance - in any given week, those other 22 carries may get 2 yards a carry, or 6, or 8, or they may lose 2 fumbles on those 22 carries and none the following week.
All that seems a lot less dramatic than the variation that I think intuitively exists in the passing game. In the passing game, those half of all plays that are the result of chance may result in interceptions, sacks, incompletions on critical 3rd downs, and the like.
Added to this is my intuition that fewer running plays are the result of chance than passing plays, and I think another reason why the playoffs haven't conformed to seeding very often has to do with this factor.
Oh, and a couple of other big factors: free agency.
In fact, free agency almost exactly corresponds to the changes in this trend, doesn't it? I think free agency has meant a couple of hugely significant things for the playoffs:
1.) Injuries have a more disproportionate impact, as teams have less experienced and lower quality depth than great teams in the 60s and 70s had. The '72 Dolphins went unbeaten with their backup QB, after all.
2.) Special Teams. How many playoff games have been lost by higher seeded teams in significant part because their special teams gave up big plays? I think there's greater variance in Special Teams performance, and in a one-and-done playoff format, special teams screwups can have a disproportionate impact.
As with everything else in postmodern society, this is subject to a swinging pendulum. Five years ago, with the Patriots being the de-facto #1 seed in the AFC, any diversion from this would have been considered beneficial. Now we're in a time where we enjoy #1-#1 matchups for the sake of rationality.
Continuing on the 2004 Patriots... I guess I agree on the seeds-not-quite-reflecting-strength theory.
Counting the Colts and Saints, there have been 88 Super Bowl participants.
59 of those participants finished tied for their conference lead in W-L record.
14 were within 1 game of their conference leader in W-L record.
15 teams were more than 1 game behind their conference leader.
Of the 15 teams that were 1+ games behind:
1967 Packers (2.5 games)
1969 Chiefs (1.5)
1970 Cowboys (2)
1974 Steelers (1.5)
1975 Cowboys (2)
1979 Rams (2)
1987 Redskins (2)
1988 49ers (2)
1995 Steelers (2)
1996 Patriots (2)
2001 Patriots (2)
2005 Steelers (3)
2006 Colts (2)
2007 Giants (3)
2008 Cardinals (3)
So the 1970's had 4 teams that were 1+ games behind the leader and the 2000's had 5 teams that were 1+ games behind the leader.
I was rating teams based on "games behind" because it's a way to reduce the nature of the arbitrary tiebreaker system. For instance, the 1980 Oakland Raiders are credited as the first Wild Card team to win a Super Bowl. And their path to the Super Bowl was more difficult because of their seeding. But they actually tied with 4 other AFC teams for the best record at 11-5 that year. So while tiebreakers may have classified them as a wild card, they may have actually been equally as good as the #1 seed.
To me, these numbers show that the 2000's might have had a slight increase in weaker teams making the Super Bowl, but all it takes is the 2007 Packers to beat the 2007 Giants and the trend changes quite a bit. 75% of this decade's Super Bowl participants were within 1 game of their conference leader's W-L record.
Re #16, I can think of another reason why an increased focus on passing would increase the number of luck dependent game outcomes. Passing plays use up less clock on average, thus resulting in more total plays each game.
P.S. Is there a reason why incomplete passes stop the clock and both complete passes and runs with no gain keep the clock running?
Re My own PS might it be at all similar to the icing rule in hockey?