## Manning index updated

Posted by Doug on January 24, 2007

I originally posted this two years ago this week and now seems like a good time for an update.

The idea is to put every quarterback's postseason record into something approximating an appropriate context. If a QB takes his 9-7 team on the road and loses to a 13-3 team, that doesn't count against him much --- nor does it count too much *for* the opposing quarterback --- because the 9-7 team has no business winning that game anyway.

In the original article, I wrote this, and it's no less relevant now:

Just to be clear, I believe that teams — not quarterbacks — win football games, so I’m not claiming this is the One True Measure Of Clutchness. Whether I like it or not though, wins are credited to quarterbacks in virtually every discussion about quarterback greatness. This is merely a way of putting a quarterback’s win-loss record into perspective.

The extent to which a team should be expected to win is given by this formula, which was the result of a regression:

Probability of winning = (1 + exp(-.43(windiff)-.24(homefield)))^(-1)

where **windiff** = the given team’s regular season wins minus its opponents’ regular season wins (actually, it's the regular season winning percentage differences multiplied by 16) and **homefield** = 1 if home, -1 if road, 0 if neutral site.

In the original article, I only considered quarterbacks who debuted in 1978 or later. My postseason database now goes back a little further, so I'll include all quarterbacks who debuted in 1972 or later and played in at least eight postseason games (and I'll throw in Terry Bradshaw, all of whose postseason appearances happened in 1972 or later). The quarterbacks are ranked by (an approximation of) the probability that an average quarterback would compile the given record (or better) by random chance.

Coach Expected Actual DIFF

===============================================

Tom Brady 7- 7 12- 2 +4.9

Terry Bradshaw 11- 8 14- 5 +3.1

Mark Rypien 3- 5 5- 3 +1.6

Joe Montana 14- 9 16- 7 +2.3

Troy Aikman 9- 7 11- 5 +1.8

John Elway 12-10 14- 8 +1.8

Phil Simms 5- 5 6- 4 +0.6

Brett Favre 10-10 11- 9 +0.8

Mark Brunell 4- 6 4- 6 +0.5

Rich Gannon 4- 4 4- 4 +0.1

Donovan McNabb 7- 5 7- 5 +0.0

Danny White 6- 5 6- 5 -0.2

Jim Kelly 9- 7 9- 7 -0.3

Steve McNair 5- 5 5- 5 -0.3

Peyton Manning 6- 6 6- 6 -0.3

Dave Krieg 3- 6 3- 6 -0.4

Joe Theismann 7- 1 6- 2 -0.6

Steve Young 9- 5 8- 6 -1.1

Dan Marino 9- 9 8-10 -1.4

Randall Cunningham 4- 6 3- 7 -1.2

Ron Jaworski 6- 3 4- 5 -1.7

Warren Moon 5- 5 3- 7 -1.9

[NOTE: records are rounded because records just don't look right if not rounded.]

Because they necessarily include at least three wins, Super Bowl titles are rewarded heavily in this system. But it's interesting to compare the quarterbacks with three or more rings. Brady ranks far ahead of Aikman, Bradshaw, and Montana because more often than not, those three were playing in games they should have been expected to win. If Tom Brady's teams are 14-2 and lose their first playoff game at home to a 9-7 team in each of the next two seasons, he'll probably still be at the top of this list. [Technical note: I've counted a QB as having played a game if he had 10 or more passing attempts. This gives full credit to Brady (and to Bledsoe) for the AFC Championship game win over Pittsburgh in 2001. If you don't think that's appropriate, subtract some portion of the .75 wins above expected that this system gives Brady credit for.]

Meanwhile Peyton Manning can climb into the positive numbers for the first time in his career with a win over the Bears next Sunday.

Random question, does anyone know the percentage of starters in the NFL that are 1st round, 2nd round, 3rd round, 2nd Day or undrafted currently and how it compares to 5,10,20 years ago?

IE, is the increase in scouting and analyzing actually working?

Brady's high rating is largely a product of their improbable 2001 championship run...

Indeed it is David. About 1.9 of Brady's wins above expected are from 2001. Without 2001, he's still at or very near the top of the list. There's just not as much distance between him and the pack.

Doug,

I'm trying your formula, and it doesn't seem right. For example, assume a 1-win difference between teams, on a neutral site. The better team will win .606 times.

If I use the Odds Ratio method, an 11-5 team facing a 10-6 team will win .569 times. 10/6 v 9-7 is .565. 12/4 v 11/5 is .577.

In order to get to .606 for a 1-win differential between teams, the sample 11-5 team needs to be treated as a true 10-6 team, while the sample 10-6 team needs to be treated as a true 8.3-7.7 team.

Unless of course the Odds Ratio method doesn't work for the NFL.

tango, the formula was generated from a regression, so it's "right" in the sense that it's the best fit to the data set used to generate it (which IIRC was all postseason games from 1978 through 2004). A different data set might yield a different formula.

That said, I think the differences we're talking about here are small enough that they're not going to alter the list in any meaningful way.

Warren Moon! LOL 🙂

Best-fit, using around, what, 200 contests? The uncertainty level must be fairly high I would think, especially considering that most contests must be from teams with a 1-win or tied differential?

I agree that the results won't alter the list in question in a meaningful way.

For all QB's on this list, I divided the playoff games played into two lists. I compared the playoff games played in the first half vs. those played in the second half, using the above formula. If there were an odd number of games played, I added the extra game to the first half.

The correlation coefficients between the actual vs expected win difference for the first half of the playoff career, and the second half of the playoff career, is +0.057. If you go by rate (Actual -Expected)/# of games, then the correlation coefficient between 1st half and 2nd half is -0.040. Either way, no significant correlation.

Interestingly, by this measure of playoff performance, 13 of the 22 were better over the first half of games played, 6 were better over the second half of games played, and 3 performed equally over both periods.

Final note, in looking at the playoff logs, Steve Young is unfairly penalized for throwing more than 10 pass attempts in the 1987 loss to Minnesota at home (13-2 vs 8-7). Montana threw more passes that game, and was the starter, so Young got mop up duty in the loss. Take that game out when he was not the starter and did not throw the most attempts in the game, and Young moves up the list and is closer to zero.

The other issue is the HFA. When the differential is zero, you have the home team (in the playoffs) winning .560. IIRC, the regular season HFA would be .540.

Assuming the Odds Ratio method works, and the HFA in the regular season matches that in the post-season, then Doug's equation works best if the .43 and .24 coefficents are replaced by .27 and .16, respectively.

I ran the numbers for Bart Starr, 9-1 all time in the post-season. By my count he's +4.14. I'd also like to note that it's ludicrous that the 9-4-1 Packers hosted the 11-1-2 L.A. Rams in the first round of the playoffs, while the Baltimore Colts -- who did not lose in their first 13 games -- missed the playoffs entirely.

Does this list include all of the QBs who have played at least 8 games since 1972? What happens if you expand the list to include QBs who started, say, a least 5 games?

Andrew, what happens is you get guys named Dilfer, Roethlisberger, and Delhomme near the top of the list.

"Andrew, what happens is you get guys named Dilfer, Roethlisberger, and Delhomme near the top of the list."

Is that necessarily a bad thing?

Not necesarily. But I do know that

somepeople definitely do think it's a bad thing and will email me to let me know.Because I am weak and lazy, I'd rather change the cutoff than try to explain why my list of clutch QBs has Delhomme ahead of Montana.

"Is that necessarily a bad thing?"

Given that the list is supposed to give an approximation of who the best playoff QBs are, I'd say yes, that is a bad thing. Those 3 are all 5-1 or 5-2, but a person would have to be crazy to say that they are top tier playoff QBs. Let them prove themselves one more time (which obviously won't happen in Dilfer's case) before we see where they rank.

If you expand the list to QBs who started 5 playoff games, Jeff Hostetler has to be there, too. He's the first guy to pop into my head based on that criteria.

"Given that the list is supposed to give an approximation of who the best playoff QBs are, I’d say yes, that is a bad thing. Those 3 are all 5-1 or 5-2, but a person would have to be crazy to say that they are top tier playoff QBs."

Funny, I thought that the whole point of this exercise was to put a QB's record in proper perspective, to point out cases where popular opinion may be wrong. But now that I know that it was just meant to reconfirm all of our preconceived notions regarding success in the playoffs, by all means, omit any data points that might disagree with them.