SITE NEWS: We are moving all of our site and company news into a single blog for Sports-Reference.com. We'll tag all PFR content, so you can quickly and easily find the content you want.

Also, our existing PFR blog rss feed will be redirected to the new site's feed.

Pro-Football-Reference.com ยป Sports Reference

For more from Chase and Jason, check out their work at Football Perspective and The Big Lead.

Adjusting QB win-loss records, part II

Posted by Doug on March 30, 2009

In part one I attempted to adjust quarterback win-loss records for the differing levels of defensive support that each quarterback got. This post is an attempt to refine that idea.

Let me start by reiterating a couple of things I said in the previous post, but probably wasn't clear enough about.

Quarterbacks do not have win-loss records. Teams have win-loss records. Quarterbacks are big parts --- the single biggest individual part in most cases --- of teams, but they are not the same thing as teams. They therefore do not have win-loss records. Wins and losses are determined by (1) the quarterback, (2) the other offensive players, (3) the defense, and (4) the special teams (and also coaching, to the extent that it influences the three things above). So when people talk about "John Elway's record," they're really talking about something that was influenced by all four of the above factors in the games that Elway started.

The last post and this one are an effort to remove factors (3) and (4) as much as possible from the computation. What would "Elway's record" have been if he had average contributions from the defense and the special teams? (And, as discussed last time, understanding exactly how Elway himself might have contributed to (3) complicates things further.) These posts make no attempt to estimate what "Elway's record" would have been if he had had Jerry Rice to throw to in the 80s, or if he hadn't had Terrell Davis to hand off to in the 90s.

In order to keep this from reading like a software license agreement, though, I'm going to use terms like "Elway's record..." or "Elway won..." or "Elway was X games above average." These statements mean, respectively, "The teams quarterbacked by Elway had a record of ....", "the teams quarterbacked by Elway won....", and "The OFFENSES quarterbacked by Elway were X games above average."

And even with all those disclaimers out of the way, I'm not really sure how much I like the rankings from part one or the rankings I'm going to develop here. The only thing I'm sure of is that I like them better than regular win-loss records. I'm not making any claims beyond that.

OK, let's get on with it.

In part one, I essentially reasoned like this:

On November 5th, 1967, Bob Griese started at quarterback for the Dolphins, and Miami gave up 35 points to the Bills. AFL teams from 1965 to 1969 won 3.6% of the games in which they gave up 34 or more points. So Griese would get credit for .964 "wins above average" (that's 1 - .036) if he is able to lead his team to a win despite the poor defensive effort. But he wasn't, and so he gets charged with -0.036 wins above average.

So the basic idea is: given the points allowed, estimate the chances of winning. Adding up those win probabilities across all games gives a reasonable expectation for the number of games the quarterback "should have won." Comparing his actual wins to his expected wins is a measure of how good he was.

We're going to do the same exact thing here, but we're going to take into account some other factors when we estimate the chances of winning.

First and most obviously, giving up 34 points isn't the same as giving up 54 points, but in the previous analysis we lumped them together. In part one, games of 16 points and 20 points were treated the same. If one quarterback just happened to have a lot more of one than the other, his rating would be a little skewed. It probably evens out in the vast majority of cases, but it would be nice to be a little more precise if possible.

Second, if you look at the boxscore linked above, you'll note that the Dolphins' defense didn't really give up 35 points in that game. Griese's offense was directly responsible for seven of those points via an interception thrown by Griese and returned by Buffalo's Tom Janik for a touchdown. When evaluating Griese's effort, we really should be treating that as a 28-points-allowed game rather than a 35-points-allowed game.

Next, winning a game when your defense gives up 28 points to the Bills, an average defensive team, is tougher than winning a game when your defense allows 28 to the 1967 Broncos, a terrible defensive team, but easier than if your team gives up 28 to a good defensive team like the 1967 Oilers. Again, this is something that probably evens out in the long haul for most players, but we may as well adjust for it if we can.

Now let's talk about points scored by the quarterback's team, but that the quarterback couldn't have had anything to do with. Consider this 1993 game between the Browns (with Vinny Testaverde starting at quarterback) and the Steelers. The Steelers scored 23 points, so this appears to be a relatively high degree of difficulty game for Vinny, and the Browns did in fact win it. But if you look at the box you'll notice that Testaverde had a huge assist in the form of two punt return touchdowns by Eric Metcalf. Just as it's easier to win if your defense plays well, it's easier to win if your returnerguys chip in with a score or two.

The same could also be said of interception returns or fumble returns, but I'm not going to include those because of the potential for correlation/causation confusion. To be more specific, teams that get an interception return TD win 76% of the time. Part of that is because seven points helps. But it doesn't help that much. The other part is that teams who are already losing are much more likely to throw a pass that results in an interception for a score. So when the Colts return a pick for a touchdown and win the game, I think it's as likely --- possibly more so --- that the Colts' offense (led by Manning) was responsible for that pick six than that the pick six was responsible for the win. In other words, I think that in most cases, the quarterback and his offense are largely responsible for interception return TDs. But I don't think the same is true for kick and punt return TDs. Punt returns are correlated with good defense and kick returns are correlated with bad defense, but since we're already controlling for the defensive performance (via points allowed), I think that they do in fact add significant information about how easy or hard it would be for a quarterback to lead his team to a win in a given game. Fumble returns I'm really not sure about, but I've decided to leave them out.

So kick and punt return TDs are in the model, fumble and interception returns are out. I think I'm making the right decisions there, but I'm willing to be convinced otherwise.

One last thing: it was suggested in the comments to part one that weather plays a role in determining how difficult it is to score a given number of points, and that's true. Someday p-f-r will have weather information for every game in history, but right now we don't. But I can put a marker into the model that flags games played in "cold weather cities" in December or January.

And that's it. Let's recap:

For every team-game since 1950, we record the following data:

1. the starting quarterback

2. the number of points scored by the opposing team MINUS seven times the number of interception return scores the opposing team had.

3. the average points allowed by the opponent in all their other games that season. Note that this accomplishes two things at once. First, it adjusts for strength of defense faced. But it also adjusts for era. For a fixed number of points allowed, it doesn't really matter whether you're playing a good defensive team in a high-scoring era or a bad defensive team in a low-scoring era. All that matters is how many points that team tends to allow.

4. the number of kick and punt return touchdowns scored by your team.

5. a dummy variable that's 1 if the game was played in a cold weather city in December or January, and 0 otherwise.

6. whether that team won or lost the game.

Then I ran a logit regression with the first five variables as inputs and the sixth as the output. Here is the formula I got (for regression mavens, there is a bit of fine print at the end):

Win Probability =~ 1 / (1 + exp(-(1.76 - .2455*PA + .00183*PA^2 + .1085*OPPDEF + 1.038*RETTD - .049*COLD + QB_SCORE)))

where

PA = points allowed minus seven times interception return TDs
OPPDEF = average points allowed by the opposing defense in their other games that season.
RETTD = number of kick/punt return TDs scored by the given team.
COLD = 1 if Dec/Jan in a cold-weather city, 0 otherwise
QB_SCORE = .... read on

I said we had five input variables, but we actually have a couple of hundred: the four named above (PA, OPPDEF, RETTD, and COLD), plus one variable for every quarterback who started at least 50 games in his career. For a given team-game, exactly one of those quarterback variables will be a one, and all the rest will be zeroes. The regression process gives us a coefficient for each of those variables --- that is, it gives us a number for each quarterback --- that tells us how much, on average, that quarterback contributed to his team's chances of winning their games.

Good quarterbacks have positive scores and bad ones have negative scores. +1.0 is a very good score, that's about what Otto Graham is. +.5 is still pretty good. Brett Favre is about +.55, for instance. Mark Brunell, Elvis Grbac, and Rodney Peete are all around zero. Former Giant Dave Brown is about a -0.5, and David Carr is roughly -1.0.

To get a feel for what these numbers mean, take a look at the other coefficients. The coefficient on RETTD, for example, is about 1, which is about the same as the difference between Rodney Peete and David Carr. If you're playing a given team and you know that you will allow, say 23 points, would you rather have Rodney Peete at quarterback, or would you rather have David Carr, but be guaranteed that your special teams would kick in with a kick or punt return score? The formula says that's a pretty tough choice.

The formula says the difference between giving up 17 points and giving up 24 points is roughly equivalent to the difference between two quarterbacks whose scores differ by 1.2 (Peyton Manning and Steve Bartkowski, for example, or Jim Kelly and Tim Couch). If you think that sounds too drastic, consider: teams that allow 17 points win about 63% of their games while teams that allow 24 points win 39%.

Another example: the 1985 Bears allowed about 20 points per game less than the 2008 Lions. The coefficient on OPPDEF is .1085. Multiply that by 20 and you get about 2. Given the same defense and special teams, would you rather play against the 1985 Bears with one of the greatest quarterbacks in history --- say Otto Graham --- or against the 2008 Lions with David Carr? The formula says it's a tough call.

What's interesting is that the formula captures something we all know: that how important the QB/offense is depends on what kind of game it is. Here is a sampling of different kinds of games, along with the chances of an average offense winning that game compared to the chances of a Favre-quarterbacked offense winning it:

Game conditions           Average Offense    Favre-QBed Offense
================================================================
allow 3 points,                  96.1%            97.7%
   average opp defense

allow 38 points,                  6.0%             9.9%
   average opp defense

allow 20 points,                 54.6%            67.6%
   weak (24) opp defense

allow 20 points,                 33.6%            46.7%
   good (16) opp defense

In the first two situations, having a good quarterback/offense instead of an average one doesn't buy you much. In the second two, it makes a pretty big difference. By the same reasoning, the quality of the quarterback is less important to teams with great defenses and teams with horrible defenses than it is to teams with average defenses. If you run the 2002 Bucs games, for instance, through the formula with the Favre score (0.55) and with an average score (0.00), here's what you get:

2002 Bucs defense with Favre-quarterbacked offense: 13.44 expected wins
2002 Bucs defense with average offense: 12.38 expected wins
Difference: 1.06 games

Now with a fairly average defensive team, say the 2008 Bills

2008 Bills defense with Favre-quarterbacked offense: 10.12 expected wins
2008 Bills defense with average offense: 8.57 expected wins
Difference: 1.55 games

Enough chatter. What you've all been waiting for is the list. If you run the formula with a zero QB_SCORE for the particular games of every quarterback, you will get the number of wins he "should have had." Comparing that to the number of wins he actually had is his Wins Above Average. All that work and what do we end up with? A list that, at least at the top, looks nearly identical to the list from part one. Here it is.

Like last time, this includes all quarterbacks with 50 or more starts since 1950, and includes all postseason and regular season games since 1950.

                 WinsAdded   rnk
================================
Peyton Manning       +35.2 (  1)
John Elway           +30.7 (  2)
Tom Brady            +27.0 (  3)
Brett Favre          +25.7 (  4)
Dan Marino           +25.4 (  5)
Joe Montana          +23.9 (  6)
Daryle Lamonica      +19.2 (  7)
Ken Stabler          +18.9 (  8 )
Jim Kelly            +17.8 (  9)
Steve Young          +17.0 ( 10)
Roger Staubach       +16.1 ( 11)
Randall Cunningham   +15.5 ( 12)
Johnny Unitas        +15.0 ( 13)
Dan Fouts            +14.9 ( 14)
Fran Tarkenton       +14.8 ( 15)
Norm Van Brocklin    +14.8 ( 16)
Y.A. Tittle          +14.6 ( 17)
Danny White          +13.7 ( 18)
Terry Bradshaw       +12.9 ( 19)
Joe Theismann         +9.8 ( 20)
Kurt Warner           +9.7 ( 21)
Otto Graham           +9.6 ( 22)
Rich Gannon           +9.4 ( 23)
Ben Roethlisberger    +9.3 ( 24)
Steve McNair          +9.2 ( 25)
Trent Green           +8.8 ( 26)
Jeff Hostetler        +8.5 ( 27)
Jay Schroeder         +8.5 ( 28)
Frank Ryan            +8.4 ( 29)
Matt Hasselbeck       +8.3 ( 30)
Drew Brees            +8.1 ( 31)
Bobby Layne           +8.1 ( 32)
Eli Manning           +8.0 ( 33)
Philip Rivers         +7.8 ( 34)
Dave Krieg            +7.8 ( 35)
Stan Humphries        +7.7 ( 36)
John Brodie           +7.6 ( 37)
Bill Nelsen           +7.0 ( 38)
Jake Delhomme         +6.9 ( 39)
Marc Bulger           +6.6 ( 40)
Mark Rypien           +6.4 ( 41)
Donovan McNabb        +6.2 ( 42)
Jim Hart              +6.0 ( 43)
Neil Lomax            +5.7 ( 44)
Len Dawson            +5.6 ( 45)
Warren Moon           +5.5 ( 46)
Charley Johnson       +5.4 ( 47)
Daunte Culpepper      +5.3 ( 48)
Boomer Esiason        +5.1 ( 49)
Bert Jones            +5.0 ( 50)
Tony Eason            +5.0 ( 51)
Joe Namath            +5.0 ( 52)
Bart Starr            +4.8 ( 53)
Phil Simms            +4.6 ( 54)
Steve Grogan          +4.5 ( 55)
Troy Aikman           +4.4 ( 56)
Jack Kemp             +4.3 ( 57)
Ed Brown              +4.0 ( 58)
Michael Vick          +4.0 ( 59)
Dan Pastorini         +3.9 ( 60)
Brian Sipe            +3.9 ( 61)
Jim McMahon           +3.5 ( 62)
Carson Palmer         +3.4 ( 63)
Pat Haden             +3.2 ( 64)
Jim Zorn              +3.0 ( 65)
Roman Gabriel         +2.8 ( 66)
Jake Plummer          +2.7 ( 67)
Jim Plunkett          +2.4 ( 68)
Don Meredith          +2.4 ( 69)
Wade Wilson           +2.3 ( 70)
Marc Wilson           +2.3 ( 71)
Neil O'Donnell        +2.2 ( 72)
Bobby Hebert          +2.2 ( 73)
David Woodley         +2.2 ( 74)
Tom Flores            +2.1 ( 75)
Tommy Kramer          +2.0 ( 76)
George Blanda         +1.7 ( 77)
Jeff Garcia           +1.7 ( 78)
Tobin Rote            +1.6 ( 79)
John Hadl             +1.6 ( 80)
Brad Johnson          +1.4 ( 81)
Billy Wade            +1.4 ( 82)
Billy Kilmer          +1.3 ( 83)
Gus Frerotte          +1.3 ( 84)
Aaron Brooks          +1.1 ( 85)
Kordell Stewart       +1.1 ( 86)
Bob Griese            +1.1 ( 87)
Earl Morrall          +0.9 ( 88)
Erik Kramer           +0.9 ( 89)
Mike Phipps           +0.7 ( 90)
Doug Flutie           +0.7 ( 91)
Mike Tomczak          +0.4 ( 92)
Sonny Jurgensen       +0.3 ( 93)
Elvis Grbac           +0.1 ( 94)
Milt Plum             +0.1 ( 95)
Scott Mitchell        -0.0 ( 96)
Rodney Peete          -0.0 ( 97)
Vince Ferragamo       -0.1 ( 98)
Charlie Conerly       -0.2 ( 99)
Mark Brunell          -0.3 (100)
Steve Bartkowski      -0.4 (101)
Babe Parilli          -0.5 (102)
Richard Todd          -0.6 (103)
Brian Griese          -0.6 (104)
Ken Anderson          -0.6 (105)
Jeff Blake            -0.6 (106)
Ken O'Brien           -0.8 (107)
Bernie Kosar          -1.1 (108)
Chad Pennington       -1.3 (109)
Jay Fiedler           -1.5 (110)
Jon Kitna             -1.5 (111)
Gary Danielson        -2.2 (112)
Don Majkowski         -2.3 (113)
Bubby Brister         -2.4 (114)
Bob Avellini          -2.5 (115)
Joe Kapp              -2.9 (116)
Greg Landry           -2.9 (117)
Eric Hipple           -3.0 (118)
Jim Everett           -3.4 (119)
Drew Bledsoe          -3.5 (120)
Lynn Dickey           -3.9 (121)
Craig Morton          -4.0 (122)
Doug Williams         -4.1 (123)
Mike Livingston       -4.1 (124)
Bill Munson           -4.2 (125)
Kerry Collins         -4.6 (126)
Chris Chandler        -4.7 (127)
Chris Miller          -4.7 (128)
Bill Kenney           -4.8 (129)
Lamar McHan           -4.9 (130)
Mark Malone           -5.1 (131)
Jeff George           -5.2 (132)
Dave M. Brown         -5.3 (133)
Eddie LeBaron         -5.3 (134)
Mike Pagel            -5.5 (135)
Tim Couch             -5.5 (136)
Bob Berry             -5.6 (137)
Tony Banks            -5.9 (138)
Cotton Davidson       -6.0 (139)
Rick Mirer            -6.2 (140)
Joe Ferguson          -6.9 (141)
Bobby Douglass        -6.9 (142)
Steve DeBerg          -7.0 (143)
Steve Beuerlein       -7.0 (144)
Ron Jaworski          -8.3 (145)
Jim Harbaugh          -8.5 (146)
Trent Dilfer          -9.9 (147)
Vinny Testaverde     -10.6 (148)
David Carr           -10.8 (149)
Joey Harrington      -12.4 (150)
Norm Snead           -14.3 (151)
Archie Manning       -15.8 (152)

Here's the same list of quarterbacks, ordered by their QB_SCORE, which is the number the regression spit out for each guy. The adjusted win pct column is computed as .500 + WinsAdded/Games. It is essentially a per-game-ified version of the above list, and it should track the QB_SCORE column closely, but not exactly. The final set of columns --- raw win pct --- show the player's raw winning percentage, for comparison purposes. Among this list of 152 quarterbacks, Jim McMahon has the 8th-highest winning percentage, but the 59th-highest adjusted winning percentage, and the 53rd-highest QB_SCORE. I'll have a little more to say about this after the list:

                                 ------- win pct --------
                  QB_SCORE  rnk   adjusted       raw
==========================================================
Tom Brady            +1.48 (  1)  0.711 (  1)  0.789 (  2)
Daryle Lamonica      +1.35 (  2)  0.711 (  2)  0.769 (  3)
Peyton Manning       +1.12 (  3)  0.684 (  3)  0.649 ( 15)
Otto Graham          +0.97 (  4)  0.625 (  8 )  0.792 (  1)
Philip Rivers        +0.91 (  5)  0.645 (  5)  0.667 ( 10)
Norm Van Brocklin    +0.89 (  6)  0.647 (  4)  0.624 ( 25)
Danny White          +0.88 (  7)  0.634 (  6)  0.657 ( 11)
Roger Staubach       +0.88 (  8 )  0.623 (  9)  0.733 (  4)
Ben Roethlisberger   +0.86 (  9)  0.614 ( 12)  0.728 (  5)
Joe Montana          +0.83 ( 10)  0.628 (  7)  0.711 (  6)
John Elway           +0.78 ( 11)  0.622 ( 10)  0.645 ( 17)
Ken Stabler          +0.77 ( 12)  0.620 ( 11)  0.656 ( 12)
Y.A. Tittle          +0.72 ( 13)  0.609 ( 14)  0.582 ( 44)
Steve Young          +0.71 ( 14)  0.608 ( 15)  0.650 ( 14)
Frank Ryan           +0.68 ( 15)  0.596 ( 20)  0.667 (  9)
Dan Marino           +0.66 ( 16)  0.599 ( 18)  0.601 ( 37)
Eli Manning          +0.63 ( 17)  0.602 ( 16)  0.590 ( 41)
Randall Cunningham   +0.62 ( 18)  0.609 ( 13)  0.594 ( 39)
Jim Kelly            +0.62 ( 19)  0.600 ( 17)  0.621 ( 28)
Jeff Hostetler       +0.58 ( 20)  0.597 ( 19)  0.625 ( 24)
Tony Eason           +0.57 ( 21)  0.589 ( 22)  0.554 ( 58)
Stan Humphries       +0.57 ( 22)  0.588 ( 24)  0.609 ( 35)
Dan Fouts            +0.57 ( 23)  0.584 ( 26)  0.503 ( 86)
Brett Favre          +0.55 ( 24)  0.588 ( 23)  0.622 ( 27)
Kurt Warner          +0.55 ( 25)  0.586 ( 25)  0.580 ( 46)
Jay Schroeder        +0.54 ( 26)  0.582 ( 27)  0.615 ( 32)
Bill Nelsen          +0.54 ( 27)  0.592 ( 21)  0.553 ( 59)
Johnny Unitas        +0.53 ( 28)  0.579 ( 28)  0.653 ( 13)
Terry Bradshaw       +0.51 ( 29)  0.573 ( 36)  0.684 (  7)
Joe Theismann        +0.50 ( 30)  0.574 ( 33)  0.629 ( 22)
Jake Delhomme        +0.48 ( 31)  0.578 ( 29)  0.618 ( 30)
Mark Rypien          +0.48 ( 32)  0.576 ( 31)  0.612 ( 34)
Trent Green          +0.48 ( 33)  0.577 ( 30)  0.487 ( 98)
Matt Hasselbeck      +0.47 ( 34)  0.574 ( 35)  0.554 ( 57)
Marc Bulger          +0.47 ( 35)  0.574 ( 34)  0.456 (117)
Drew Brees           +0.44 ( 36)  0.575 ( 32)  0.514 ( 78)
Fran Tarkenton       +0.40 ( 37)  0.561 ( 38)  0.533 ( 67)
Rich Gannon          +0.40 ( 38)  0.568 ( 37)  0.576 ( 51)
Michael Vick         +0.39 ( 39)  0.557 ( 41)  0.571 ( 52)
Bobby Layne          +0.39 ( 40)  0.560 ( 39)  0.615 ( 33)
Steve McNair         +0.36 ( 41)  0.557 ( 42)  0.589 ( 42)
Neil Lomax           +0.35 ( 42)  0.557 ( 40)  0.470 (107)
Carson Palmer        +0.35 ( 43)  0.552 ( 45)  0.485 (100)
Pat Haden            +0.33 ( 44)  0.555 ( 43)  0.627 ( 23)
Bert Jones           +0.32 ( 45)  0.551 ( 46)  0.475 (106)
Daunte Culpepper     +0.32 ( 46)  0.553 ( 44)  0.434 (128)
John Brodie          +0.30 ( 47)  0.548 ( 47)  0.487 ( 97)
Charley Johnson      +0.29 ( 48)  0.546 ( 48)  0.509 ( 82)
Donovan McNabb       +0.28 ( 49)  0.544 ( 49)  0.641 ( 18)
Ed Brown             +0.27 ( 50)  0.543 ( 50)  0.591 ( 40)
Jack Kemp            +0.26 ( 51)  0.540 ( 52)  0.620 ( 29)
Joe Namath           +0.26 ( 52)  0.539 ( 53)  0.500 ( 87)
Jim McMahon          +0.25 ( 53)  0.534 ( 59)  0.680 (  8 )
David Woodley        +0.25 ( 54)  0.538 ( 54)  0.649 ( 16)
Dave Krieg           +0.25 ( 55)  0.542 ( 51)  0.549 ( 60)
Len Dawson           +0.25 ( 56)  0.535 ( 56)  0.623 ( 26)
Marc Wilson          +0.23 ( 57)  0.537 ( 55)  0.525 ( 71)
Tom Flores           +0.22 ( 58)  0.533 ( 60)  0.492 ( 91)
Brian Sipe           +0.22 ( 59)  0.535 ( 57)  0.504 ( 85)
Dan Pastorini        +0.21 ( 60)  0.532 ( 62)  0.484 (101)
Steve Grogan         +0.21 ( 61)  0.533 ( 61)  0.543 ( 62)
Jim Hart             +0.20 ( 62)  0.534 ( 58)  0.492 ( 93)
Bart Starr           +0.20 ( 63)  0.530 ( 64)  0.640 ( 19)
Wade Wilson          +0.20 ( 64)  0.531 ( 63)  0.514 ( 79)
Jim Zorn             +0.18 ( 65)  0.528 ( 66)  0.415 (136)
Don Meredith         +0.18 ( 66)  0.528 ( 67)  0.576 ( 49)
Phil Simms           +0.18 ( 67)  0.527 ( 68)  0.598 ( 38)
Boomer Esiason       +0.17 ( 68)  0.529 ( 65)  0.466 (110)
Warren Moon          +0.16 ( 69)  0.526 ( 69)  0.493 ( 90)
Troy Aikman          +0.15 ( 70)  0.525 ( 70)  0.583 ( 43)
Bobby Hebert         +0.15 ( 71)  0.521 ( 71)  0.544 ( 61)
Roman Gabriel        +0.13 ( 72)  0.519 ( 74)  0.566 ( 53)
Billy Wade           +0.13 ( 73)  0.517 ( 76)  0.488 ( 96)
Neil O'Donnell       +0.13 ( 74)  0.521 ( 72)  0.542 ( 65)
Jake Plummer         +0.12 ( 75)  0.519 ( 73)  0.500 ( 88)
Tommy Kramer         +0.12 ( 76)  0.518 ( 75)  0.491 ( 95)
George Blanda        +0.11 ( 77)  0.516 ( 77)  0.514 ( 77)
Jim Plunkett         +0.11 ( 78)  0.516 ( 78)  0.519 ( 73)
Jeff Garcia          +0.10 ( 79)  0.514 ( 79)  0.492 ( 92)
Tobin Rote           +0.08 ( 80)  0.514 ( 81)  0.443 (123)
Mike Phipps          +0.08 ( 81)  0.510 ( 87)  0.535 ( 66)
Kordell Stewart      +0.08 ( 82)  0.513 ( 82)  0.581 ( 45)
Gus Frerotte         +0.08 ( 83)  0.514 ( 80)  0.479 (105)
Erik Kramer          +0.08 ( 84)  0.512 ( 84)  0.457 (116)
Aaron Brooks         +0.07 ( 85)  0.512 ( 83)  0.424 (131)
Billy Kilmer         +0.07 ( 86)  0.511 ( 85)  0.525 ( 70)
Earl Morrall         +0.07 ( 87)  0.509 ( 90)  0.638 ( 20)
Brad Johnson         +0.07 ( 88)  0.511 ( 86)  0.576 ( 50)
John Hadl            +0.07 ( 89)  0.510 ( 89)  0.512 ( 81)
Doug Flutie          +0.06 ( 90)  0.510 ( 88)  0.559 ( 56)
Bob Griese           +0.05 ( 91)  0.507 ( 91)  0.616 ( 31)
Mike Tomczak         +0.04 ( 92)  0.505 ( 92)  0.577 ( 48)
Sonny Jurgensen      +0.01 ( 93)  0.502 ( 93)  0.486 ( 99)
Elvis Grbac          +0.01 ( 94)  0.501 ( 94)  0.562 ( 55)
Milt Plum            +0.00 ( 95)  0.501 ( 95)  0.577 ( 47)
Rodney Peete         -0.00 ( 96)  0.500 ( 96)  0.517 ( 75)
Scott Mitchell       -0.00 ( 97)  0.500 ( 97)  0.438 (126)
Vince Ferragamo      -0.01 ( 98)  0.498 ( 98)  0.508 ( 83)
Mark Brunell         -0.01 ( 99)  0.498 ( 99)  0.519 ( 74)
Charlie Conerly      -0.01 (100)  0.498 (100)  0.637 ( 21)
Steve Bartkowski     -0.02 (101)  0.497 (101)  0.458 (114)
Ken Anderson         -0.02 (102)  0.497 (102)  0.522 ( 72)
Richard Todd         -0.03 (103)  0.495 (104)  0.450 (119)
Babe Parilli         -0.03 (104)  0.495 (103)  0.515 ( 76)
Jeff Blake           -0.04 (105)  0.494 (105)  0.390 (139)
Brian Griese         -0.04 (106)  0.493 (107)  0.542 ( 63)
Ken O'Brien          -0.04 (107)  0.493 (106)  0.450 (120)
Bernie Kosar         -0.06 (108)  0.490 (108)  0.491 ( 94)
Jon Kitna            -0.08 (109)  0.487 (109)  0.397 (138)
Chad Pennington      -0.10 (110)  0.484 (110)  0.542 ( 64)
Drew Bledsoe         -0.11 (111)  0.482 (111)  0.508 ( 84)
Jim Everett          -0.14 (112)  0.478 (112)  0.418 (135)
Jay Fiedler          -0.16 (113)  0.476 (113)  0.603 ( 36)
Craig Morton         -0.17 (114)  0.474 (114)  0.562 ( 54)
Kerry Collins        -0.17 (115)  0.473 (115)  0.480 (104)
Chris Chandler       -0.20 (116)  0.470 (116)  0.445 (122)
Bubby Brister        -0.21 (117)  0.469 (118)  0.494 ( 89)
Greg Landry          -0.21 (118)  0.469 (117)  0.458 (113)
Lynn Dickey          -0.23 (119)  0.464 (119)  0.418 (134)
Don Majkowski        -0.23 (120)  0.458 (123)  0.464 (111)
Gary Danielson       -0.23 (121)  0.463 (120)  0.467 (109)
Jeff George          -0.26 (122)  0.459 (122)  0.370 (144)
Joe Ferguson         -0.27 (123)  0.461 (121)  0.457 (115)
Doug Williams        -0.28 (124)  0.453 (124)  0.483 (102)
Chris Miller         -0.30 (125)  0.450 (128)  0.372 (143)
Steve DeBerg         -0.30 (126)  0.451 (126)  0.378 (140)
Vinny Testaverde     -0.30 (127)  0.451 (125)  0.422 (132)
Eric Hipple          -0.31 (128)  0.448 (129)  0.483 (103)
Ron Jaworski         -0.33 (129)  0.445 (130)  0.513 ( 80)
Bob Avellini         -0.35 (130)  0.450 (127)  0.451 (118)
Jim Harbaugh         -0.36 (131)  0.441 (132)  0.469 (108)
Bill Kenney          -0.36 (132)  0.438 (134)  0.442 (125)
Mike Livingston      -0.40 (133)  0.444 (131)  0.419 (133)
Journeymen           -0.42 (134)  0.434 (135)  0.432 (130)
Steve Beuerlein      -0.43 (135)  0.432 (136)  0.462 (112)
Joe Kapp             -0.43 (136)  0.441 (133)  0.531 ( 68)
Eddie LeBaron        -0.46 (137)  0.432 (137)  0.333 (150)
Lamar McHan          -0.48 (138)  0.431 (138)  0.338 (149)
Bill Munson          -0.49 (139)  0.430 (139)  0.443 (124)
Tony Banks           -0.49 (140)  0.425 (140)  0.449 (121)
Dave M. Brown        -0.50 (141)  0.412 (142)  0.433 (129)
Trent Dilfer         -0.51 (142)  0.417 (141)  0.529 ( 69)
Mark Malone          -0.54 (143)  0.408 (144)  0.436 (127)
Tim Couch            -0.55 (144)  0.406 (145)  0.373 (142)
Rick Mirer           -0.56 (145)  0.409 (143)  0.353 (145)
Norm Snead           -0.66 (146)  0.405 (146)  0.344 (146)
Bob Berry            -0.68 (147)  0.386 (150)  0.408 (137)
Mike Pagel           -0.68 (148)  0.396 (147)  0.321 (151)
Replacement level?   -0.73 (149)  0.391 (148)  0.341 (148)
Cotton Davidson      -0.81 (150)  0.387 (149)  0.377 (141)
Archie Manning       -0.82 (151)  0.384 (151)  0.257 (155)
David Carr           -0.90 (152)  0.363 (153)  0.291 (153)
Bobby Douglass       -0.93 (153)  0.367 (152)  0.308 (152)
Joey Harrington      -0.96 (154)  0.336 (154)  0.342 (147)
Emergency QBs        -1.26 (155)  0.325 (155)  0.266 (154)

First note that I've added three "players" in there. The line labeled "Emergency QBs" is a composite of all quarterbacks who had five or fewer starts in their careers. The line labeled "Replacement level?" is a composite of all quarterbacks who started between 6 and 20 games. The name reflects my not-very-carefully-thought-through idea that a typical backup quarterback is about at this level. Finally, "Journeymen" represents all players who started between 21 and 49 games.

The mathematically curious among you might take a minute or two to figure out why the QB_SCORE rankings don't match up with the Adjusted Winning Percentage rankings exactly. If every quarterback faced the same set of circumstances, they would. But take Otto Graham for example. He ranks 4th in QB_SCORE, which means that if all else was equal we'd expect him to have the fourth-highest winning percentage. But the defenses Graham played with were much better than average, so they didn't get the full benefit of Graham's greatness because they didn't need it as much as an average team might. Remember, the value of a good quarterback is maximized on an average team.

Finally, here is the same list of quarterbacks, ranked according to the winning percentage that an average quarterback would have been expected to post with their particular slate of games.

Otto Graham          0.667
Jim McMahon          0.646
Charlie Conerly      0.639
Earl Morrall         0.629
Jay Fiedler          0.627
Ben Roethlisberger   0.614
Trent Dilfer         0.612
David Woodley        0.611
Terry Bradshaw       0.611
Bart Starr           0.610
Roger Staubach       0.610
Bob Griese           0.609
Donovan McNabb       0.597
Joe Kapp             0.590
Craig Morton         0.588
Len Dawson           0.588
Joe Montana          0.583
Jack Kemp            0.581
Tom Brady            0.578
Milt Plum            0.577
Johnny Unitas        0.574
Pat Haden            0.573
Mike Tomczak         0.572
Frank Ryan           0.570
Phil Simms           0.570
Ron Jaworski         0.569
Kordell Stewart      0.568
Brad Johnson         0.565
Elvis Grbac          0.560
Daryle Lamonica      0.559
Troy Aikman          0.559
Chad Pennington      0.558
Bobby Layne          0.555
Joe Theismann        0.555
Brian Griese         0.549
Doug Flutie          0.548
Don Meredith         0.548
Ed Brown             0.548
Roman Gabriel        0.547
Steve Young          0.541
Jake Delhomme        0.540
Mark Rypien          0.536
Ken Stabler          0.536
Eric Hipple          0.535
Jay Schroeder        0.534
Brett Favre          0.534
Steve McNair         0.532
Doug Williams        0.530
Steve Beuerlein      0.529
Mark Malone          0.529
Jeff Hostetler       0.528
Jim Harbaugh         0.528
Ken Anderson         0.526
Drew Bledsoe         0.525
Bubby Brister        0.525
Mike Phipps          0.525
Tony Banks           0.524
John Elway           0.523
Bob Berry            0.523
Danny White          0.523
Bobby Hebert         0.522
Neil O'Donnell       0.521
Philip Rivers        0.521
Jim Kelly            0.521
Dave M. Brown        0.521
Stan Humphries       0.521
Mark Brunell         0.520
Babe Parilli         0.520
Rodney Peete         0.517
Michael Vick         0.515
Billy Kilmer         0.514
Bill Munson          0.512
Steve Grogan         0.511
Vince Ferragamo      0.510
Rich Gannon          0.508
Dave Krieg           0.506
Kerry Collins        0.506
Don Majkowski        0.506
Joey Harrington      0.506
Bill Kenney          0.504
Jim Plunkett         0.504
Gary Danielson       0.503
John Hadl            0.503
Dan Marino           0.502
Bernie Kosar         0.501
Bob Avellini         0.501
Journeymen           0.498
George Blanda        0.498
Joe Ferguson         0.497
Kurt Warner          0.494
Cotton Davidson      0.491
Greg Landry          0.489
Marc Wilson          0.488
Eli Manning          0.488
Randall Cunningham   0.486
Sonny Jurgensen      0.484
Wade Wilson          0.483
Jake Plummer         0.481
Matt Hasselbeck      0.480
Jeff Garcia          0.478
Norm Van Brocklin    0.477
Chris Chandler       0.476
Mike Livingston      0.475
Y.A. Tittle          0.473
Tommy Kramer         0.473
Fran Tarkenton       0.472
Billy Wade           0.471
Vinny Testaverde     0.471
Brian Sipe           0.470
Warren Moon          0.467
Tim Couch            0.466
Peyton Manning       0.465
Gus Frerotte         0.465
Tony Eason           0.464
Charley Johnson      0.462
Joe Namath           0.461
Steve Bartkowski     0.461
Bill Nelsen          0.460
Tom Flores           0.459
Jim Hart             0.457
Ken O'Brien          0.457
Richard Todd         0.455
Lynn Dickey          0.454
Dan Pastorini        0.452
Replacement level?   0.450
Erik Kramer          0.445
Rick Mirer           0.444
Emergency QBs        0.441
Bobby Douglass       0.441
Jim Everett          0.440
Drew Brees           0.439
Norm Snead           0.439
John Brodie          0.439
Scott Mitchell       0.438
Boomer Esiason       0.437
Carson Palmer        0.433
Tobin Rote           0.430
David Carr           0.428
Steve DeBerg         0.426
Mike Pagel           0.424
Bert Jones           0.424
Chris Miller         0.423
Dan Fouts            0.419
Neil Lomax           0.413
Aaron Brooks         0.412
Jeff George          0.411
Trent Green          0.410
Jon Kitna            0.410
Lamar McHan          0.407
Eddie LeBaron        0.402
Jeff Blake           0.396
Jim Zorn             0.387
Marc Bulger          0.382
Daunte Culpepper     0.381
Archie Manning       0.374

Some technical fine print:

1. Brian Burke will point out --- and he won't be wrong --- that it would probably be better to use a dummy variable for each points allowed value, rather than a single variable. I tried that. It didn't change the coefficients much, but it did make the formula much more difficult to express and describe. So I decided it wasn't worth it.

2. Similarly, I put in a dummy variable for each year to try to capture any year/league-specific effects that might have been lurking. They didn't change anything.

3. I also added age dummies, which also didn't do much.

This entry was posted on Monday, March 30th, 2009 at 3:46 am and is filed under General. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.