SITE NEWS: We are moving all of our site and company news into a single blog for Sports-Reference.com. We'll tag all PFR content, so you can quickly and easily find the content you want.

Also, our existing PFR blog rss feed will be redirected to the new site's feed.

Pro-Football-Reference.com ยป Sports Reference

For more from Chase and Jason, check out their work at Football Perspective and The Big Lead.

Adjusting quarterback win-loss records, part I

Posted by Doug on March 25, 2009

Last summer I wrote a post that rated field goal kickers according to their accuracy compared to league average. And I also adjusted each kicker's field goal percentage to account for the distance of his attempts. If one kicker attempted a bunch of chippies while another was kicking a lot more long ones, then we should take that into account when comparing their percentages.

This is highly non-controversial. Obvious even.

So we're going to do it again. But instead of kickers, we'll look at quarterbacks. Instead of field goal percentage, we'll look at winning percentage. And as our measure of difficulty, we won't use distance but points allowed by the quarterback's team. Just as kicking a 45-yarder is more difficult than kicking a 23-yarder, it's harder for a quarterback to win a game if his defense gives up 30 than if they give up 10.

Highly non-controversial, right?

So here's the plan, which mirrors the kicker plan exactly:

STEP 1: compute each quarterback's winning percentage in each of six categories: (1) defense allows 0--10 points, (2) defense allows 11--15 points, (3) defense allows 16--20 points, (4) defense allows 21--25 points, (5) defense allows 26--33 points, and (6) defense allows 34+ points.

STEP 2: in each category, compute how many games that quarterback won compared to how many an average QB would have been expected to win.

STEP 3: sum up the QB's wins above or below average, across all six categories.

Let's run through Joe Namath as an example:

defense allows 0--10 points: Joe was the beneficiary of this kind of defensive performance 17 times. An average QB would be expected to win 15.9 of 17 games. Joe went 17-0. So he's +1.1 wins.

defense allows 11--15 points: 18 games. Joe is 16-2. An average QB would win 13.7. So Joe is +2.3 here.

defense allows 16--20 points: 23 games. Joe is 13-10. An average QB would win 12.1. So Joe is +0.9.

defense allows 21--25 points: 25 games. Joe is 10-15. An average QB would win 9.7. So Joe is +0.3.

defense allows 26--33 points: 24 games. Joe is 6-18. An average QB would win 4.1. So Joe is +1.9.

defense allows 34+ points: 25 games. Joe is 2-23. An average QB would win 0.9. So Joe is +1.1.

Add it all up (and ignore the rounding error) and Namath comes out at +7.6. Do that for every quarterback who has started 50 or more games since 1950 and you get the following list. Lots of commentary and fine print below:

                    G    W    ExpW   Diff
=========================================
Peyton Manning     191  124   92.8  +31.2
John Elway         252  162  134.5  +27.5
Tom Brady          128  101   73.9  +27.1
Brett Favre        291  181  154.7  +26.3
Dan Marino         258  155  129.8  +25.2
Joe Montana        187  133  109.1  +23.9
Ken Stabler        158  103   80.7  +22.3
Johnny Unitas      194  124  106.7  +17.3
Daryle Lamonica     97   70   53.1  +16.9
Jim Kelly          177  110   93.2  +16.8
Steve Young        157  102   85.8  +16.2
Roger Staubach     131   96   79.9  +16.1
Norm Van Brocklin  105   63   47.7  +15.3
Terry Bradshaw     177  121  106.3  +14.7
Dan Fouts          178   89   76.4  +12.6
Randall Cunningham 144   85   72.8  +12.2
Danny White        102   67   55.5  +11.5
Bobby Layne        139   83   71.6  +11.4
Kurt Warner        112   65   53.6  +11.4
Y.A. Tittle        139   78   66.8  +11.2
Frank Ryan          90   58   47.8  +10.2
Bill Nelsen         79   42   32.1   +9.9
Fran Tarkenton     250  130  120.1   +9.9
Otto Graham         78   61   51.3   +9.7
Stan Humphries      87   53   43.7   +9.3
Joe Theismann      132   83   74.3   +8.7
Jeff Hostetler      88   55   46.6   +8.4
Steve McNair       163   96   87.7   +8.3
Ben Roethlisberger  81   59   51.0   +8.0
Rich Gannon        139   80   72.3   +7.7
Steve Grogan       138   75   67.4   +7.6
Joe Namath         132   64   56.4   +7.6
Dave Krieg         184  101   93.6   +7.4
Drew Brees         109   56   48.7   +7.3
Matt Hasselbeck    112   62   54.8   +7.2
Bert Jones          99   47   40.1   +6.9
Jim Hart           182   87   80.1   +6.9
Trent Green        115   56   49.1   +6.9
Philip Rivers       54   36   29.2   +6.8
Jake Delhomme       89   55   48.2   +6.8
Marc Bulger         90   41   34.2   +6.8
Jay Schroeder      104   64   57.6   +6.4
Mark Rypien         85   52   45.6   +6.4
Eli Manning         78   46   39.8   +6.2
Ed Brown            98   55   49.1   +5.9
Charley Johnson    124   59   53.2   +5.8
Billy Kilmer       121   63   57.3   +5.7
John Brodie        164   76   70.6   +5.4
Don Meredith        89   49   43.7   +5.3
Warren Moon        213  105  100.2   +4.8
Brian Sipe         113   57   52.2   +4.8
Jack Kemp          111   67   62.4   +4.6
Jim Plunkett       154   80   75.4   +4.6
George Blanda      108   55   50.5   +4.5
Dan Pastorini      122   59   54.6   +4.4
Tony Eason          56   31   26.9   +4.1
Daunte Culpepper    99   43   38.9   +4.1
Bob Griese         162   98   94.1   +3.9
Sonny Jurgensen    149   69   65.1   +3.9
Troy Aikman        180  105  101.3   +3.7
Phil Simms         169  101   97.3   +3.7
Len Dawson         167   99   95.6   +3.4
Earl Morrall       108   67   63.6   +3.4
Neil Lomax         102   47   43.7   +3.3
Bart Starr         167  103   99.7   +3.3
David Woodley       58   37   33.8   +3.2
Boomer Esiason     178   83   79.8   +3.2
Jim Zorn           106   44   40.9   +3.1
Jim McMahon        103   70   66.9   +3.1
Michael Vick        71   40   36.9   +3.1
Jake Plummer       142   71   68.1   +2.9
Donovan McNabb     143   91   88.5   +2.5
Jeff Garcia        122   60   57.6   +2.4
Wade Wilson         74   38   35.6   +2.4
Charlie Conerly     92   58   55.7   +2.3
Marc Wilson         61   32   29.7   +2.3
Neil O'Donnell     107   58   55.7   +2.3
Brad Johnson       132   76   74.1   +1.9
Bobby Hebert       103   56   54.1   +1.9
Rodney Peete        89   46   44.2   +1.8
Brian Griese        83   45   43.3   +1.7
Mike Phipps         73   38   36.3   +1.7
Pat Haden           60   37   35.5   +1.5
Mike Tomczak        78   45   43.5   +1.5
Roman Gabriel      159   86   84.8   +1.2
Vince Ferragamo     59   30   28.8   +1.2
Babe Parilli       104   50   48.9   +1.1
Tom Flores          67   31   30.1   +0.9
Kordell Stewart     86   50   49.1   +0.9
Gus Frerotte        95   45   44.2   +0.8
Tobin Rote         119   51   50.2   +0.8
Elvis Grbac         73   41   40.2   +0.8
Carson Palmer       66   32   31.4   +0.6
Aaron Brooks        92   39   38.5   +0.5
Billy Wade          86   41   40.7   +0.3
Ken Anderson       178   93   92.7   +0.3
Craig Morton       154   86   85.8   +0.2
Doug Flutie         68   38   37.9   +0.1
John Hadl          169   82   82.0   +0.0
Jeff Blake         100   39   39.0   -0.0
Tommy Kramer       114   56   56.0   -0.0
Mark Brunell       160   83   83.1   -0.1
Erik Kramer         70   32   32.4   -0.4
Jay Fiedler         63   38   38.4   -0.4
Scott Mitchell      73   32   32.5   -0.5
Bernie Kosar       115   56   56.6   -0.6
Chad Pennington     83   45   45.8   -0.8
Don Majkowski       57   26   26.8   -0.8
Steve Bartkowski   131   60   60.9   -0.9
Bubby Brister       77   38   39.4   -1.4
Jon Kitna          116   46   47.5   -1.5
Richard Todd       112   50   51.5   -1.5
Milt Plum          103   56   57.9   -1.9
Chris Chandler     155   69   71.0   -2.0
Bob Avellini        51   23   25.8   -2.8
Gary Danielson      61   28   31.0   -3.0
Eric Hipple         58   28   31.1   -3.1
Ken O'Brien        112   50   53.2   -3.2
Jim Everett        158   66   69.3   -3.3
Bill Kenney         77   34   37.7   -3.7
Joe Kapp            52   26   29.8   -3.8
Jeff George        127   47   51.0   -4.0
Mark Malone         55   24   28.0   -4.0
Joe Ferguson       175   80   84.3   -4.3
Dave M. Brown       60   26   30.8   -4.8
Chris Miller        94   35   39.8   -4.8
Kerry Collins      171   82   86.9   -4.9
Mike Livingston     75   31   36.0   -5.0
Drew Bledsoe       199  101  106.1   -5.1
Greg Landry         99   44   49.1   -5.1
Eddie LeBaron       81   26   31.3   -5.3
Tim Couch           59   22   27.5   -5.5
Steve Beuerlein    104   48   53.6   -5.6
Cotton Davidson     54   20   25.7   -5.7
Lynn Dickey        113   46   51.7   -5.7
Rick Mirer          68   24   29.8   -5.8
Bob Berry           52   20   25.8   -5.8
Doug Williams       88   42   47.8   -5.8
Bill Munson         66   27   33.2   -6.2
Jim Harbaugh       145   68   74.7   -6.7
Bobby Douglass      53   16   22.8   -6.8
Trent Dilfer       119   63   69.8   -6.8
Mike Pagel          54   17   23.8   -6.8
Tony Banks          78   35   41.9   -6.9
Lamar McHan         73   24   31.0   -7.0
Steve DeBerg       144   54   61.2   -7.2
Ron Jaworski       151   77   86.4   -9.4
Norm Snead         158   52   61.6   -9.6
Joey Harrington     76   26   36.2  -10.2
David Carr          79   23   34.1  -11.1
Vinny Testaverde   219   92  103.4  -11.4
Archie Manning     139   35   52.4  -17.4

Fine print:

1. This includes all regular- and post-season games.

2. You might be wondering about era/league effects. It's easier to win if your defense allows 20 points in the 1961 AFL than it is to win if you defense allows 20 points in the 1974 NFL. Though I didn't mention it above, I actually did attempt to account for this. What I did was to compute the "average quarterback's expected wins" for a given category by looking at all games in that category in the same league within two years. For example, if I'm examining Joe Namath's 1966 season, the expected winning percentage for category (1) games, e.g., is computed by looking at all category (1) AFL games from 1964 to 1968.

Thoughts:

1. Don't forget that the line labeled "Peyton Manning" is really Peyton Manning, Marvin Harrison, Edgerrin James, Jeff Saturday, Tony Dungy/Jim Mora/Tom Moore, Mike Vanderjagt, and a cast of thousands. What it isn't, though, is the Colts' defense. Or at least not as much as Manning's raw record is. More on this later.

I've said before that quarterbacks don't win games (teams do), and I still believe that. But if people are going to talk about QB wins --- and it looks like it's unfortunately too late to put the lid back on that can of snakes --- they may as well try to put them into context. This post is is an effort to do that. To oversimplify things a little (or maybe more), Peyton Manning's record is the product of the efforts of 22 guys. This exercise attempts to narrow that down to 11.

2. I think this exercise has provided me with a new all-time favorite example of Simpson's Paradox. Check out Daunte Culpepper and Trent Dilfer:

                 +-------------------------------------------------------------------+
                 |                     Points allowed                                |
+------------------+-------------+-----------+------------+------------+------------+
| QB               | 10 or under | 11--15    | 16--20     | 21--25     | 26 or more |
+------------------+-------------+-----------+------------+------------+------------+
| Daunte Culpepper | 5-0 1.000   | 7-4 0.636 | 17-6 0.739 | 6-7 0.462  | 8-39 0.170 |
| Trent Dilfer     | 30-5 0.857  | 9-7 0.563 | 13-8 0.619 | 9-12 0.429 | 2-24 0.077 |
+------------------+-------------+-----------+------------+------------+------------+

Culpepper has him in every category, but Dilfer (63-46) has a better overall record (Culpepper's is 43-56) because their distribution of opportunities has been so different. Matt Hasselbeck and Bob Griese are another example:

                 +-------------------------------------------------------------------+
                 |                     Points allowed                                |
+-----------------+-------------+------------+-------------+------------+------------+
| QB              | 10 or under | 11--15     | 16--20      | 21--25     | 26 or more |
+-----------------+-------------+------------+-------------+------------+------------+
| Bob Griese      | 53-1 0.981  | 20-7 0.741 | 14-11 0.560 | 7-11 0.389 | 4-31 0.114 |
| Matt Hasselbeck | 25-0 1.000  | 6-2 0.750  | 12-5 0.706  | 11-8 0.579 | 8-35 0.186 |
+-----------------+-------------+------------+-------------+------------+------------+

And you can pair Marc Bulger with just about anyone. Bart Starr, for instance:

                 +-------------------------------------------------------------------+
                 |                     Points allowed                                |
+-------------+-------------+------------+-------------+-------------+------------+
| QB          | 10 or under | 11--15     | 16--20      | 21--25      | 26 or more |
+-------------+-------------+------------+-------------+-------------+------------+
| Marc Bulger | 8-0 1.000   | 8-1 0.889  | 8-4 0.667   | 9-6 0.600   | 8-38 0.174 |
| Bart Starr  | 51-3 0.944  | 17-5 0.773 | 20-10 0.667 | 10-15 0.400 | 5-25 0.167 |
+-------------+-------------+------------+-------------+-------------+------------+

3. Starr is barely above average in this analysis. In Part II, where we make a few more adjustments, he'll actually come out as below average. What are we to make of this? Those hoping for a Starr-is-a-fraud rant at this point will be disappointed. Maybe it's just because of the soft spot I have for Troy Aikman, who finds himself ranked similarly. Or maybe it's because I know that points scored and points allowed are correlated. Bart Starr wasn't playing defense, but he and the rest of his offensive teammates could have, and probably did have, an indirect impact on the number of points the defense allowed. How much of an impact? That's tough to say, but that's the case you have to make if you think Starr is an all-time great, or even, frankly, an all-time good. What this does, in my mind, is eliminate the argument that Starr was good at doing just enough for his team to win. He wasn't any better at that than Marc Bulger or Tony Eason or Randall Cunningham.

Lest you think this is a canned stat-head rant, let me also call attention to Ken Anderson. If there's one thing that all historical football stat nerds seem to agree on, it's that Ken Anderson is underrated. But this analysis says otherwise, ranking him dead average.

Stay tuned for part II, where I'll use a completely different method to derive a similar list.

This entry was posted on Wednesday, March 25th, 2009 at 3:58 am and is filed under General. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.