SITE NEWS: We are moving all of our site and company news into a single blog for Sports-Reference.com. We'll tag all PFR content, so you can quickly and easily find the content you want.
Also, our existing PFR blog rss feed will be redirected to the new site's feed.
Pro-Football-Reference.com » Sports Reference
For more from Chase and Jason, check out their work at Football Perspective and The Big Lead.
ShareThisWhat’s a starting QB worth? Part II
Yesterday I asked you to speculate on how many points per game teams have historically lost when their starting quarterback was replaced by a backup.
Before I tell you the results, let's walk through a theoretical exercise....
If you take an average offensive team, let's say the 2008 Redskins, and you assume that they lost ALL their offensive starters for the season, how many points do you think they'd score per game in 2008?
They'd be rolling with the likes of Todd Collins, Ladell Betts, Devin Thomas, Malcolm Kelly, Fred Davis, and a backup offensive line. Arguably, you might give them the services of Antwaan Randle El instead of Kelly. They might go out and sign some veteran Shaun Alexander-type players off the street.
I'd say that offense scores about 12 points per game. They couldn't be that much worse than last year's Chiefs or 49ers, who averaged about 14 points per game. The last three expansion teams combined have averaged 16 points per game and all were above 13. Twelve points per game would be the second-lowest figure of the last seven seasons (thank you 2006 Raiders) and would have been last in the league in seven of the last ten years.
If you're willing to buy 12 points per game, then you've just agreed that an entire average offensive starting eleven is worth about 9 or 10 points per game.
Now, what percentage of an average offense does the quarterback account for? 20%? 30%? 40%?
If you say 30%, then the other ten are worth a combined 70%, or 7% each on average. That makes the QB worth more than four times as much as a typical non-QB offensive player. Is the QB worth more than the two guards, the center, and the right tackle combined? Is he worth more than the tight end, both wide receivers, and the running back combined? I don't think so.
I'm not sure how reliable the data accompanying these purty pictures are, but this link (and those that follow) give the average salaries by position as follows:
QB - 1.97 million
OL - 1.27 million
WR - 1.05 million
RB - .96 million
TE - .86 million
That would imply that 16% of the total salary paid to offensive players goes to quarterbacks. I believe that includes all players, not just starters, but I don't see why the starting-eleven ratio would be too much different.
In short, if NFL General Managers believed that quarterbacks were worth (more than) four times more than wide receivers, why do they only allocate twice the resources to quarterbacks? And if NFL GMs don't think quarterbacks are worth four times more than wide receivers, then why do you?
Anyway, if you don't agree, fill in your own number and go with it. My number would be around 20% or 25%, which would make the QB worth about two-and-a-half to three times what a typical non-QB offensive starter is worth.
So a typical QB accounts for about 20% to 25% of the offense, and a typical starting eleven is worth about 10 points per game. Thus, a typical starting quarterback is worth around two to two-and-a-half points per game.
The result of the study I described yesterday is the empirical verification of the theoretical observations above. Actual teams who have replaced their starter with a backup have on average dropped off by 2.3 points per game.
As I mentioned yesterday, I tried lots of different variations on the definition of a starting quarterback. These tweaks made the sample sizes grow or shrink, and they made the "average starting quarterback" better or worse. But in virtually every case, the number was between 2 and 3.
How does that translate to wins?
A regression of Simple Rating System rating to wins says that adding one point per game will add about .45 wins over the course of a season. So a typical starting quarterback is worth 2.3 points per game, and 2.3 points per game is worth just about one win per season. (A Pythagorean calculation, by the way, would give a very similar result.)
If this still seems too low, let me rattle off some examples:
In 2007, the Texans scored more points with Sage Rosenfels starting than with Matt Schaub starting.
In 2006, the Eagles scored more points with backup Jeff Garcia than with Donovan McNabb.
In 2005, the Jags were better with backup David Garrard than with starter Byron Leftwich.
If you're tempted to raise an objection on the grounds that David Garrard is too good a backup to be considered typical, I will overrule it. Part of the point of this is that sometimes, backups are better than you think they are. David Garrard was the very definition of a backup quarterback when he replaced Byron Leftwich in 2005. And Leftwich, who had an 8-3 record as a starter and a 15/5 TD/INT ratio, certainly looked like an above average starting quarterback.
The 2001 Vikings scored more points (per game) with Spergon Wynn and Todd Bouman than with Daunte Culpepper.
The 2000 Saints' offense was much more productive with a young backup named Aaron Brooks than it was with veteran starter Jeff Blake, who had been having a fine season.
The 1998 Broncos scored more with Bubby Brister playing than with John Elway.
The 1996 Dolphins scored more with Craig Erickson than with Dan Marino.
And the list goes on.
There are, of course, lots of examples of the opposite thing happening. But when you add them all up, it comes out to an average of 2.3 points per game.
In closing, I'll mention that you could run through the same exercise for a particular QB that we just did with an average QB. It'll still just be conjecture, but it'll be fun. Take Tom Brady for instance.
If the Patriots lost their entire starting eleven tomorrow, how many points would they score? I'm inclined to say maybe 16 per game, because Belichick is a genius and the Patriots do everything The Right Way and John Lynch would probably move to tight end and all that sort of thing. With their actual starting eleven, I'd go ahead and give them credit for the 36 points per game they scored last year. On one hand, they probably caught a bit of lightning in a bottle last season and can't keep up that pace. But on the other hand, they weren't playing with their actual starting eleven all season either. If they had all eleven guys for 16 games this season, they probably could match last season's output. So I'll estimate that the Patriots' starting offense is worth 20 points.
Brady is probably a bigger part of the Patriots' offense than a typical quarterback is to a typical offense, so maybe give him credit for a third of that. Then round up, and you're at 7 points per game. That's right around three wins. I believe Vegas is currently showing an over/under of 12 or 12.5 wins for New England, so the above says that the Patriots would be a 9- or 10-win team if they lost Brady tomorrow.
[EDIT: it just occurred to me that the Vegas lines have some probability of a Brady injury already built into them. If New England were assured of Brady's services for 16 games, the over/under would probably be 13 wins. Maybe 13.5, though I don't think it would go that high. So I think that means that my estimate is that New England would be a solid 10-win team (maybe 11) without Brady.]
Does that sound about right?
This entry was posted on Tuesday, August 19th, 2008 at 5:17 am and is filed under General. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.
In light of your results, and assuming a team would have to make such a decision, what would be better:
a) having your starting quarterback but having to play on the road; or
b) having to start the backup quarterback but getting to play at home.
Your numbers would say, slightly, home field advantage.
Second question, is there any evidence that teams play better the first week a backup is starting than, say, the fourth week?
It sounds ridiculously low to me - but who am I to question the numbers?
BTW you gotta wonder what in the world the Miami/Denver-coaches did to make journeymen-backups outplay there HOF-starters! Anything but samplesize/opponent-issues will blow my mind!
Excellent question JKL - i would love to know if the "who is that guy"-effect declines - you know because defences get more gametape of the backup.
How many cases did you get in the results, & could you put it out in tabe form? I'm curious how often the replacement QB pulls a Brady/Gehrig & doesn't let go, or if the old saw about not losing your job over an injury is true.
Really interesting.
A couple points. 1. Offense also affects points allowed. The deeper an offense penetrates, even without scoring, the less likely an opponent is to score themselves. So the answer of 2.3 pts/g may only be half the answer.
2. Regarding the relative value of a QB, an economist would point out that utility curves are not linear. An owner only has to pay someone enough to get them to come play for your team rather than someone else's, and no more. The marketplace for NFL skills is very small on both the demand side and supply side and has lots of distortions.
There's a lot of economic rent to go around. Rent is the term for surplus income above the minimum level someone would be willing to supply a service or good. In other words, if you cut all NFL QB pay by 90%, they'd all still be willing to play. That's still more than they'd make doing just about anything else. Considerations like that really complicate the analysis.
There is, however, a method in cooperative n-player game theory that prescribes what share of a gain each member of a coalition should get based on bargaining power. You can think of an offense as a coalition of 11 cooperative members of a coalition each bargaining for a slice of the pie. Game theory will tell you if the division of spoils is stable or not, but it cannot precisely solve the perfect equilibrium in this case. I can't remember the name of the concept at the moment.
Read my post in yesterday's comment section. The overall numbers "sound right." (I predicted 3 ppg.) HOWEVER, for each one of Doug's examples of the backup OUTSCORING the starter, there is another one where the backup was, say, 4 points worse (to "balance" the numbers). I will say that replacing Brady with Matt Cassel (or say Culpepper or another FA) for the entire season would result in a MAJOR drop in ppg. I would even wager that losing JUST Brady would result in the Pats dropping more than a TD per game, simply because the opposing D's would basically play a T-Jax defense: crowding the line to make Cassel beat them with the pass. If they were to pick up a higher quality back-up, the projection obviously changes. If any NE fans read this blog, give us your opinion--if Cassel had to play all 16 games, what would NE's ppg and record look like? (Colts fans, Cowboys fans, and fans of another team who do not have a quality backup, feel free to do the same. As a Saints fan who has seen the offense this preseason with Brunell, I would say we drop 5 ppg and thus end up at 6-10)
Cassel has just about played himself out of his backup slot. If Brady goes down and the first backup stinks like Cassel has in the preseason, they'll have a second backup in place the next week. Then a 3rd or a 4th, until they get somebody who can play.
Joseph, I think one of the points here is that losing a starting QB is typically not as bad as it would seem. To me, this makes sense. QB's always get more attention than they deserve, which makes people think they are irreplaceable. But there is a reason that Cassel has stuck around with the Patriots for a few years - the coaches must see something in him. They trust that he can be somewhat productive if needed.
We don't REALLY know what Cassel would do if he had to start a few games, because he hasn't done it yet. Just ask Tom Brady how that works out. Going into 2001, how many people would have guessed that a Drew Bledsoe injury would HELP the Patriots? Going into 1999, how many people would have guessed that a Trent Green injury would HELP the Rams?
It seems to me that typically if a good NFL team loses a single player to injury, they aren't affected all that much. One player just doesn't have that much impact on the game. The problem is when the injuries start piling up that teams start really feeling the effects.
Joseph, which part of my Brady value assessment do you think is off?
Re #8 - we don't know that Trent green's injury helped the Rams. When Green played in Warner's place in 2000, even though Green's knee was still bothering him - it wasn't completely sound until 2002 - he pretty much matched Warner's numbers. See Warner vs. Green's DVOA for 2000 at FO.
It seems low, but I think it's because there have been several back ups that were previously (or would become in the future) experienced starters that came in and played well. Guys like Garcia, Garrard, Brister, etc.
If you had to go a season with the likes of Jim Sorgi or Matt Cassel, then you're probably in trouble. I know the sample size is small, but there have been 4 games for the Colts where Sorgi played the majority of the game (at least 3 quarters). The Colts are 1-3 in those games and they never scored more than 17 pts. With Manning they routinely win double-digit games and score over 25 ppg.
I also think Brian in post #5 may be on to something. Instead of looking at ppg for offense, how about overall point margin in those games? Maybe the starting QB is better at sustaining drives, converting 3rd downs and limiting the number of drives the defense has to take the field. It would also help in clearing up games where one QB scores 24 pts in an easy win versus a back up having 24 pts in a game they were blown out (maybe they only had 10 going into the 4th and added two late TDs that really didn't matter).
mrh, the point is that at the time of Green's injury, we assumed that the Rams had no hope of being competitive that year. Even before the injury, expectations were low since the Rams had been bad the whole decade. But the additions of Faulk and Green provided a little optimism.
I remember when I first heard that "Curt Warner" was going to be the starter, and had no idea who he was, and could only think of the former Sea/Ram RB.
I think you also have to consider that those games all were played in either week 16 or week 17 when the Colts didn't care if they won or lost. Did he get to play with the full complement of weapons in those performances?
Doug,
The analysis overall is correct (I predicted 3 ppg). I think that for some teams/players (especially the elite at their position), the dropoff is even worse. I don't mean to sound critical, just that I think that Brady and Manning are worth more than even 7 ppg to their respective teams. IMO, this is why the Colts brought in Quinn Gray and Lorenzen to compete with Sorgi. They needed a better backup option in case Manning had a severe injury and couldn't start for several games.
#7 Downpuppy--as of right now, who "could" the Pats bring in besides Culpepper (who imo is the best available QB out there right now, but imo is a bad fit for their system)?
Doug---That is very interesting stuff. IMO, it all points out one thing that I have been posting for awhile now--when the pressure mounts in the playoffs, some guys can handle it and some can't. It's not always about HOW MANY points a QB can lead his offense to, it's also about how he handles the situation. How many QB's could have done what Brady did in '01 on that very last drive to get into field goal position? How many would have thrown an INT and led their Team to a loss? And it doesn't always have to be at the very end of the game. Every drive gets magnified in a win or go home situation. Some guys found a way to handle the pressure(a MAJORITY of the time) and some did not. Most of those that did are now wearing Super Bowl Rings.
I ran through a similar exercise a few months ago using NFLPA payroll data identifying starters' average salary by position and DPAR numbers from Football Outsiders and got a very similar result: the average starting QB is worth 1/11 to 1/7th of the top 30 "regular players" (including the top regular situation substitutes).
That means the average QB is worth maybe four other average regular players.
That's *a lot* -- yet it also is not nearly as much as all those fans who think the QB "lifts and carries the team to the playoffs/title" believe. (Like all the Jets fans I know who believe that with 39-year old Favre they are already in the playoffs this year, after last year's 4-12).
In context, the QB is worth about as much as the second best player on a basketball team -- and no basketball fan I know says "Our team has to sign a new second-best player to carry us to the title!"
Of course this is the *average* starting QB that the data I used describes. Peyton Manning (and the other players signed by the Colts specifically to optimize his talents) takes a rather larger percentage of the Colts' payroll and had rather higher DPAR. A lot of fans, when thinking of "how much a QB is worth" picture Peyton Manning or Steve Young or Johnny Unitas at the top of their game versus some shlub, and think "that much". But that's the most extreme case, far from the average case.
Also, to see how *dependent* the supposedly great, "team carrying" QB is on the rest of the team, think about how many universally recognized "great QBs" spent their careers on bad losing teams. The answer is, precious darn few.
That's very different than from baseball and basketball players -- and even from NFL running backs -- where there's no problem making a list of Hall of Famers who spent most of their careers on bad losing teams.
In football, although hoards of fans think the QB "carries the team", the rest of the team's level of play feeds into the QB's performance numbers at least as much as his level of play feeds into theirs. Thus so few "great" QBs from bad teams. The rest of the team makes the "great" QB much more than the typical fan believes. QBs always get far too much of both credit and blame for team performance from the fans. The rest follows. (If Peyton and Archie swapped teams...?)
This isn't to say that being worth four other players isn't at lot -- it *is* a lot, it's by far the most important single position on the team.
It's just that with 30 or so regulars on a team, it's not nearly so much as many fans believe. To the team as a whole, its being worth about as much as the second-best player is to a basketball team with a roster of 12.
Joseph, did you miss the last three paragraphs of the post?
So can we officially label Denny a troll yet?
"IMO, it all points out one thing that I have been posting for awhile now–when the pressure mounts in the playoffs, some guys can handle it and some can’t. It’s not always about HOW MANY points a QB can lead his offense to, it’s also about how he handles the situation. How many QB’s could have done what Brady did [time and again]..."
~~~~~
I don't know how that follows. To me the QB being worth "only" three other players of the first 22 (or four of the first 30) or 2.3 points a game, shows just the opposite -- how the team makes the reputation of the QB.
I can think of *plenty* of times Brady made bad plays in the clutch, and then somebody else on the field bailed him out, and the game ultimately added to the legend of how Brady always wins the close games. (Brady throws a playoff game pick right into the arms of a Charger that should be a sure game loser, but the dumb-butt decides to run it back and fumble it right back. Brady wins another close game by being error-free in the clutch! Etc. etc.)
OTOH, in the last SB Brady performs a real, true, last minute should-be game winning TD drive. But then somebody on his own team lets an easy pick to secure the game sail through his hands ... then the opposing QB chucks a ball that somebody catches against his helmet, for cryin' out loud ... and then Eli, who around game 14 was still considered a "bust" by many Giants fans, instead of confirming his status as a clutch pick-throwing chucker, is suddenly seen as "now become a man, a leader to last-minute victory!!" No credit for Brady who lost with the better 18-0 team.
The QB near always gets *way* more credit or blame than he deserves, according to what the team did. What the rest of the team does has a huge impact on "how good" he is popularly taken to be.
And "clutch play" is pretty much the same fan myth for players in the NFL as it is for other sports.
http://www.advancednflstats.com/2008/01/is-red-zone-performance-real.html
Joseph - How should I know who's best? My point is that nobody knows how good a backup will be until they get the ball.
There are a vast number of backup QBs on NFL rosters, and armies more in lesser leagues & driving trucks. Brady was a 4th stringer at the end of 2000. So there's no reason to stick with a bad one, and I get the impression Hoodie Bill agrees.
I think Brady and Manning have to be worth at least 7-10+ PPG. Both teams average around 30 PPG. Do you honestly think those teams would score more than 20 PPG with the backup QB?
Another thing that needs to be taken into consideration is that a better QB also has an effect on the defense. If you have lead then the defense has less to worry about and performs better and it also has a better chance to cause a turnover which could lead to another score.
To JKL's 2nd question in comment #1:
My theory is that in the first week, the team as a whole is likely to rise to the occasion and play over their heads a bit. Human nature, being what it is, would suggest that it's hard to stay risen to an occasion (and play over one's head) 4 games in a row. Also, 4 games is adequate to prove you deserve to be a backup and not a starter.
So I crunched a few numbers and determined that, for any individual quarterback in 2008, he figured in about 24.25% of his team's plays via a pass or sack. The rest are runs (I couldn't separate out QB runs, so the actual percentage is probably a little higher), defensive plays, or special teams plays. If you thought the QB was utterly and solely responsible for the success of the passing game, you could use that figure to determine how valuable he is. I don't know how you could argue that he's worth more, unless you think he's awesome at handoffs or really pumps up the kicker on the sidelines.
Of course, nobody would think he is. If we go with Doug's high estimate that a QB is worth about 50% of his team's offense (I'm assuming his 20-25% number already assumed the QB was useless on about half of his team's plays, i.e., running plays. I get the same thing if I take his 25% guess and apply it to the roughly 43% of all plays that are offensive plays), then you have a QB being worth about 50% of his team's efforts on about 25% of his team's total plays: that's 1/8, or 12.5%.
Suddenly, I don't feel so bad about having Gus Frerotte as my backup.
Hey Doug,
I didn't miss them, I just think that there should maybe be 2 studies. Or maybe it should be emphasized more. (Sorry, I'm dealing with stress here because certain people aren't listening/doing anything about some problems.) Don't mean to act like a jerk or anything.
Maybe 3 studies--one for elite QB's (or where the starter is head and shoulders above the backups [see 2007 Panthers], one for average, and one where the backup eventually hastened the former starter's release (I know the Bledsoe/Brady Pats weren't part of the study, but the Leftwich/Garrard Jags were). My guess=elite value is 7 ppg/2.5 wins (your Brady #'s); average value=2.3 ppg/1 win (your overall study); backup turned starter value=0 or negative ppg/0 win effect, or better for the backup.
Thanks for the great study. This will help fans of some teams going into the opener.
Backup QB's may be backups for a reason, but they're freaking N-F-L backups. These guys are ALL athletes, they can ALL throw. Your average NFL backup QB is pretty unbelievable when you get down to it. How many of the starters today started out as backups at least for a few years? Did they magically get better when they became starters?
My gut feeling is that if you have an actual honest-to-goodness AVERAGE starting QB, and you have an actual honest-to-goodness AVERAGE backup, and your starter gets hurt, your numbers will go down, but I think that has more to do with the disruption involved in the actual changing of the QB, not the quality of the QB's involved. If the switch was in the offseason so there wasn't that dissonance of changing QB's mid-stride, I think the difference would be awfully close to zero.
It might be interesting to see how teams fare when the QB is changed mid-season, and it doesn't fall into your query. That would include all those other situations -- teams who hadn't really found a good fit at QB, teams with lousy starters that got hurt, etc.
gedankenexperiment...
If you took Joey Harrington and put him on the Patriots, let him learn the offense, give him a month and half to throw thanks to the pats offensive line, give him a running game he can rely on, and give him a defense that'll keep him in the game so he's not throwing it up and praying half the game, give him good field position to start his drives, give him Randy Moss to throw to in that month and a half.... Would he still be the same Joey Harrington we all know and pity?
I know what I think, but I'm curious what others would expect out of him. Would he be terrible? Would he be a Trent Dilfer'ish QB? Would he excel? What are the odds he'd just just as well as Brady?
Another random thing:
Jake Plummer on the Cardinals was 30-52 with 90 TDs and 114 INTs.
Jake Plummer on the Broncos was 39-15 with 71 TDs and 47 INTs.
Maybe he had some sort of late-career epiphany and suddenly became orders of magnitude better and it just happened to coincide with his trade from a crap team to a good team. I'm betting he always was a good quality QB, and as suggested above, QB's get far too much credit/blame for the team's performance.
That said, I think the one QB you just couldn't replace is Peyton Manning. It's not that he's so much better than everybody else, though he's pretty phenomenal. He just seems the least... modular. He micromanages that offense like no QB I've ever seen. You put a more typical QB there and I'm just don't know how that'd affect things. Put Manning on another team where he doesn't have "his" offense in place and he'd still be good, but I don't know just how good.
I apologize if this was a bit incoherent.
In context, the QB is worth about as much as the second best player on a basketball team — and no basketball fan I know says “Our team has to sign a new second-best player to carry us to the title!”
I guess you don't know any Cavaliers fans.
If the Pats score 35 pts a game, and subs score 12, shouldnt that offense be worth 23 pts?
As an old Bulls fan, I knew that when the Bulls signed their 3rd best player - Dennis Rodman - they would win the title. A basketball team with a superstar as a 2nd player is a title contending team.
To judge the top end, what is the difference in points when a superstar (Brady,Manning etc.) go down? If you were to look at the top qbs, is the difference around 7 points?
What is the value of a running back? Is it 2 points, or even less?
Regarding new second best players in basketball, check out the Bulls before Scottie Pippen and the Bulls after Scottie Pippen.