Malcolm Gladwell, google searches, and quarterback draft status versus performance as predictor of future playing time
Posted by Jason Lisk on November 24, 2009
Last week, a high level cat fight broke out over Malcolm Gladwell's book "What The Dog Saw" when Dr. Steven Pinker wrote a critique in the NY Times. That critique included a reference to one essay in the book, which originally ran last December in the New Yorker, entitled "Most Likely to Succeed: how do we hire when we don't know who's right for the job?" In that essay, Gladwell states, in reference to what he calls the quarterback problem, that "[t]here are certain jobs where almost nothing you can learn about candidates before they start predicts how they’ll do once they’re hired." Pinker responded that "[i]t is simply not true that a quarterback’s rank in the draft is uncorrelated with his success in the pros."
Gladwell fought back on his blog. His responses were primarily attacks upon the individuals later cited by Pinker to support the position that draft position does matter, contrary to what Gladwell claimed, with minor reference that the critiques failed to appreciate the difference between aggregate performance and per play performance. He closed with:
I have enormous respect for Professor Pinker, and his description of me as “minor genius” made even my mother blush. But maybe on the question of subjects like quarterbacks, we should agree that our differences owe less to what can be found in the scientific literature than they do to what can be found on Google.
This, of course, piqued my interest. I admit to having heard reference to Gladwell's essay that originally ran last December, but had not paid it much attention. When I see defenses that are primarily based on attacks of the person, and what I see as an initially questionable assumption (per play statistics are all important; aggregates do not matter), well, I feel compelled to dig further. I happen to believe that the merits of an argument rise and fall on the quality of the facts and analysis, and not on who made it. This is true whether the arguments are presented in a scientific journal or on a blog. Oh, and I wanted to add something that could be found with a Google search.
I have little interest in whatever motivations drive Pinker and Gladwell's exchange. I'm interested only in the football aspect. Does draft position matter in predicting success? To see where Gladwell is coming from, we need to turn to his source. That source was an article published in the Journal of Productivity Analysis, entitled "Catching a draft: on the process of selecting quarterbacks in the National Football League amateur draft" by David Berri and Rob Simmons. In that article, Berri/Simmons looked at quarterbacks drafted between the 1st and 250th pick of a draft since 1970, found all cases where a quarterback from that group started at least one game in an NFL season, and then calculated performance using Berri's QB Score measure, on both an aggregrate and per play basis. The authors concluded, when looking at nearly 40 years of data, that a relationship between a quarterback's draft position and his subsequent NFL performance on a per-play level could not be found. In other words, later selected quarterbacks played as well as quarterbacks selected at the very top of the draft, when the number of opportunities was ignored, and they were assessed on how they played per play.
When the spat broke out between Gladwell and Pinker, Berri also posted some thoughts on his Wages of Wins website.
My sense is that Pinker never read our article. What he did find on the Internet is evidence that a quarterback’s aggregate performance (i.e. passing yards, seasons played, Pro Bowl appearances) is indeed related to draft position. And as Rob and I detailed in our article, this is true. Aggregate performance and draft position are statistically related. But as Rob and I argue, this is because in the NFL (like we see in the NBA) draft position is linked to playing time. And this link is independent of performance. In fact, Rob and I find that draft position – again, independent of performance – impacts a quarterback’s pay many years into a quarterback’s career.
It is the bolded part of that quote that I want to focus on. This is a key underlying assumption for the position that Gladwell (through Berri/Simmons) ultimately takes (Although apparently it was a position that Gladwell had assumed long before that article was written). If quarterbacks get playing time because of their status, independent of how good they are, then Gladwell's position has some support.
The math in the Berri/Simmons article is fine. They use their own measure, QB Score, rather than others like Adjusted Net Yards per Attempt or QB Rating, but ultimately, all those per-play measures are including roughly the same things, and the differences are not going to change the conclusions. We can say with reasonable certainty that, as Berri/Simmons find, later round quarterbacks who actually play in the NFL perform about as well as early draft picks who actually play. This part is adequately portrayed in the article.
What I cannot find support for in the article, other than the reference to a quarterback's pay, is the underlying key assumption that playing time is linked to draft position, and this link is independent of performance. Clearly, earlier drafted quarterbacks get an opportunity at a younger age on average than later drafted quarterbacks who eventually become starters, so draft position (along with opportunity and team need) does dictate how soon a quarterback gets a chance. Berri, however, separates out per-play performance based on years of experience and reports that correlations are non-existent on a per-play basis even looking at quarterbacks with 5+ years of experience in the league. So, I don't think Berri is making the "playing time related to draft position independent of performance" based only on rookie starters at age 22 and 23, with a belief that everything being equal thereafter.
To test the interplay of performance versus draft status, I looked at all quarterbacks going back to 1960 who threw 200 or more passes in the NFL at age 24. From this list, I excluded all undrafted free agents and all supplementary draft picks (where I can't assign a specific draft number), so that we are only measuring players who were drafted in the regular draft. I also excluded players who had not turned 29 years of age as of the 2008 season. I chose age 24 for a couple of reasons. First, prior to age 23, there are very few seasons where quarterbacks threw 200 passes. Matt Stafford, for example, has joined Fran Tarkenton and Drew Bledsoe as the only three quarterbacks to throw that amount at age 21. The second reason I chose age 24 is because, for most quarterbacks, it is not a true rookie season, and we get a much larger variety for draft position, as a higher number of guys not selected in the first round start to get some playing time.
I ran a regression with the following input variables for that group of quarterbacks:
1) the quarterback's per play value above or below league average at age 24 ("PERFORM"), using the formula for value used by Chase in his most recent "Greatest Quarterback of All-Time" series; and
2) the quarterback's draft position ("DRAFT"), using a draft value number used by Chase originally here.
The three separate outputs used were games from ages 25-29, games started from ages 25-29, and performance above or below replacement value from ages 25-29. I used the age 25-29 period as the output so that a) we would have a larger data set and not have to exclude all currently active players, and b) so that injuries later in the career or decisions to retire at one age versus the other (which we would assume are not dependent on either draft position or performance at age 24) are not factored. Of course, injuries still can determine how many games a specific QB played, but this limits that somewhat. Oh, and for players who had a strike year or played before 1978, all the outputs were normalized to a 16-game schedule (so that 80 games is the max available).
Here are the results:
GAMES STARTED FROM AGES 25 TO 29= ~ 34.69 + 8.49*PERFORM + 0.39*DRAFT
GAMES PLAYED FROM AGES 25 TO 29= ~ 46.78 + 6.56*PERFORM + 0.23*DRAFT
VALUE OVER REPLACEMENT FROM AGES 25 TO 29= ~ 1930 + 866*PERFORM + 31*DRAFT
All the variables were strongly significant. As it turns out, how the player performs at age 24 and the draft status both matter. The fact that draft status matters, though, doesn't automatically mean that all of it is due to teams starting high draft picks for no reason other than to justify their selection (though there is some of that). Some of the "draft status matters" is that the scouting is proven correct, and the limitations of separating quarterback performance measures from the contributions of teammates is shown, because the higher drafted quarterbacks perform better than lower drafted quarterbacks who happened to play about as well at age 24. The Troy Aikmans of the world sometimes pan out, and do so more frequently, than the below average performers who were not highly drafted.
That said, the results also support that some highly drafted quarterbacks do get more opportunities than their performance dictates. This is something that I tried to look at last year in regard to Joey Harrington, and I do think that teams commit more false positive errors in regard to high picks (continuing to give them plays when the evidence suggests they are highly likely not to pan out) than false negatives (giving up on a high pick too soon, only to have him succeed elsewhere). If you compare the variables for Games Started versus Value over Replacement, you will see that the PERFORM variable is about 100 times larger for Replacement Value compared to Games Started. In contrast, the DRAFT variable is only about 80 times larger. Thus, performance at age 24 is relatively more important in determining value over replacement (for those that continue to play) than for games started. To put it into specific examples, occasionally, Joey Harrington and Rick Mirer get to bounce around and start far more games than they should, and this happens more frequently for previous high picks than low picks who played about like Harrington or Mirer at age 24.
What is clear to me, though, is that performance matters. A lot. I know this is a shocking finding in a performance driven business like the NFL. I'll also add that my choice of a 200 pass attempts cutoff (instead of say, a 100 pass attempts cutoff) probably increased the importance of the DRAFT variable by excluding guys who were already being weeded out, despite their draft status, by age 24. Poster boys for the early draft busts such as Akili Smith, Art Schlichter, Todd Blackledge, Heath Shuler and Andre Ware didn't even make the data set because they didn't throw enough passes at age 24. Cade McNown was out of the league. Ryan Leaf, who is THE #1 poster boy, is included, and he started 3 more games in his NFL career. Performance (or at least not acting like a jerk while not performing) matters.
Here is a look at the projected Games, Games Started, and Value over Replacement for ages 25-29, for the players who threw 200 passes at age 24 since 2004.
| First | Last | Year | G | GS | RV | RV/G | |
|---|---|---|---|---|---|---|---|
| Byron | Leftwich | 2004 | 61 | 58 | 3848 | 63.1 | |
| Mike | Vick | 2004 | 55 | 52 | 3049 | 55.4 | |
| Eli | Manning | 2005 | 66 | 66 | 4470 | 67.7 | |
| Kyle | Boller | 2005 | 49 | 40 | 2204 | 45.0 | |
| J.P. | Losman | 2005 | 44 | 34 | 1571 | 35.7 | |
| Ben | Roethlisberger | 2006 | 55 | 50 | 3074 | 55.9 | |
| Andrew | Walter | 2006 | 30 | 15 | -232 | -7.7 | |
| Derek | Anderson | 2007 | 55 | 46 | 3003 | 54.6 | |
| Jay | Cutler | 2007 | 61 | 57 | 3862 | 63.3 | |
| Tarvaris | Jackson | 2007 | 47 | 36 | 1926 | 41.0 | |
| Vince | Young | 2007 | 55 | 51 | 3091 | 56.2 | |
| Trent | Edwards | 2007 | 45 | 34 | 1739 | 38.6 | |
| Brodie | Croyle | 2007 | 41 | 29 | 1201 | 29.3 | |
| Kellen | Clemens | 2007 | 42 | 30 | 1264 | 30.1 | |
| Tyler | Thigpen | 2008 | 46 | 34 | 1782 | 38.7 | |
| JaMarcus | Russell | 2009 | 41 | 34 | 1168 | 28.5 |
Ben Roethlisberger is notably underprojected here, but how many Quarterbacks have won 26 regular season games and a Super Bowl before age 24? 2006 was a down year for Big Ben, and our formula doesn't know about Ben's unique career before then. I included a projection for Russell based on his per play stats. The fact that a player like Tyler Thigpen (who was below average and a 7th round pick and hardly a star last year) has a similar projection in terms of games started to Jamarcus Russell (who has been horrible at age 24 as a the first overall pick) sums it up in a nutshell.
If you believe that the only reason Carson Palmer has played a lot more than Gibran Hamdan is because Palmer was drafted alot higher, then you can accept Gladwell's position. Otherwise, you probably cannot, at least to the extent Gladwell portrays, because we haven't accounted for the myriad of late round picks where the initial scouting met the performance teams were seeing in practice, and they never got any extended opportunity to play outside of the practice squad and pre-season contests. When we look at the top 20% of late round picks (those who are judged good enough to play or forced into action because of injuries) and they are roughly similar to the top 80% of high draft picks, that does not mean that late round picks are equal to early picks, and the NFL has a quarterback problem where nothing that happens before can predict what will happen in the future. Per play stats matter, and it's important to look at quarterbacks from that perspective, otherwise we reward compilers who get opportunities without merit. More opportunities matter too, though, and a quarterback who plays well over a larger sample size is likely better than a quarterback who plays well over a small one, particularly when qb stats are more volatile due to outside factors such as teammates.
This is a distinction that, as far as I can tell, Gladwell fails to grasp.
November 24th, 2009 at 11:37 am
Very interesting analysis. A question:
You say that "the higher drafted quarterbacks perform better than lower drafted quarterbacks who happened to play about as well at age 24." I imagine Berri would reply that this is just a function of playing time, since value over replacement in part reflects playing time. Does draft pick also predict value per game?
Of course, Berri seems to assume that "performance" is determined mainly by the rate stats. For him, league average rate stats over 8 games and zero starts is an equal performance to the same stats over 16 starts. That helps create the illusion of no correlation between draft pick and performance.
November 24th, 2009 at 12:31 pm
Good article except for one thing:
Your Equations are
GAMES STARTED FROM AGES 25 TO 29= ~ 34.69 + 8.49*PERFORM + 0.39*DRAFT
GAMES PLAYED FROM AGES 25 TO 29= ~ 46.78 + 6.56*PERFORM + 0.23*DRAFT
VALUE OVER REPLACEMENT FROM AGES 25 TO 29= ~ 1930 + 866*PERFORM + 31*DRAFT
If the the Perform and draft numbers are of the same scale ( and as far as I can tell they are) their performace at age 24 is at least 25 times more significant than the draft position. In fact if you totally ignored the draft position it would not significantly impact your projections.
So your equations reinforce prof. Berri's conclusion.
November 24th, 2009 at 12:41 pm
A related question: the famous Thaler-Massey study found that draft pick did predict performance, but that even after accounting for performance the high picks produced less surplus value. A key assumption of theirs was that players with 1-8 starts contributed the same value regardless of draft status, as did all players with 9+ starts (who didn't make the Pro Bowl). If your data permits -- and you have the interest! -- could you see if high and low draft picks do contribute the same value over replacement within those categories? Would be very interesting....
November 24th, 2009 at 12:41 pm
This feels so phenomenally wrong to me... It feels like there's an implicit assumption that the choice of who gets the start is random. It's a conscious choice by people who know rather a lot about QBs. All those late round QBs who never got to start an NFL game are excluded from the data set, and those fellows didn't end up on that list randomly. I'd bet they're significantly worse on average than the players who do get to dirty up their uniforms. So how much of the "draft position doesn't matter (much)" is due to coaches not randomly selecting a starting QB but actually picking the guy they think would perform better?
November 24th, 2009 at 1:11 pm
Arturo, The draft variable ranges from about 3 (Mr. Irrelevant) to 73 (1st pick in the draft). The Perform variable is measured in yards per attempt relative to league average, so in most cases we're talking about -2.00 to 2.00. That's why the weight on the PERFORM variable is so much heavier than the weight on the DRAFT variable, not because the draft variable is unimportant.
November 24th, 2009 at 1:13 pm
Guy, I'm not sure if I follow your question. That said, I did a lengthy examination of draft pick vs. NFL production, which disagreed with the Massey-Thaler paper; JKL linked to my research in this post (in #2 on his list). Not sure if that answers your question.
November 24th, 2009 at 1:14 pm
Mattie,
Yes, there's no question about that. I don't understand why that is not implicit to everyone.
November 24th, 2009 at 1:41 pm
Chase beat me to the punch, Arturo. But I'll go ahead and give you an example. The 1st overall pick used a DRAFT value of 73.2. The 32nd overall pick is roughly 30. If you plug those numbers into the formula, and assign both the same PERFORM at age 24, you will see that the first overall pick is expected to start almost 17 more games than the late first rounder. That is equal to a scenario where two quarterbacks were drafted at the same draft position, but one is 2.0 yards per play better at age 24. So, both are highly significant.
Guy, I think Yards over Replacement Value is showing how good the player is, and not just the fact they were playing. Chase uses a 75% figure for each year, so there are a fair portion of quarterbacks who rate below replacement level in a given year. While I don't have the specific attempts broken out, we can look at RV/G to get a per game production breakdown to see if higher picks perform better in the future. Here are projected RV/G numbers for hypothetical two quarterbacks, one was drafted first overall, the other at pick #100. I list what their projected RV/G would be from age 25-29, assuming they had equal "PERFORMANCE" on a per play basis at age 24.
2.0 yards better than average at age 24: QB1- 77.3; QB100- 65.1
1.0 yards better than average: QB1-72.1; QB100-57.4
league average: QB1- 66.0; QB100-47.7
-1.0 yards worse than average: QB1-58.4; QB100-35.0
-2.0 yards worse than average: QB1-48.8; QB100- 17.9
You'll notice the decay rate is higher for the later round quarterback as performance decreases. Some of this is probably due to "partial games" where the qb comes off the bench as a backup, so on a per play basis it might not be so pronounced.
But notice that a league average high draft pick is projected to perform about as well as an elite (2.0 yards better than average) late draft pick in terms of RV/G over the next 5 years, and the projected games are equal (66.0 versus 65.1).
Does that answer your question?
November 24th, 2009 at 5:46 pm
Jason: yes, thanks, much appreciated.
Chase: thanks, I'll check out your study.
Economists have created quite a cottage industry, purporting to show that decision-makers in professional sports get major decisions wrong. But as best I can tell, every one of these -- with the notable exception of the Romer 4th down study -- has subsequently been debunked by people with real subject matter expertise. Yet the studies keep on coming.....
November 24th, 2009 at 7:04 pm
Hi Jason. Good post! Our blog was one of the sources Pinker used in his critique of Gladwell, and we just posted a reply to the latest exchange between he two on our blog, which is at: http://blog.criteriacorp.com/blog/bid/11115/Gladwell-vs-Pinker The Berri choice to exclude QBs who didn't play five years in the league is a pretty fundamental error to make. Your smaller cutoff of 100 or 200 passes makes much more sense. Essentially, Berri is saying let's examine how effectively a selection system (the draft) works by looking at only those QBs who turned out to be most successful (ie played 5 years) This is a pretty cut and dry case of manipulating, or at least mangling, a data set, in my opinion. Incidentally, we linked to PFR in our blog post as the source for the stats used. Good stuff!
November 24th, 2009 at 8:13 pm
@MattieShoes- I understand what you are saying, and to an extent, you are right. If you have talent, you will eventually find your place. But, the people who "know rather a lot about QBs" chose players in their respective rounds for a reason. The fact that #1s fail, and undrafted players sometimes flourish points to the fact that these experts don't know as much as they wish think they know.
Does performance really rule all? Brees is great, but was pushed aside for Rivers because you can't spend millions of dollars on a backup. Leinart was handed the starting job over Warner simply because he was drafted highly, not because of anything he had actually earned (kudos for Arizona for righting that wrong. They were rewarded with a Super Bowl birth).Even in Oakland, Jeff Garcia, a 4-time Pro Bowler coming off 3 straight seasons with a 90+ QB rating couldn't get even a fair tryout to start over JaMarcus Russel and his 65.5 career rating.
The better player doesn't always earn playing time, and that is unfortunate. Imagine if Tom Brady was drafted in the 6th round by Cleveland in 2000 instead of by New England. Tim Couch, last year's savior was entrenched in the starting spot. After all, they spent a 1st round pick on him and invested a bunch of money. The coaches aren't even looking at Brady as anything more than a 3rd stringer...if he even makes the team. I mean, he wasn't even a starter in college.
We can all agree that Couch never became a star, and couldn't really even pass as a viable starter, and that Brady is one of the better QBs in the league. Now, do you think that there is anything that Tom Brady could have possibly shown the Cleveland coaching staff (assuming the staff if competent, which in itself is a stretch) during practice with the 3rd team offense that would make them start him over Couch? Never. It took Cleveland 59 starts to realize that he wasn't the answer. (Unrelated-Poor guy. Nobody deserves to be cheered by their home fans when they get seriously injured.)
Brady never earned the starting spot in New England because he proved himself better than Bledsoe. He got it because he proved himself less injured in week 2 of the 2001 season. As great as a coach as Bellichick has shown him self to be (grrr), I doubt they would have given him playing time over Bledsoe based soley on what he saw in practice. You never really know what you have until the player sees the field. And a lot of times, guys that would be worthy never get the opportunity. And some guys get chance after chance simply off the hope that they someday live up to their draft position (Couch-59 starts, David Carr-79, Joey Harington-76)
November 24th, 2009 at 8:44 pm
Since Cleveland came back in the league, their win % with Couch starting is still higher than it is without him.
November 24th, 2009 at 10:40 pm
Roby,
1. Brees was a UFA with a shoulder injury. Even though he was coming off 2 good years, SD would have been wary of keeping him, at least with a big new contract, regardless of the presence of Rivers, who turned out to be pretty damn good himself. For the record, Brees's deal in NO was essentially a 1 year "prove-it" deal with very little risk for the team if his injury turned out to be chronic. For a counter-example, check out the Browns who re-signed Derek Anderson coming off his fluke good year even though they had just drafted Brady Quinn in the first round. Alternatively, would you say the Bengals were wrong to bench Kitna coming off a pretty good year for the untested Palmer?
2. Warner was OK in his first year in Arizona but nothing special. When Leinart won the job, nobody said "They're benching Warner? That's crazy!" (His YPA was actually lower with Arizona than it was in the previous alleged disaster year with the Giants.) Nobody foresaw Warner's reemergence as a star in his old age.
3. While Brady could not have won the job outright over Couch in Cleveland, Couch wasn't exactly Iron Man Favre. Brady would eventually have won the job there the same way he won it in NE, due to injury, the same way Romo won his job in Dallas (also over Bledsoe). That's how a lot of guys win jobs at a lot of positions in every sport. Ever hear of Wally Pipp? It's not unique to QBs.
4. I won't defend Couch or Carr as necessarily deserving of all the chances they got, but Harrington got that many starts due to pure luck. After he failed in Detroit (55 starts over 4 years), his career as a starter should have been over. Instead, he went to Miami to be the backup but Culpepper quickly got hurt. That gave him another 11 starts as an injury sub in Miami over 1 year. Miami was so impressed that they let him go to Atlanta, where Vick got pinched for dogfighting shortly thereafter. Unfortunately, the Falcons had just traded Matt Schaub away, and it was too late in the offseason to pursue a viable starter. Just like that, Joey got 10 more starts. Atlanta, just like Miami, let him go after just 1 year. So you see, he only got 55 legit starts over 4 years with his original draft team, plus 21 more thanks to being in the right place at the right time. Joey Harrington is the luckiest bad QB I can name, in terms of falling into starting jobs after everyone had given up on him as a starter.
November 24th, 2009 at 10:44 pm
Actually, I may have messed up saying Romo won the job from Bledsoe due to injury. It may have been a coaching decision, which just strengthens the point that coaches will sometimes give the undrafted guy a shot, even over a former #1 overall pick and 4 time Pro Bowler.
November 25th, 2009 at 12:10 am
Alright, I counted 2300+ words in the blog with keywords Rick Mirer, Tim Couch, Joey Harrington, get opportunities without merit, and NFL quarterbacks.
Not a single mention of David Carr.
I do believe it is time to start tossing David Carr into the same conversation.
I am just saying that's all.
November 25th, 2009 at 8:50 am
Chase/Jason:
Ok. I read the Perfomance value from the wrong colummn. But the results are really interesting. I ran the equations in excel and for a bad 1st,2nd or 3rd pick (-2 Yards) I get 46.18 42.67 40.72 starts which is better that any Replacement level Qb drafted higher than the 99th pick. So the value invested in the pick by the team does seriously skew how long they'll live with a bad result (which makes practical real world sense).
Alternatively, a good qb drafted as mr. irrelevant (2 Yards over Replacement, Draft Value of 5) is predicted to start about 54 games. So talent will be found if it's there (again given the amount of injuries and the nature of the league this seems to make practical sense)
November 25th, 2009 at 9:18 am
"So the value invested in the pick by the team does seriously skew how long they'll live with a bad result."
I'm sure teams give their high draft picks more time to fail. But that could be perfectly efficient, given the difficulty of establishing a QB's true talent and the potential for growth/improvement. And in some cases, the immediate alternative may not clearly be superior. It seems very hard to distinguish this evaluation process from an inefficient failure to recognize sunk costs and move on.
November 25th, 2009 at 9:45 am
It's undeniable that bad highly drafted QBs get more opportunities than bad low-drafted QBs. What's at issue is whether a significant proportion of good low-drafted QBs get missed entirely. I really don't think that's the case - certainly not to the degree that would be required to validate Berri's conclusions.
Furthermore, it's daft to suppose that it is possible to evaluate a quarterback as accurately from at most 100 or so hours of college game tape and one or two workouts as it is from one or more seasons' worth of daily practice. Belichick may not have intended to start Brady as early as he did, but he had already earmarked him as Bledsoe's long term replacement before he ever took the field. If young third stringers don't get evaluated, how is Hoyer the Patriots' current back-up? How did Flynn beat out Brohm?
As to David Carr, he undoubtedly got more playing time than he deserved, but none of the decisions to start him in any given season seem irrational. Through 2004 (the third year of his career) his development curve appeared well within the bounds of what might be expected for a reasonably good player drafted by an expansion team and thrown in from day 1 behind a lousy line with poor receivers (Johnson was a very good player by 2004, but there was no talent behind him, and in his 2003 rookie season he was good for a rookie, but far from elite). 2005 was a disaster, but pretty much every element of the Texans was a disaster that year - even Johnson missed time and was banged up and ineffective when he did play. The offensive co-ordinator was fired about 2 weeks into the season. Carr looked bad, but it wasn't unreasonable to think that might have been mostly down to the atrocious supporting cast (see Cutler, Jay). It wasn't until 2006, with a non-farcical line and a proven scheme, that it finally became clear that whether Carr had ever had any potential or not, he was sure as hell a bust now. Kubiak probably could have benched Carr for Rosenfels at that point, and there might have been some improvement, but it's not as if any sane person now or ever believes Sage Rosenfels might be able to be a franchise quarterback. Leaving Carr in the team to prove to the owner that he really couldn't get it done was probably the right move.
November 25th, 2009 at 2:21 pm
A player's draft position is probably a reasonable estimation of his college performance as well as his scouting report. When projecting the performance of an NFL player in his first or second year, you would have to find the optimal weight of pre-NFL projection vs. his actual NFL performance so far.
My question is: at what point should a player's college performance and scouting report (i.e. draft position) be *completely* disregarded? After how many games/years? Is there any evidence that NFL teams as a group use draft position (in the "potential" sense, not the "financial" sense) to determine playing time longer than they should?
For example, baseball projection systems use a 5/4/3 weighting of the last three years to project performance, using minor league equivalency stats where appropriate, but performance older than three years is useless.
November 26th, 2009 at 1:54 pm
Maybe never.
Elway was the number one choice in 1983, but wasn't really a top flight passer until 1993
November 30th, 2009 at 10:43 pm
Thanks. That's a much more insightful analysis than the quick one I did based on your Pro-Football-Reference.com data that got Malcolm Gladwell so peeved.
December 19th, 2009 at 11:09 pm
I wonder if in order to get around the selection problem you could use preseason stats from the quarterbacks' rookie years. Since all rookie quarterbacks get playing time it would allow you to compare all of their performances rather than just the ones who were picked to play in the regular season. I know preseason performance isn't a perfect indicator of future performance and that the quality of the opponent would vary widely.
January 9th, 2010 at 1:52 am
That this issue deserves any serious debate at all underscores the more interesting but difficult question: is it worth investing high, or very high, draft picks in QB's?
To answer this would likely require some estimation of a QB's value relative to other positions. Intuitively, I would expect it would be like drafting a pitcher in baseball - lower probability of working, higher reward if it does. QB's have a more diverse set of abilities - judgement and decision making,leadership, release, various aspects of throwing, maturity, ability to hold up under intense public scrutiny, etc. Quite understandable that evaluation of these is more difficult than some other positions.
A simple model might be to correlate Pro-Bowl selections of various positions with draft selection levels over a long period of time