Pro Football Reference Blog

I'm sure most of you have read about the NFL draft study conducted by economists Cade Massey and Richard Thaler. If not, here is the short version: Massey and Thaler took about a decade's worth of statistics on the NFL draft and concluded that the best picks to have (in a particular sense that I'll discuss in a second) are those in the 25--50 range. They claim that the first pick is, all things considered, the least valuable pick in the first round and that the Raiders should be willing to swap their pick for the #30 pick. Straight up.

How can this be?

Massey and Thaler are not claiming that the players available at #30 will turn out to be better than those available at #1. They are instead claiming that the players available at #30 will likely turn out to be only a little worse than those available at #1, but that they will cost a lot less. From their paper:

We think that while our results are surprising, they are plausible. We suspect that some teams have not fully come to grips with the implications of the salary cap, a relatively new innovation. Buying expensive players, even if they turn out to be great performers, imposes opportunity costs elsewhere on the roster. Spending $10 million on a star quarterback instead of $5 million on a journeyman implies having $5 million less to spend on offensive linemen to block or linebackers to tackle.

In other words, the #30 pick is not better than the #1 pick. But the player picked at #30, together with some other player or combination of players you can sign with the money you didn't have to pay the #1 pick, is better than the player picked at #1.

They state all their findings in terms of "surplus value" and the units used are dollars, so this seems to be advice for the owners: pick #30 will provide you a greater return on your investment than pick #1 will.

But what if the units were wins instead of dollars? Theoretically, wins and dollars should align very closely, and that seems to be the assumption that Massey and Thaler implicitly make. But does it really play out that way? In reality, does the extra surplus value gained by the teams at the bottom of the first round get translated (somehow) to wins on the football field? In other words, does this paper speak to fans, or just to the owners?

As is always the case when academics publish a paper that implies that the conventional wisdom is drastically wrong, the main counterargument is a simple and somewhat persuasive one. Namely, the most highly regarded and well-trained experts in a multi-billion dollar enterprise can't possibly be that wrong. They must know something that the study fails to capture. Massey and Thaler anticipate this criticism and address it with some convincing counterarguments of their own, but there is still the possibility that they are missing something.

My wild guess as to what that something might be is as follows. Maybe it's difficult or impossible to turn that theoretical surplus value you'd get by trading down into a real tangible value on the playing field between the time the draft picks are assigned and the time the next season starts. In other words, yes, the Raiders could trade #1 straight up for #30; the quality of player they'd end up with declines just a little, but they save a ton of money. Massey and Thaler claim this is a good thing. But what are they going to do with that extra money? What if there are no free agents available who are worth what they're asking? What if none of their current young standouts are amenable to signing an extension at a reasonable price?

Sure, if there was an All-Pro offensive lineman on the market and signable for the difference in salary between JaMarcus Russell and Drew Stanton, the Raiders should go that route. But I'm not aware of any such lineman. They could have upgraded from Dominic Rhodes and Tony Stewart to Jamal Lewis and Daniel Graham, or something like that, but how sure are we that Russell/Rhodes/Stewart won't provide better bang for the buck than Stanton/Lewis/Graham would? At $30 million for five years, how much surplus value can Daniel Graham possibly have? According to Thaler and Massey's data, the #1 pick doesn't generate as much surplus value as the #30 pick, but it does offer a substantial amount of surplus value. So why not pay a bit of a premium for the guy with the better chance of being a star, even if it's only slightly better?

I don't know if I believe that theory or not. I'm just throwing out a possibility. But whatever's going on, the real test of Massey and Thaler's theory --- from a fan's point of view --- is whether or not teams with the high draft picks are more likely to improve than teams with lower picks. If they're correct, then bad teams that for whatever reason are not picking at the top of the first round should improve more quickly than similarly bad teams that are "stuck" with their early picks.

I'm going to try to address that question in detail in a future post. For now, let me set up the framework of my methodology while providing some interesting draft-related trivia.

On page 52 of their paper, Massey and Thaler show a graph of surplus value by draft slot. It rises from pick #1 until about pick #43, then it starts to fall again and keeps falling until the end of the draft. Again, they claim that, dollar for dollar, the end of the first round through the second round are the best picks to have. Massey and Thaler are very careful to point out that they don't have much confidence that this represents the exact relationship between pick number and surplus value:

Clearly, considerable caution should be used in interpreting this curve; it is meant to summarize the results simply. We do not have great confidence in its precise shape.

Nonetheless, it does represent their best guess in some sense, so that's what I'm going to use. But bear in mind that this study necessarily carries the same disclaimer.

The first thing I did was to eyeball that graph and make a numerical estimate of the value of each draft slot. Then I went through every team of the free agent era (1993--present) and calculated how valuable their draft slots were.

For example, the team with the best overall collection of slots was the 1997 Dolphins; they had 14 total picks, including four in the third round. By my accounting, the value of the Dolphins draft was 170.6 on a scale where league average is necessarily 100. To be more precise, I took the total Massey-Thaler value of all picks in the draft, then I divided that by the number of teams in the league at the time. Then I divided the total Massey-Thaler value of the Dolphins picks by the league average. Finally, I multiplied it by 100 to make it easier on the eyes. To summarize: 170.6 means that they had 70.6% more draft value than an average team.

The team with the lowest total draft value was... Ah, I'll let you guess. But I'll give you a hint: they only had one pick in the whole draft, and it was early in the first round.

Just for fun, I also made the same calculation as above, but with the NFL draft pick value chart, as provided by Gil Brandt here. Some interesting teams:

• The 2004 Titans had thirteen total picks. They had no first-round picks, but had three second-rounders and two third-rounders. The pick value chart says their draft slots were worth 99.7, almost precisely average. The Massey-Thaler chart says it was 165.0, the third-highest figure of the period.
• The 1998 Jets were similar to the Titans above. According to the pick value chart, their draft slots were worth about half that of an average team. Thaler and Massey say they were worth about 40% more than an average team.
• The 1997 Seahawks had picks #3 and #6 in the first round, but no other pick until round five. The pick chart says their picks were worth 191, about twice that of an average team. The Massey-Thaler chart says they were only a 58, just over half that of an average team.
• The 1998 Chargers' picks were worth 68.6 according to Massey and Thaler, but 148.3 according to the pick chart. Do you remember that draft, Charger fans?

In a forthcoming post (hopefully later this week, but no guarantees), I'll align this draft value data with the team records to see if it plays a role in team improvements and declines.