Pro Football Reference Blog

Links to part I, part II, part III,

A few weeks ago, I asked the fine folks at the footballguys message board to rank the following list of players according to overall career (not peak) value.

Dale Carter
Cris Carter
Kevin Carter
Dexter Coakley
Warrick Dunn
Dan Hampton
Seth Joyner
John Lynch
Clay Matthews
Eric Moulds
Anthony Munoz
Roman Phifer
Bruce Smith
Neil Smith
Aeneas Williams

The idea was to take players from a variety of positions, decades, and overall quality levels, and see if my method put them in roughly the same order as the fan consensus. After posting the list, I realized that I should have included a few more offensive linemen. Munoz is the only one, and he's easy to rate. The omission of quarterbacks was intentional; I'll talk a little about that later.

I realize this isn't a tightly controlled scientific survey, but here is the consensus of the 13 sets of rankings that were offered at the time I tallied them (a few more sets of rankings trickled in later; they're not included here):

                      PT  HI  LO
==================================
Anthony Munoz         15   1   2
Bruce Smith           25   1   3
Cris Carter           59   3   6
Aeneas Williams       67   3   8
Neil Smith            72   3   9
Dan Hampton           76   2   9
John Lynch            93   3  11
Clay Matthews         96   3  13
Dale Carter          121   3  14
Warrick Dunn         123   6  13
Seth Joyner          136   7  13
Kevin Carter         153  10  14
Eric Moulds          164  10  15
Dexter Coakley       171  11  15
Roman Phifer         189  12  15

PT is the number of points each candidate received, with a first-place voting counting one point, a second-place vote counting two points, and so on. HI and LO are the best and worst rankings anyone submitted for that player. So somebody ranked Warrick Dunn 6th, somebody else ranked him 13th, and everyone else was in between those two (inclusive).

Just from looking at the HI and LO columns, it's pretty clear that there is no such thing as a consensus ranking. There was agreement that Munoz and Smith were the clear 1 and 2, and that Phifer, Coakley, and Moulds were near the bottom. But aside from that there were lots of differing opinions, as would be expected.

When I compute my career approximate value estimates for each of those players, here are the rankings I get:

 Bruce Smith              222
 Anthony Munoz            175
 Clay Matthews            141
 Cris Carter              139
 Dan Hampton              125
 John Lynch               122
 Neil Smith               120
 Aeneas Williams          116
 Warrick Dunn             114
 Kevin Carter             113
 Seth Joyner              105
 Eric Moulds               97
 Roman Phifer              89
 Dexter Coakley            82
 Dale Carter               75

The numbers are not in any meaningful units. I've only included them so you can see , for example, that there is a big gap between Munoz and Matthews but a small gap between Matthews and Cris Carter.

With the exception of Dale Carter, my algorithm wasn't the high or the low vote on any of these 15 players.

It's interesting that the strong consensus among the message board voters was Munoz over Bruce Smith, and my system says the opposite. Here is what I wrote about that over at the message board:

My system says Smith over Munoz. I'm not going to argue that as being right (and the SP consensus wrong), but I will explain why it turned out that way. It's because Smith played almost 100 games more than Munoz did. That's about six seasons worth. In his worst six seasons, Smith had 34 sacks. While Munoz may have been better than Smith in his prime, I don't see any way he was so much better that if you match Munoz' 13 seasons against Smith's 13 best seasons, that the difference is worth more than 5 or 6 seasons of an average DT/DE starter.

But all that is quibbling. Let's face it: none of us has the foggiest idea whether Smith or Munoz had the more valuable career. And no system will change that. But this thread has been immensely valuable to me, because it's given me a lot of insight about the process by which people mentally evaluate players. What the Munoz-over-Smith consensus tells me is that, roughly, people don't see much difference between a 13-year HoF career and a 19-year HoF career. We don't/can't mentally compute the value of Smith's 19 great seasons against Munoz's slightly greater (on average) 13 seasons. We just say something like, "they were both great and had long careers, but Munoz was a little greater."

Clay Matthews is another guy that my system likes a little better than the consensus for essentially the same reason. He was a 16-year starter and a darn good one. But it's really tough to factor in how that compares to a "mere" 12-year starter like Neil Smith who was probably a bit better in his prime.

For phase two of my little experiment, I presented the voters with five pairs of players, and asked them to tell me which one had the better career in each case:

Emmitt Smith or Mike Singletary
Cris Carter or Darrell Green
Tiki Barber or Will Shields
Irving Fryar or Leslie O'Neal
Mark Duper or Todd Steussie
Eddie George or Darren Woodson

The schtick here is that each pair is rated essentially equal by my system. Although there is certainly disagreement, we all feel pretty comfortable placing skill-position guys into historical context. So I took some defensive players and linemen with AV ratings very similar to some famous skill guys in order to see if the fan raters also, as a group, thought they were about equal. The results:

Smith over Singletary 10-3
Green over Carter 11-2
Shields over Barber 10-2
O'Neal over Fryar 12-1
Steussie over Duper 10-3
Woodson over George 8-5

So in every case but one, the fan consensus sided with the non-skill guy. While I don't want to overreact to a small sample, the main goal of this thing is to generate numbers that match perception. So I may have to tweak up the value on offensive linemen just a little. Since my algorithm starts by giving each offense a fixed number of points to divvy up, tweaking up the offensive lineman percentage will necessarily tweak down the ratings on skill players, which will have the effect of lowering their value relative to not only offensive linemen, but defensive players as well.

So, question 1: based on what you see here, and in part III, do I have (non-QB) skill players rated too high?

Now, about quarterbacks....

The folks over there were having a good time rating players, so I gave them another list:

Jerome Bettis
Derrick Brooks
Mark Brunell
LeRoy Butler
Aaron Glenn
Marvin Harrison
Brad Hopkins
Steve McNair
Simeon Rice
Shannon Sharpe
Pat Swilling
Darryl Talley
Zach Thomas
Steve Wisniewski
Gary Zimmerman

All these players played (or have so far played) for between 12 and 14 seasons. The main thing I wanted to see was where the rankers put Brunell and McNair. Here were the results:

                      PT  HI  LO
==================================
Derrick Brooks        19   1   4
Marvin Harrison       23   1   4
Shannon Sharpe        35   1   9
Gary Zimmerman        42   2   7
Steve Wisniewski      68   3  12
Pat Swilling          74   5  12
Jerome Bettis         81   1  14
Zach Thomas           83   5  14
LeRoy Butler          92   3  13
Steve McNair          97   6  14
Simeon Rice          103   5  13
Darryl Talley        110   5  14
Mark Brunell         122   4  15
Aaron Glenn          122   4  15
Brad Hopkins         129   9  15

This was a much more tightly bunched group of players. My system says:

Derrick Brooks           184
Marvin Harrison          156
Zach Thomas              145
Steve McNair             138
Shannon Sharpe           135
Mark Brunell             132
Gary Zimmerman           127
Pat Swilling             118
Simeon Rice              105
Jerome Bettis            104
LeRoy Butler             103
Steve Wisniewski         102
Brad Hopkins              95
Darryl Talley             93
Aaron Glenn               86

First note that the fans ranked all three non-QB skill guys (Harrison, Sharpe, and Bettis) at least as high as my system did.

But, they did not like the QBs, which I somewhat expected. My p-f-r blog colleague JKL did a nice job of elaborating a case for Brunell, which more or less mirrors my general defense of high quarterback ratings in my system:

If you are going to base it on pro bowls and all pro selections, you will necessarily overvalue the line positions at the expense of the quarterback. If a quarterback has an off year or an injury that causes some missed games, he is likely not making the pro bowl. An offensive lineman, has he made at least three in the past, no problem.

Quarterback is, if not the most important position, the most variable position in terms of performance, and an above average QB is, in my opinion, worth more than an above average performer elsewhere.

Here's a thought experiment. You are sitting at the 8th overall pick and go on the clock. Mysteriously, a strange man in a black cloak appears and lets you look into a crystal ball to see the careers of the players you are considering. Player A is a quarterback who will be the next Mark Brunell. Player B is an offensive guard who will be the next Steve Wisniewski, and Player C is a safety who will be the next Leroy Butler.

Assuming you don't have enough picks to trade up and take all three as soon as possible, and have to take just one, which one is it?

For me, it's pretty much a no brainer, assuming I dont already have Peyton Manning or Tom Brady on my roster (and if I am picking 8th that is likely true). I run the Brunell card up as fast as I can. You get the equivalent a player who sat for two years only because he was behind the guy who would go on to break the all time touchdown record. A guy who at age 25, starting for an expansion team, put up respectable numbers with junk at wide receiver and a makeshift line around rookie Boselli. The next year, when Jimmy Smith and Keenan McCardell became starters and the line was improved, he had his breakout, and would post a top 10 in Adjusted Yards per Attempt and Passer Rating in 6 of the next 7 seasons, through age 32. And those pass efficiency stats don't include his added ability as a runner, where he had at least 200 yards rushing every season except for one by age 32.

I know he "only" made three pro bowls. You give me a quarterback who is going to hit the ground running as a starter, be a top 5 quarterback in terms of combined pass efficiency and running each year until he is in his early 30's, and I will take that every time.

The list Doug put out is a good one. Not a bad name on there, so I don't want to denigrate any particular player. I'll just say that I don't see how a quarterback who is that productive and is a key part in several productive, efficient offensive seasons can be ranked at the very bottom.

I'm inclined to agree with JKL's larger point. LeRoy Butler, for example, may rank higher on the all-time safety list than Brunell ranks on the all-time quarterback list, but the importance of the quarterback position relative to safety (and any other position) justifies Brunell's ranking ahead of Butler. Anyone who is capable of starting at quarterback for seven above-average offensive teams had an extremely valuable career.

Do you agree?

Eventually, I will spell out the entire AV algorithm, but I'm still trying to get it tuned, and posts like these are part of that process.