Posted by Doug on April 22, 2008
A few weeks ago, I asked the fine folks at the footballguys message board to rank the following list of players according to overall career (not peak) value.
The idea was to take players from a variety of positions, decades, and overall quality levels, and see if my method put them in roughly the same order as the fan consensus. After posting the list, I realized that I should have included a few more offensive linemen. Munoz is the only one, and he's easy to rate. The omission of quarterbacks was intentional; I'll talk a little about that later.
I realize this isn't a tightly controlled scientific survey, but here is the consensus of the 13 sets of rankings that were offered at the time I tallied them (a few more sets of rankings trickled in later; they're not included here):
PT HI LO ================================== Anthony Munoz 15 1 2 Bruce Smith 25 1 3 Cris Carter 59 3 6 Aeneas Williams 67 3 8 Neil Smith 72 3 9 Dan Hampton 76 2 9 John Lynch 93 3 11 Clay Matthews 96 3 13 Dale Carter 121 3 14 Warrick Dunn 123 6 13 Seth Joyner 136 7 13 Kevin Carter 153 10 14 Eric Moulds 164 10 15 Dexter Coakley 171 11 15 Roman Phifer 189 12 15
PT is the number of points each candidate received, with a first-place voting counting one point, a second-place vote counting two points, and so on. HI and LO are the best and worst rankings anyone submitted for that player. So somebody ranked Warrick Dunn 6th, somebody else ranked him 13th, and everyone else was in between those two (inclusive).
Just from looking at the HI and LO columns, it's pretty clear that there is no such thing as a consensus ranking. There was agreement that Munoz and Smith were the clear 1 and 2, and that Phifer, Coakley, and Moulds were near the bottom. But aside from that there were lots of differing opinions, as would be expected.
When I compute my career approximate value estimates for each of those players, here are the rankings I get:
Bruce Smith 222 Anthony Munoz 175 Clay Matthews 141 Cris Carter 139 Dan Hampton 125 John Lynch 122 Neil Smith 120 Aeneas Williams 116 Warrick Dunn 114 Kevin Carter 113 Seth Joyner 105 Eric Moulds 97 Roman Phifer 89 Dexter Coakley 82 Dale Carter 75
The numbers are not in any meaningful units. I've only included them so you can see , for example, that there is a big gap between Munoz and Matthews but a small gap between Matthews and Cris Carter.
With the exception of Dale Carter, my algorithm wasn't the high or the low vote on any of these 15 players.
It's interesting that the strong consensus among the message board voters was Munoz over Bruce Smith, and my system says the opposite. Here is what I wrote about that over at the message board:
My system says Smith over Munoz. I'm not going to argue that as being right (and the SP consensus wrong), but I will explain why it turned out that way. It's because Smith played almost 100 games more than Munoz did. That's about six seasons worth. In his worst six seasons, Smith had 34 sacks. While Munoz may have been better than Smith in his prime, I don't see any way he was so much better that if you match Munoz' 13 seasons against Smith's 13 best seasons, that the difference is worth more than 5 or 6 seasons of an average DT/DE starter.
But all that is quibbling. Let's face it: none of us has the foggiest idea whether Smith or Munoz had the more valuable career. And no system will change that. But this thread has been immensely valuable to me, because it's given me a lot of insight about the process by which people mentally evaluate players. What the Munoz-over-Smith consensus tells me is that, roughly, people don't see much difference between a 13-year HoF career and a 19-year HoF career. We don't/can't mentally compute the value of Smith's 19 great seasons against Munoz's slightly greater (on average) 13 seasons. We just say something like, "they were both great and had long careers, but Munoz was a little greater."
Clay Matthews is another guy that my system likes a little better than the consensus for essentially the same reason. He was a 16-year starter and a darn good one. But it's really tough to factor in how that compares to a "mere" 12-year starter like Neil Smith who was probably a bit better in his prime.
For phase two of my little experiment, I presented the voters with five pairs of players, and asked them to tell me which one had the better career in each case:
The schtick here is that each pair is rated essentially equal by my system. Although there is certainly disagreement, we all feel pretty comfortable placing skill-position guys into historical context. So I took some defensive players and linemen with AV ratings very similar to some famous skill guys in order to see if the fan raters also, as a group, thought they were about equal. The results:
Smith over Singletary 10-3
Green over Carter 11-2
Shields over Barber 10-2
O'Neal over Fryar 12-1
Steussie over Duper 10-3
Woodson over George 8-5
So in every case but one, the fan consensus sided with the non-skill guy. While I don't want to overreact to a small sample, the main goal of this thing is to generate numbers that match perception. So I may have to tweak up the value on offensive linemen just a little. Since my algorithm starts by giving each offense a fixed number of points to divvy up, tweaking up the offensive lineman percentage will necessarily tweak down the ratings on skill players, which will have the effect of lowering their value relative to not only offensive linemen, but defensive players as well.
So, question 1: based on what you see here, and in part III, do I have (non-QB) skill players rated too high?
Now, about quarterbacks....
The folks over there were having a good time rating players, so I gave them another list:
All these players played (or have so far played) for between 12 and 14 seasons. The main thing I wanted to see was where the rankers put Brunell and McNair. Here were the results:
PT HI LO ================================== Derrick Brooks 19 1 4 Marvin Harrison 23 1 4 Shannon Sharpe 35 1 9 Gary Zimmerman 42 2 7 Steve Wisniewski 68 3 12 Pat Swilling 74 5 12 Jerome Bettis 81 1 14 Zach Thomas 83 5 14 LeRoy Butler 92 3 13 Steve McNair 97 6 14 Simeon Rice 103 5 13 Darryl Talley 110 5 14 Mark Brunell 122 4 15 Aaron Glenn 122 4 15 Brad Hopkins 129 9 15
This was a much more tightly bunched group of players. My system says:
Derrick Brooks 184 Marvin Harrison 156 Zach Thomas 145 Steve McNair 138 Shannon Sharpe 135 Mark Brunell 132 Gary Zimmerman 127 Pat Swilling 118 Simeon Rice 105 Jerome Bettis 104 LeRoy Butler 103 Steve Wisniewski 102 Brad Hopkins 95 Darryl Talley 93 Aaron Glenn 86
First note that the fans ranked all three non-QB skill guys (Harrison, Sharpe, and Bettis) at least as high as my system did.
But, they did not like the QBs, which I somewhat expected. My p-f-r blog colleague JKL did a nice job of elaborating a case for Brunell, which more or less mirrors my general defense of high quarterback ratings in my system:
If you are going to base it on pro bowls and all pro selections, you will necessarily overvalue the line positions at the expense of the quarterback. If a quarterback has an off year or an injury that causes some missed games, he is likely not making the pro bowl. An offensive lineman, has he made at least three in the past, no problem.
Quarterback is, if not the most important position, the most variable position in terms of performance, and an above average QB is, in my opinion, worth more than an above average performer elsewhere.
Here's a thought experiment. You are sitting at the 8th overall pick and go on the clock. Mysteriously, a strange man in a black cloak appears and lets you look into a crystal ball to see the careers of the players you are considering. Player A is a quarterback who will be the next Mark Brunell. Player B is an offensive guard who will be the next Steve Wisniewski, and Player C is a safety who will be the next Leroy Butler.
Assuming you don't have enough picks to trade up and take all three as soon as possible, and have to take just one, which one is it?
For me, it's pretty much a no brainer, assuming I dont already have Peyton Manning or Tom Brady on my roster (and if I am picking 8th that is likely true). I run the Brunell card up as fast as I can. You get the equivalent a player who sat for two years only because he was behind the guy who would go on to break the all time touchdown record. A guy who at age 25, starting for an expansion team, put up respectable numbers with junk at wide receiver and a makeshift line around rookie Boselli. The next year, when Jimmy Smith and Keenan McCardell became starters and the line was improved, he had his breakout, and would post a top 10 in Adjusted Yards per Attempt and Passer Rating in 6 of the next 7 seasons, through age 32. And those pass efficiency stats don't include his added ability as a runner, where he had at least 200 yards rushing every season except for one by age 32.
I know he "only" made three pro bowls. You give me a quarterback who is going to hit the ground running as a starter, be a top 5 quarterback in terms of combined pass efficiency and running each year until he is in his early 30's, and I will take that every time.
The list Doug put out is a good one. Not a bad name on there, so I don't want to denigrate any particular player. I'll just say that I don't see how a quarterback who is that productive and is a key part in several productive, efficient offensive seasons can be ranked at the very bottom.
I'm inclined to agree with JKL's larger point. LeRoy Butler, for example, may rank higher on the all-time safety list than Brunell ranks on the all-time quarterback list, but the importance of the quarterback position relative to safety (and any other position) justifies Brunell's ranking ahead of Butler. Anyone who is capable of starting at quarterback for seven above-average offensive teams had an extremely valuable career.
Do you agree?
Eventually, I will spell out the entire AV algorithm, but I'm still trying to get it tuned, and posts like these are part of that process.