Posted by Doug on October 11, 2008
As you probably know, I am affiliated not just with pro-football-reference.com but also with a fantasy football info site called footballguys.com.
Although I do think every reader of this blog who plays fantasy football will get their money's worth and more from a footballguys subscription, this post isn't a commercial for footballguys. And despite how it starts out, this isn't a post about fantasy football either. It's a post about correlation vs. causation, about process vs. results.
Every year, footballguys has a contest (with $35,000 cash awarded, I might add) for all subscribers. It works like this:
1. In early August, prices are set for about 250 NFL players. The prices don't change, nor are players added or deleted from the pool.
2. Each contestant has $250 to spend on a roster of 22 players. That's your team for the whole year. Set it and forget it. Except you can't forget it because it's so darn fun to follow.
3. Every week, the scores of your top QB, top two RBs, top three WRs, top remaining RB/WR/TE, top kicker, and top defense are added up, and that's your score for the week.
4. Every week, the bottom thousand or so teams are eliminated from contention, never to return. By week 13, there are 250 contestants still alive, and the team that scores the most combined points in weeks 14, 15, and 16 among those teams is the grand prize winner.
There's a little more to it than that, but that's the main idea. Here are the full rules if you're interested.
At first glance, there doesn't appear to be a lot of strategy involved, but there is much more than meets the eye. I can't even begin to describe how interesting the full contest database is as a playground for probability concepts.
This post investigates whether or not Seattle wide receiver Courtney Taylor was a good pick in this contest. Taylor had 5 catches for 50 yards in the first three weeks and then was released. He cost only $3, but that's a similar price to several alternatives who have put up much better numbers: Antonio Bryant, DeSean Jackson, Steve Breaston, Muhsin Muhammad, Brandon Lloyd, Devery Henderson, etc. Courtney Taylor was a bad choice.
One way to determine who the most valuable NFL performers have been in this contest is to see what percentage of that player's original owners are still alive in the contest. Right now, after five weeks, 48.4% of all the contest teams are still alive. So it's fair to say that a player has been valuable if more than 48.4% of his owners are still alive.
24% of Tom Brady's owners are still alive.
59% of Jay Cutler's owners are still alive.
This is pretty obvious.
64% of Steve Slaton's owners are still alive.
36% of Adrian Peterson's owners are still alive.
Even though Slaton and Peterson have similar fantasy point totals, Slaton's owners are doing far better. Why? The main reason is opportunity cost. Slaton cost a buck and Peterson cost $53. The non-Slaton part of a given Slaton owner's team figures to be much better than the non-Peterson part of a given Peterson owner's team.
Given this criteria, the most valuable wide receivers in this contest have been:
1. Larry Fitzgerald
2. Brandon Marshall
3. DeSean Jackson
4. Greg Jennings
These all make sense. DeSean Jackson doesn't have the numbers of the other three, but he only cost $4. And then we have:
5. Courtney Taylor.
At the beginning of the contest, 555 contestants selected Taylor. 353 are still alive, which is 63.6%. Given the sample size, there is less than a one-in-a-million shot of this kind of split being caused by random chance.
Yes, Taylor was cheap ($3). But there are plenty of similarly-priced WRs who have performed better. And there are other examples of this. Derek Hagan's owners are doing better than the contest average. So are Jason Hill's, despite the fact that Hill hasn't caught a pass all year! Even contestants who selected Hagan and Courtney Taylor are surviving at a much higher rate than the overall population despite wasting two roster spots.
In keeping with the theme of this post from last Thursday, this is another fine example of the distinction between correlation and causation. Taylor certainly hasn't been causing his owners to stay alive. But Taylor ownership is unquestionably linked with contest success. So there must be some other variable lurking in the background that is causing both Taylor ownership and success.
That variable is probably something like "paying attention." On August 1, when the prices were set, Taylor looked like Seattle's 4th receiver at best. Between then and the deadline for submitting entries, Bobby Engram got hurt, Deion Branch lost any hope he once had of being ready for week one. So Taylor was a starting wide receiver (at least for the first few weeks) who could be had at a low price. That's a smart guy to pick up. It didn't work out, but it signals a generally smart contestant.
Here's something more specific: Courtney Taylor ownership is highly correlated with Brandon Marshall ownership. 46% of Taylor owners also own Brandon Marshall. Only 26% of non-Taylor owners own Marshall. That makes perfect sense; grab Taylor to help get you through Marshall's suspension. Owners who are thinking along those lines are owners who are making generally smart moves with the rest of their roster. That's why they are succeeding.
I'm reminded of Moneyball, where Billy Beane talks about process versus results. Often, results are influenced by a variety of factors that are outside your control. If you judge your process solely by the results it generates, you're not doing a very good job of evaluating yourself in most cases. Given the unpredictability of player performances, it just doesn't make sense to say, "since Courtney Taylor hasn't produced good numbers, he was not a smart pick." Regardless of the results he produced, we have strong evidence that Courtney Taylor was a smart pick, that his selection is the result of a good process that didn't happen to work out in this particular case.
The process/results distinction often comes into focus when writers, commentators, and fans talk about fourth down plays and other strategical decisions. It worked, so it was the right choice. It didn't work, so it was a bad choice. The result was X, so therefore the process was X. We just don't get enough fourth-down plays in a season (or a decade of seasons) to be able to judge the fourth-down decision process based on fourth-down results. That's why we need analysis like Romer's. That's why, while it's not necessarily proof, it is acceptable to argue along the lines of, "because Coach X goes for it on fourth down more than most coaches, and Coach X also has more overall success than most coaches, going for it more often on fourth might be a good idea." Again, that probably can't be the entire argument, but it does count in my mind.