SITE NEWS: We are moving all of our site and company news into a single blog for Sports-Reference.com. We'll tag all PFR content, so you can quickly and easily find the content you want.
Also, our existing PFR blog rss feed will be redirected to the new site's feed.
Pro-Football-Reference.com » Sports Reference
For more from Chase and Jason, check out their work at Football Perspective and The Big Lead.
Insane idea: quit scouting
[First, a final reminder to enter the p-f-r draft contest. Entries must be time stamped before a pick is made.]
When I started this blog, I wanted it to be unique; I wanted it to be a place where you could read something different from what's being printed elsewhere. On the eve of the draft, I have opinions on Calvin Johnson and Adrian Peterson and Amobi Okoye, but it's not worth my time or yours for me to write them down. They simply don't add anything to the discussion. But the media frenzy surrounding the NFL draft and the absence of anything else football-related to talk about for the past two months leave me in a rough spot. If I want to say something that hasn't yet been said, it's going to have to be a little bit crazy. So I'm opening up a new category: Insane ideas. The ideas posted in this category are not meant to be taken too seriously --- I'm labeling them insane, after all. So I recognize that ultimately most of them will probably not be good ideas, but I am not just blatantly trolling either. I do believe there may be a kernel of a good idea in these posts.
So here goes with insane idea #1. Bear with me while I set it up...
Let's say you're in a football stadium with 80,000 of your closest friends watching a big game. For much of the time, you and your 80,000 friends are content to sit down, because it's more comfortable than standing up. But when something exciting starts to unfold, people start to stand up. Maybe for awhile, you can remain comfortably seated and still see the action. But as soon as some guy in front of you stands up, you have to stand up to have any chance of seeing over him. This, of course, forces the people behind you to stand up, which forces the people behind them to stand up, and so on. Pretty soon, everyone is standing.
But now everyone is worse off than they would have been if they all had simply remained seated. Any particular individual can gain a better view by standing up, which is (one reason) why people do it, but if everyone stands up then no one has a better view. The only difference is that everyone is now less comfortable.
This is not a novel observation. But let's take it a bit further. It's not quite true that everyone has the same view standing up as they do sitting down. The difference between tall people and short people gets magnified when people stand up. To put it another way, sitting is something of an equalizer. If the guy in front of me is taller than I am, then I might be able to see over him when we're sitting, but I can no longer see over him when we're standing. So if I'm short, a logical response might be for me to simply opt out of the standing up game completely. When everyone else stands up, I'll just stay seated. Sure, I'd like to watch the live action, but I'm simply too short to see over the guy in front of me. Standing up doesn't do me any good. Next game, I'll bring a stepstool to stand on. Or maybe I'll bring a little pocket TV. Or maybe I'll just stay home and watch the game on TV. That's not quite as enjoyable as live action, but I wasn't getting all the benefits of the live action anyway, and I was paying a lot of money for the privilege. If I don't buy season tickets, I could afford, for example, a much nicer car.
OK, at this point what we've established nothing more than that I am probably the most annoying and overly analytical football game companion imaginable and that you should not offer me a spare ticket if you happen to have one. But I'm setting up an analogy, and here it is:
Fans = NFL front offices
Standing up = expending money, time, and energy on scouting
If every team did its scouting using Street & Smith's college football preview magazine and watching a few college football games a week (as I understand was essentially the case when the draft was young), things would be easy for everyone and everyone would be basically in the same boat. But then Gil Brandt and his obnoxious Cowboy cronies started to stand up. They beat the bushes looking for small college players. They started conducting interviews and actually try to get to know the players. Now they can see the game pretty clearly, and that puts me and my Street & Smith's at a disadvantage. So I'd better stand up too. Pretty soon everyone is standing up and for the most part everyone is in the same boat again, except that we're all spending an obscene amount of time and money.
But we're not quite in the same boat. Some teams (if you're following along, these are the short people) are spending an obscene amount of money and still not seeing the game. What options do they have? Well, they can either buy a stepstool, or they can stay home and watch the game on TV. In this scenario, buying a stepstool corresponds to pouring a lot more resources into your scouting department: hiring more people and/or better people and giving them a bigger budget. But this kind of stepstool is expensive and it comes with no guarantees. It might collapse the first time you try to hoist your short self up onto it.
I'm more interested in the other option: watching the game on TV. I.e., settling for a slightly less appealing but much less expensive alternative, and then using the excess money to improve my life in some other way. What does this correspond to in the analogy? Is there a slightly less appealing but much less expensive alternative to spending resources scouting prospects?
I'd like to suggest that at this point the amount of freely-available information about the draft is so mountainous that, if you believe there is wisdom in crowds, it's not completely insane to believe that compiling all this public information in an intelligent way could constitute a reasonable substitute for a scouting department. In other words, the internet and a bunch of draft publications (maybe even Street & Smith's!) might be your TV.
I am not saying that teams who utilize this plan would typically have good drafts. But is it so hard to believe that you could, in the long run, have a drafting record that is only slightly below average with this plan? I am certainly not saying that your 19-year-old cousin who set up a draft website in his parents' basement is as good at evaluating talent as professional NFL scouts are. Rather, I am suggesting that, if he's a smart kid, he can learn a whole lot by piggybacking on the enormous body of work that is being done and reported on by professional scouts and others. There are a lot of sharp football minds, including some former professional scouts, who break down gobs of game film, who go to the college all star games and all the other events, who talk to the players, who talk to agents and NFL personnel men, and whose full time job is to provide information about draft-eligible college football players to anyone who wants to pay a small fee for it. There is also probably a lot of information to be inferred from reading factual accounts of which front offices were present at which pro days, which prospects were making visits to which teams, and so on.
If I paid Chase $50,000 per year to spend his every waking hour unearthing every possible nugget of information he could get his hands on, and finally synthesizing it into a draft board, could it pass for the draft board of a real NFL team? Having never seen one of the latter, I can't say. But seriously, how much worse could the result be than the drafts the Detroit Lions have had during the last decade? Or the Browns? In addition, I could employ a Massey-Thaler-like strategy of trading my high picks for multiple middle round picks, thereby minimizing my exposure to a single piece of bad information that Chase might have missed because he's not a real scout.
Now I know what you're thinking: concocting a plan to produce drafts better than the Lions of the late 90s and early 00s isn't exactly going to pave the road to the Super Bowl, which should be my ultimate goal. You're absolutely right. But remember the short guy who decided to watch the games on TV even though he'd have preferred to watch in person. Making that change wasn't necessarily a net loss of overall happiness for our short friend because, if you'll recall, he used his season ticket money to upgrade his next new car purchase from a Kia to a Honda.
And that's the crux of the matter. It's all fine and well to save millions (tens of millions?) on scouting every year, if I can find some other way to spend that money that will help my football team more than my cheap scouting budget will hurt it. With a salary cap in place, I can't spend the extra money on veteran players, but there may be other options. Maybe I could upgrade my next coaching hire from Wade K. Phillips to Bill H. Cowher. Maybe I could hire the absolute best offensive and defensive coordinators in the league and pay them enough so that they wouldn't be tempted to flee for a head coaching job. Maybe I could upgrade my facilities to make my team a more attractive destination for free agents. Maybe a bigger and/or better staff of team doctors and trainers (or groundskeepers) would help me keep my team healthier than average in the long run. We're talking about a lot of money saved here.
Like just about everything in life, winning football games is about the allocation of resources. Every team has a budget and must decide how to divvy it up among the various enterprises that comprise the final product. Most of the time, spending extra resources to improve one aspect of the team means that some other aspect will have to make due with less.
Now that doesn't mean that you can just spend your resources any old way and get the same result. For example, I would not as a general policy recommend saving 6 million dollars per year by always fielding a starting quarterback who is willing to play for the league minimum. Why not? Because the dropoff in production would probably be so steep that you'd be hard pressed to make it up by spending that money elsewhere. The difference here is that I'm (sort of) confident that a huge drop in expenditures in the scouting department might only lead to a small drop in production. If that's the case, then it's likely that the production drop can be more than recouped with a huge expenditure in some other area.
As promised, this idea is insane. But also as promised, I think there is a good idea at the core of it: skimp on scouting. Not because scouting isn't important; it's very important. But why foot the bill, when others (including your competitors!) are willing to do a lot of the work for you? If your scouting department has been doing a below average job, I'm about 95% sure that Chase could do almost as good a job for less than 1% of the salary. What I'm less sure of is whether or not I could find a way to use the savings productively.
This entry was posted on Friday, April 27th, 2007 at 3:59 am and is filed under Insane ideas, NFL Draft. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.

Interesting . . . I think you could make a solid argument that moving money from your scouting budget to your coaching budget is a smart move. Guys are getting drafted because of physical talent and potential, but I'd argue that in a lot of cases it's coaching that makes the biggest difference in an NFL player's ultimate career success.
I think there's a decent chance that a football version of Billy Beane or Paul DePodesta will emerge in some front office in the near future and change a way a lot of teams think about the draft and roster management in general. There have been too many busts (inefficiencies?) in the current system - it's begging for a "qaunt" overhaul.
Unlike the short person at the game, NFL teams CAN get taller!
What about the people that stay at home and watch on TV but stand up anyway?
I just recently heard a story about the Cleveland Browns in 1954 (I can't remember where I heard/read the story). With the first pick in the draft the Browns drafted Bobby Garrett a QB from Stanford. This was in the days before teams interviewed players or knew anything about them, except they may have seen them play just enough to know they were talented.
It turned out that Garrett had a stuttering problem and was unable to relay the plays to his team in the huddle, so didn't last very long in Paul Brown's complicated system and was cut from the team.
(The draft was 30+ rounds in those days!)
They can try, but they might not succeed. Further:
1. it will be expensive, probably costing the team resources that will have to be taken away from somewhere else.
2. the teams in front of them might also be getting taller at the same time.
1. it will be expensive, probably costing the team resources that will have to be taken away from somewhere else.
2. the teams in front of them might also be getting taller at the same time.
1. There is no cap on front office salaries, however, so spending more on scouting doesn't have to take away money somewhere else. Therefore, if a team wants to spend $10 million on an 80%-accurate fortune teller, it has no negative effect elsewhere.
2. True, but at least they have the opportunity to be the tallest. This argument is kind of like limiting practice time. If each team is only allowed 6 hours of practice per week, then you could say it's a level playing field. However, then teams are not maximizing their potential. Let's say Team A practices 8 hours a day, 5 days a week, and believes this is the amount of practice necessary for its players to maximize their performance on Sundays. Now, Team B feels it needs 12 hours a day of practice to maximize its performance. However, just because Team B practices for longer, it doesn't mean it has maximized its potential any more than Team A. In fact, Team A could be reaching 90% of its potential while Team B only reaches 85%.
This, in essence, is what scouting is, except instead of practice time, it uses money as a resource. Why prevent teams from improving in any way they deem necessary?
Shoot...post #6 is in reply to post #5. The first set of numbering is supposed to be quoted.
How do you get a quote to be blocked off?
I’d like to suggest that at this point the amount of freely-available information about the draft is so mountainous that, if you believe there is wisdom in crowds, it’s not completely insane to believe that compiling all this public information in an intelligent way could constitute a reasonable substitute for a scouting department.
I agree, with the caveat that I think the public information begins to suffer from "groupthink". JaMarcus Russell goes from a candidate for the first pick, to "The Raiders have no choice but to take Russell". Calvin Johnson goes from being on top of everyone's board, to being the best WR prospect of all time ("if we all agree that Calvin Johnson is really good, he must, in fact, be better than just really good"). If you could keep 10 "experts" from peaking at others projections, and just use their own opinions, then you would be better off. Then, I think a team could just examine those 10 independent opinions, put no money into scouting directly, and fare reasonably well.
Eddo, use the command:
blockquote and
/blockquote
to quote text, but be sure to put those command in these brackets:
The greater than/less than brackets that is.
Thanks, Richie, I had tried less than-quote-greater than. Left off the "block" part.
The fact that mock drafts are pretty close to the actual draft is pretty strong evidence that I could do a decent job drafting for Doug's team. Of course, this extends quite a bit beyond scouting. We can predict which free agent will sign where, and for how much. We know (lots of the time) whether a GM will use the franchise tag. And how often have you called out a play before it happens, and then see it work to perfection? We could call be pretty decent play-callers, too.
The thing is, the marginal difference is pretty significant. Teams are also a bit more heterogeneous than you're giving them credit for; the best WR isn't always the best WR for you.
But I think overall, I pretty much agree with you. Except you're not paying me enough.
All that mock draft accuracy tells us that the results of the draft are predictable. Knowing who will be picked where is only part of the issue; the other, more important part, is knowing which potential draftees will play well for your team.
In some cases, this requires more than the information that is public. Game tapes and 40 times can only tell you so much. Sometimes you need to see a cornerback in person to watch him and him only as plays develop. Other times you need to interview a quarterback or a lineman to determine if they will fit in well with the scheme you plan to run. This is where scouts come in.
Here's a pretty basic analogy: going out to eat with friends. Knowing your friends, you can say Albert will order spaghetti, Bob will order steak, Charlie will order a Caesar salad, and Danny will order chicken. This is fairly predictable; after all, you know what your friends like. However, if you've never been to the restaurant before, you can't predict if the food they order will be satisfying. How could you predict this? By going to the restaurant ahead of time and sampling each dish.
This is how the current system works. You and your friends are NFL front offices. The foods you like are the needs of your teams. The items you order are your draft picks. Your prediction of everyone's order is a mock draft. Scouting would be sampling the restaurant ahead of time.
Wow, there are a lot of ridiculous metaphors flying around here.
I don't think there is a way to spend your money more effectively than on scouting. It's clear to me that good drafting does help your team - the teams that draft well do so consistently enough that I don't think it's all luck, and those teams are also consistently among the best teams in the league. So if good drafting is a big part of a successful team, the best way to maximize your returns is on a scouting department that does it's job well and gives you good drafts. There isn't some other way to use your resources more productively if you are interested in improving the quality of your team.
This post reminds me of the first section of the Massey-Thaler paper discussing psychology studies on non-regressive predictions, overconfidence (Oskamp's experiment), expectation inflation, and false consensus and how they might apply to the NFL draft. For example:
I like Doug's analogy. Not only is standing up incredibly time-consuming and expensive, teams probably don't realize all the additional biases it introduces that impair rational judgement. Not only do I believe the Wisdom of Crowds would allow an amateur draftnik to do a reasonable job drafting, I'd bet one could even write a halfway-decent computer program (much like those fantasy sports draft robots) to assemble the publicly-available data and make a team's selections each year.
I agree that good drafting helps and that there probably are some differences in drafting ability among GMs, I don't think I've seen anyone ever define "good" drafting or try to quantify drafting skill in such a way that shows which teams (or even individual personnel guys) consistently exhibit it.
It's hard enough trying to separate personnel/scouting from coaching/environment in player performance evaluation. With just about seven picks per team per year, it would seem very difficult to make conclusions about distinct levels of drafting ability with much statistical confidence.
Doug,
Well put. I actually agree with everything you said wholeheartedly. Really, what it boils down to, the draft is essentially a crap shoot. Being a Packers fan from Wisconsin, I've seen my share of busts come down the pike. These teams can analyze these players all they want, but injuries will always play the most major role. Injuries are what keep great college players from adapting to the NFL. Eddie Lee Ivery. Need I say more.
Doug,
Not sure if you are accepting other insane ideas, but here's one. Why doesn't the NFLPA push for a rooke salary cap like the NBA has? It seems weird to suggest that the players should try to limit salary, but the huge signing bonuses given to all the draftees means that veterans are going to make less. Yet, veterans are the only members of the NFLPA. Wouldn't it be in their best interest to lower the max contract for draft picks?
The draft process "creates" resources for teams to use on the drafting process. Consumers spend a siginificant amount of their budgeted time that they will spend on the league ( producer ). In a way, it strengthens the product bonding process with time/interest invested by the consumer.
As a counter-argument, most teams rely (not exclusively, of course) on one of two scouting services for information on college prospects. Among the teams that do not are Baltimore, Chicago, Indianapolis, and New England - four of the best teams, and best drafting teams, in recent years.
Of course in fairness I should mention the other two teams, which aren't so good, that don't rely on the services - Washington and Oakland. For Washington, I wonder how much college scouting they actually do, since they've had few draft picks each year.
Anyway, this suggests that independently evaluating college players can be an advantage, if it's done well.
See link in name for article on Ravens scouting.
Isn't what you're suggesting basically exactly what the Cardinals did under Denny Green (and which to my mind appears to have worked out pretty well for them, although Green didn't last long enough to see the benefits)?
Of course, the Cardinals didn't spend the money elsewhere. The Bidwells just pocketed it.
I think your argument is plausible but offer these 4 observations: 1) as a Knowledge@Wharton newsletter I got today reminded me, a number of successful football decision makers - Jimmy Johnson, the Eagles are named in there - have emphasized team chemistry as a very important ingredient to success; 2) certain critical information like drugs and other offfield activities but also peer evaluations are not necessarily public; 3) Eddo is right that certain players are better fits than others for a team; 4) successful coaches will always insist on personnel control and will also want to use their position to get jobs for their out of work friends, so the Bill Cowher example won't happen in the real world.
Isn't Matt Millen the ultimate counter argument to teams spend too much effort scouting?
#21
The Redskins are actually decent when they draft players, they just never have many picks.