The Best Defense of All Time: Methodology Discussion
Posted by Chase Stuart on Wednesday, December 17, 2008
In case you haven't noticed, the 2008 Pittsburgh Steelers have one of the greatest defenses in regular season history. Pittsburgh ranks 1st in points allowed, yards allowed, yards per pass allowed, passing yards allowed, yards per rush allowed, and second in rushing yards allowed, sacks, rushing touchdowns allowed and third in passing touchdowns allowed. That's an incredibly balanced and terrific defense.
Believe it or not, the team that leads the league in points allowed usually isn't the leader in yards allowed; only four teams in the last 20 years have led the league in both categories -- the '06 Ravens, the '04 Steelers, the '02 Bucs and the '96 Packers. The '85 and '86 Bears, the '81 Eagles, the '79 Bucs, the '76 Steelers, the '72 Dolphins and the '70 Vikings are the only other post-merger teams to pull off this double double.
So what's the best way to rank the defenses? Let's run down the major statistics people use to rank the defenses.
Points Allowed: People who use this stat like to say things like, "at the end of the day, all I care about is how many points a defense allowed." While that's a good statement, points allowed also includes return touchdowns. That's a big problem -- if your QB throws a pick-six or your special teams allowed a score, that shouldn't hurt the defense. Further, this is heavily influenced by opponent's field position and opponent's time of possession, two factors that don't impact all defenses equally. But many view this as the mother of all defensive stats, so it's here to stay.
Yards Allowed: This is how the NFL officially ranks the defenses. Yards allowed avoids the problem of return touchdowns and opponent's field position, but it's a far from perfect statistic. Not all yards are equal, and teams that face a lot of pass attempts (which is not a sign of a bad defense) are prone to giving up lots of yards. Combining passing yards and rushing yards is Just Plain Wrong. But it's not as bad as...
Yards per Play Allowed: Every once in awhile, you'll hear analysts or read writers using a stat called yards per play (or for defenses, yards per play allowed). This is an awful stat that should be deleted from our lexicon; it's subject to Simpson's Paradox; for example, Clinton Portis is averaging more yards per rush and more yards per reception than Warrick Dunn, but fewer yards per play (or touch) than Dunn. Yards per play allowed is even worse than yards per play; let me explain why.
I've often wondered whether offense is simply the flip side of defense. I think a great offense will score about as many points against a great defense as an average offense will score against an average defense or a bad offense will score against a bad defense. Great offenses can lead teams to Super Bowls ('06 Colts, '99 Rams, '98 Broncos, any of Joe Montana's teams) just like great defenses can lead teams to Super Bowls ('02 Bucs, '00 Ravens, '85 Bears, any of the '70s Steelers teams). But there is at least one big difference between offense and defense. If you have a great passing offense and a bad rushing offense, you can be a great offense; if you have a bad passing offense and a great rushing offense, you can be a great offense. If you have a bad unit on defense, you can't be a great defense. Unlike offense, a defense is like a chain -- it is only as strong as its weakest link.
If a defense is dynamite against the run but terrible against the pass, teams will simply pass on them all day. On offense, you can easily hide a bad unit; on defense, there is nothing you can do. An offensive coordinator wants to be efficient; a defensive coordinator wants to be balanced.
In that way, if you measure yards per play correctly -- taking total yards, subtracting 50 points for fumbles lost, removing all sack yards, subtracting 45 yards for interceptions, and then dividing that number by pass attempts, rush attempts and sacks -- it's a decent measuring stick. But it still won't work for defense, where one number won't tell the story. A defense that's 10th in both passing and rushing defense (measured the correct way) should be better than one that is 1st in one type of defense and 19th in the other. Therefore, we need to break defenses down into their two components.
Adjusted net yards per pass allowed (ANYAA) : You knew this was coming. There is no better way to measure a pass defense than ANY/A allowed. The calculation explanation is at this link. People use passing yards allowed, or yards per pass allowed, or even just sacks to measure a pass defense, but this metric incorporates all of them.
Adjusted yards per carry allowed: This is simply rushing yards plus 20 yards for every rushing touchdown divided by the number of carries. With both ANY/AA and AYPCA, I am going to simply use the rate number. Usually when I'm looking at per attempt numbers, I'll compare that number to the league average and multiply the difference by the number of attempts; that's a great way to measure value. But sample size isn't a big issue here -- every team plays a full season -- and the "weakest link" argument comes in here, too. If a rush defense is great, it won't face that many attempts, and therefore would be undervalued. Further, if a defense is on the field all the time because the offense is bad, it's much more likely to be at the top or bottom of the rankings if you multiply the defense's rate numbers by the number of attempts. I'm comfortable sticking with averages here.
Offensive touchdowns allowed: This improves on points allowed in a few ways. One, teams aren't penalized or rewarded based on how field goal attempts go; a missed 30 yarder or a made 50 yarder are misleading indicators of a team's defensive ability. Further, it eliminates all return touchdowns. It reward teams that make goal line stands. Obviously it's not perfect -- field position still plays a big role here and allowing field goals isn't a great thing -- but I think it's a useful statistic to keep in mind.
Fumbles, Safeties and Touchdowns: A fumble is worth 50 yards. A defensive touchdown is worth 6.4 points -- after all, it gives the opponent the ball once again at about the 27 yard line. But what is a safety worth? The team that gets the safety usually gets the ball at around the 40 yard line. However, safeties usually come in situations when the team with the ball isn't likely to score next, anyway. Pretend that instead of a safety, the ball carrier is tackled at the one foot line. First and 10 from that yard marker is worth -1.6 points, and we know the team with the ball must either be facing second, third or fourth down and at least ten yards to go. I think it's safe to say that if you convert the average safety into the average "down at the one", the team with the ball is in a -2.0 situation. Thinking of it another way, the average field position of the defense following that situation is probably about the 50 yard line, which is a +2.0 situation for that team. This means the value of a safety is really just 1.6 points; the two points on the board plus the 0.4 point loss in field position following the safety (moving from the 50 to the 40). In a vacuum, a safety is really valuable because it's two points and the ball; but in a vacuum, tackling the ball carrier at his one is really valuable too, because it means you're probably going to get the ball really soon in really good shape. So net, a safety is worth only about one-fourth as much as a defensive touchdown. If we convert fumbles to yards -- and 12-13 yards is worth about one point -- a fumble is worth 4.0 points. That makes five defensive touchdowns equal to eight fumble recoveries equal to 20 safeties. Does that seem right to you? I'm going to hold off on this category for a little bit and see some of the responses here.
That leaves me with two scoring (Points Allowed and Offensive Touchdowns Allowed) and two yardage (ANYAA and AYPCA) categories to use when ranking the defenses. For example, four teams since 1960 have pulled off the rare 1-1-1-1. Here's a look at them.
2002 Bucs: They made Super Bowl champions out of Brad Johnson and John Gruden. This defense also scored five TDs in the regular season and four more in the playoffs. 196 points allowed, 1.1 OTA per game, 2.34 ANYAA and 4.18 AYPCA; NFL averages that season were 352 points allowed, 2.3 OTA, 5.47 ANYAA and 4.89 AYPCA.
1986 Bears: Not the Super Bowl version, mind you; if there's a blemish on that defense it was that they ranked 6th in yards per carry allowed and 5th in AYPCA. The '86 version allowed 187 points, 1.0 OTA/G, 2.63 ANYAA and 3.61 AYPCA. The league averages were 333, 2.2, 5.02 and 4.56.
1969 Vikings: Post-season failures by the team enabled this dynastic defense to fade into oblivion. The '69 Vikings went 1-1-1-1 in our four categories; the '70 version went 1-1-1-3 and the '71 team went 1-1-2-2. The Vikings had a three year stretch that was never matched. If you add their ranking in each season, you get a total of sixteen. The next best stretches are the '84-'86 Bears (24), the '86-'88 Bears (27), the '85-'87 Bears (27), '74-'76 Rams (30), '62-'64 Packers (33), '72-'74 Steelers (33). The closest modern teams are the '95-'97 49ers (38), the '99-'01 Ravens (43), '02-'04 Bucs (50) and '04-'06 Ravens (51).
The '69 Vikings allowed 133 points, 0.9 OTA/G, 0.96 ANYAA and 3.47 AYPCA. NFL averages were 301, 2.3, 4.70 and 4.62.
1963 Bears: George Halas' last championship squad. Signature game of the year? In Los Angeles, Mike Ditka caught four scores while the defense forced eight turnovers. 144 points allowed, 1.2 OTA/G, 0.81 ANYAA and 3.84 AYPCA. NFL (and AFL) averages were 322, 2.6, 4.93 and 4.77.
Three other teams have ranked last in each of the four categories. Here's how each of the teams graded along with the NFL averages on the right:
PA OTA/G ANYAA AYPCA PA OTA/G ANYAA AYPCA atl 1992 414 2.8 6.89 5.81 || 296 1.9 4.81 4.52 tam 1986 473 3.4 7.08 5.86 || 323 2.2 4.87 4.47 nwe 1972 446 3.6 7.52 5.94 || 277 2.0 4.15 4.68
A couple of things are noteworthy about the bottom two teams on that list. The Bucs defense was so bad that it caused Steve Young to go 2-12 as a starter. That '72 Patriots defense? Despite having a bad week just about every game they played, they allowed 501 yards in one game against the perfect Dolphins and lost 52-0 in their other game; that series was most definitely a sweep.
As for the Steelers? By the slimmest of margins, Pittsburgh is currently pulling off the 1-1-1-1, but the Ravens are nipping at their heels in the OTA/G category, the Titans are right there in points allowed and the Vikings are close in AYPCA. They're not the only ones making history, though: the Detroit Lions are 32-32-32-32 to date, and things at the back aren't even that close.
This entry was posted on Wednesday, December 17th, 2008 at 9:17 AM and filed under History, Statgeekery. Follow comments here with the RSS 2.0 feed. Skip to the end and leave a response. Trackbacks are closed.
I don't buy that 8 FR = 20 safeties or 1 FR = 2.5 safeties.
Maybe I missed why a fumble is worth 50 yards, is that because it's the expected average yard marker where a fumble is recovered? But isn't a safety the same as a fumble recovered at the 40 yard line (expected post free kick field postion) PLUS two points? Shouldn't a safety be worth MORE than a fumble - both give you the ball, both give you on the whole about the same expected field position, and a safety adds two points to your score.
Maybe a safety is properly "scored" because of the field position trade-off but a FR is over-valued? It doesn't seem like that roughly 2 FR = 1 TD to me but I haven't given it any detailed thought.
Also, the safety a given. Stopping the offense at the one yd line is an assumption - that's far from the only alternative outcome to tackling the ball-carrier in the end zone on the play in question, much less the subesquent plays (assuming the safety doesn't take place on 3rd or 4th down).
My guess is a safety = 2 pts plus the value of a FR minus 10 yards.
Posted on 17-Dec-08 at 9:56 am | Permalinkmrh,
A fumble is worth 50 yards for the same reason an interception is worth 45 yards -- it prevents a team from punting the ball and/or gaining more yards on offense. A fumble is worth slightly more because interceptions happen slightly farther downfield than fumbles.
I agree that it's a bit surprising to see a fumble being worth 2.5 safeties. That's one of the reasons I'm opening this up here.
(While stopping the offense at the one yard line is an assumption, I think it's one we need to make -- we don't have stats for "tackles at the one yard line" but we do for safeties. So to measure the value of a safety, I think we want to look at it in comparison to the value of the next best thing. Just like when we compared the value of a touchdown by seeing the value of having the ball at the one yard line.)
Posted on 17-Dec-08 at 10:21 am | PermalinkI like this discussion.
When I'm looking for a general quality of a defense (not analysis of "the best defense of all time"), I keep it simple. I like to look at the team's ranking in points allowed and the ranking in yards allowed. Obviously, as you note, each number has its problems. However, if I see a team is either top-10 or bottom-10 in BOTH categories, I feel I can make a reasonable assumption about the quality of the defense.
Posted on 17-Dec-08 at 11:21 am | PermalinkI'm not convinced (at least not by your argument) that balance is any more important for defense than it is for offense. Just like an offense can exploit a bad defensive unit, a defense can exploit a bad offensive unit. For example, it can put 8 in the box against an offense with a poor passing game, or it can put in extra defensive backs against an offense with a poor running game. I'm not sure balance means much on either side of the ball, or even offense/defense balance. It may be easier to improve your weakest link, but it's no more effective than strengthening you strongest link by the same amount.
I think the biggest reason to avoid per play metrics is that it ignores the down-and-distance situation and the ability to move the chains (or prevent them from moving). For example, a defense that consistently allows 4 yards on every play (without forcing a turnover) would be horrible, yet they would likely rank very highly in yards per play allowed.
Posted on 17-Dec-08 at 12:15 pm | PermalinkInterceptions do occur farther down the field than fumbles (lost), but their returns tend to be longer. Yardage-wise, they both get returned to around the original line of scrimmage, on average, but interceptions are more likely to be returned for touchdowns.
Posted on 17-Dec-08 at 12:23 pm | PermalinkJim A, you're right that my argument is light on proof -- that might make a good blog post of its own one day.
One example in NFL history borders on meaningless, but I am thinking of teams like the 2006 Vikings when I make that claim. Minny ranked 1st in AYPCA, and in fact, rank in the top five rush defenses since 1960. Their pass D ranked as ever so slightly above average in ANYAA, although you can imagine they ranked last in raw passing yardage because teams passed on them all the time (to avoid that tough run D). The conclusion is the Vikings ranked 14th in points allowed despite having a historically good run defense. It went to waste as teams ran 348 times against Minnesota but called 642 passes.
Posted on 17-Dec-08 at 12:30 pm | PermalinkJim A,
I got the numbers 45 and 50 from the Hidden Game of Football, as we often mention on this blog. I read through their research and find it pretty compelling, but it is also 20 years old. If you've got some new data on fumbles and interceptions, I'd be curious to see it.
Posted on 17-Dec-08 at 12:35 pm | PermalinkFollow-up to the discussion with Jim A on balance.
Eleven teams ranked inside the top 3 in ANYAA and outside the top 20 in AYPCA (most recently the '02 Packers); they averaged a ranking of 2.1 in ANYAA and 24.1 in AYPCA, along with an average of 8.5 in Points Allowed and 9.4 in OTA.
Seventeen teams ranked inside the top 3 in AYPCA and outside the top 20 in ANYAA, including the Ravens, Vikings and 49ers from last season. They averaged a ranking of 2.1 in AYPCA and 23.4 in ANYAA, along with an average of 16.2 in Points Allowed and 14.3 in OTA. As Jaws might say, points come out of the passing game. However, I'm not going to categorically say a great pass D and a terrible run D is better than a great run D and a terrible pass D, although one might interpret the data that way.
As far as balance goes, I also looked at teams that ranked 12-14 in AYPCA and 12-14 in ANYAA. On average, they ranked 13th in AYPCA, 13th in ANYAA, 13th in OTA and 14th in Points Allowed. So that doesn't support the argument of balance being incredibly important for a defense. Interestingly, last year's Super Bowl champions ranked 12th in AYPCA, 12th in ANYAA but 17th in PA and 19th in OTA.
Posted on 17-Dec-08 at 1:20 pm | PermalinkAnother example of defensive "balance." I believe last year's Saints only allowed 2 or 3 100 yd rushing backs. However, everyone knew that they could pass on Jason David and co. I don't think it was as severe as the 06 Vikings, but it does support the argument.
Another point of support--last week's Jets/Bills game. How many people have (probably correctly) assumed that if the Bills just RAN THE BALL!!, they win the game?
By the way, balance is important to the offense, also. However, would you know that last year's Patriots were the most effective team RUNNING THE BALL!, according to Football Outsiders DVOA metric? (I'm pretty sure they were #1 passing too.)
Posted on 17-Dec-08 at 1:44 pm | PermalinkI did some research on interception and fumble returns a few years ago and posted the results on the old PFRA board, as I recall. The main conclusions were as I posted above, though I'll try to hunt down the details.
I'd definitely like to see an update to the 45 and 50 yard penalties for interceptions and fumbles, especially in light of the recent 20-yard TD bonus finding.
Posted on 17-Dec-08 at 1:44 pm | PermalinkNice article. I find these year's Steelers, Ravens of 06 defenses as well as the Bucs 02 achievements extremely amazing since they are in the salary cap era and high penalties for no reason against defenses era. The mid 80s Bears might have won an extra title if they had a healthy McMahon. I can't believe I typed that.
@joseph - I'd be cautious throwing out the FO stats around as this year they have metrics showing the Iggles as the best team in the NFL. Yup, nothing screwy there.
Posted on 17-Dec-08 at 1:52 pm | PermalinkThis makes my head spin...but in a good way. I'm hoping you do something similar with offense...I'm more interested in that, especially considering what the Patriots did last year and how the 1999-2001 Rams measure up considering how turnover-prone they were. One thing: when explaining how yards per play should really be calculated, you mention "total yards" and "removing all sack yards". Does that mean you're taking sack yards out of the equation and treating total yards as different from total offense(total yards being pass + rush; total offense being pass + rush - sack)?
Posted on 17-Dec-08 at 2:09 pm | PermalinkDetermining the importance of run/pass efficiency "balance" is complicated by the theory that passing is more important to winning than running. This has been argued statistically for many years, going back to football stat pioneer Bud Goode and popularized most recently by FO.
But I'd love to see someone take a crack at measuring such balance with actual numbers.
Posted on 17-Dec-08 at 2:12 pm | PermalinkCorrect, Rock. I'm just trying to make clear that sack yards need to be removed from the offensive yardage total at some point in the process. If you take sack yards away from team passing yards (as they are), then team passing + team rushing does the trick. Or you could just add up all the individual passing yardage numbers to all the individual rushing numbers and then remove sack yards.
Posted on 17-Dec-08 at 2:13 pm | PermalinkJim A -- I think passing is slightly more important than running now. But it wasn't that in the '60s or '70s.
http://www.pro-football-reference.com/blog/?p=482
Posted on 17-Dec-08 at 2:16 pm | PermalinkRe #2-
I think the logic is skewed here. In making a turnover w/o points worth more than a turnover with points.
A safety is a turnover. That has some value. Because we know where the safety occurs and where the ball is expected to be after the turnover, we can establish the yardage value of having the ball but that's to some extent obscuring the value of the turnover itself. With a FR (or INT), we don't know where it will be. A FR at the opponent's 1 yd line is worth more than an FR at your 1 yd line - but the methodology just simplifies the calculation of all the values of all the possible recovery locations to say it's worth 45-50 yards or about the field position gained by preventing a punt. The 45-50 yards is the value of a turnover (the approximate sum of all the expected values of all the possible recovery/return locations except the end zone) without any intrinsic or scoreboard points, i.e. a non-TD FR = 0 scoreboard points. A safety is a turnover WITH intrinsic points so shouldn't it be worth more than a turnover without intrinsic points?
Think about it this way: Tm A causes a fumble and recovers it at their 40 yard line. You're saying that is worth 4 pts (because a FR = 50 yards even though a punt here won't gain 50 yards). Tm A tackles a player on Tm B for a safety and then gets the ball at the Tm A 40 yard line after the free kick and that's worth 1.6 points? Shouldn't it be worth approximately 4 pts for the trunover plus 1.6 points for the safety = 5.6 points? (Just like a TD is worth less than 7 points in this system, a safety should be worth less than 2 pts because of the field position loss - I accept that).
Posted on 17-Dec-08 at 3:40 pm | PermalinkVery interesting conversation. Just a few thoughts:
Posted on 17-Dec-08 at 6:41 pm | Permalink1. Safeties are weird. When I think about them, I imagine them as one of the most tragic acts a defense can perpetrate against an opposing offense, but in reality, they often don't break a game wide open, and often the gambit of possession changes winds up creating more favorable field position for the team the safety was against than would likely be the case had the safety not happened (ask Bill Belichick, vs the Broncos). Consequently, I don't have a huge problem with your valuation of the safety as you do, in light of the possible alternatives.
2. Balance on offense is nice. It is crucial on defense. On offense, you get to choose what play to run. On defense, you must live with what they choose to try to do to you. Sure you can play various schemes, but you have to deal with runs when they happpen and passes when they happen. On offense, there have been teams that succeeded without offensive balance throughout history.
3. Another possible concept to consider when evaluating defenses might be "short drives allowed". Three-and-outs are a big deal. Often after a turnover, the defense might give up 4 yards and a field goal. It's not their fault the team scored the FG. They did their job, shutting down the offense. The field position was a consequence of either turnovers or special teams returns. By my thinking, a 3-and-out FG drive is NO DIFFERENT than a 3-and-out punt drive, and the defense shouldn't be demerited in any way for the 3 points yielded.
I haven't looked into it, but I'm picturing 2 stat categories I just thought up: (1) "field goal attempts to touchdown drives" ratio and (2) "yards per field goal (attempt) drive". By my thinking, if a team is giving up 20 yards on drives that wind up in FG attempts, that's better than allowing teams to drive 50 yards before settling for a FG attempt.
One huge issue is strength of schedule. The results can be very different once you take into account the strength of the offenses faced. Teams like the Purple People eaters faced some pretty weak opposition from their division in the early-mid seventies.
Posted on 17-Dec-08 at 8:52 pm | PermalinkGood point, Dave. It's something to consider and something we can do a good job accounting for.
Posted on 17-Dec-08 at 8:55 pm | PermalinkOn the defensive td vs. fumble vs. safety, don't we have to consider that in order to score a defensive td, you first have to recover a fumble (or intercept the ball). Since prior to the fumble or int, the opposition likely was in a positive point expectancy situation (this of course assumes that the median opp field position at the time of defensive td turnover is past its own 27). Thus, the value of the defensive turnover would not only be the +6.4, but also the additional value robbed from the opponent at the time of the turnover. Does that make sense?
Posted on 17-Dec-08 at 9:48 pm | PermalinkSome quick numbers on fumble returns vs. interception returns. I compiled data on 2750 fumbles and 4170 interceptions from 2000-2007 regular season data. The average fumble (lost) was returned to 2 yards behind the original line of scrimmage (offense view) and the average interception was returned to the line of scrimmage. About 7% of fumbles and 10% of interceptions were returned for touchdowns, so that would give interceptions about another half yard penalty relative to fumble returns.
Anyway, the point is that in terms of yards and points, there shouldn't be much difference between the interception penalty and the lost fumble penalty.
Posted on 17-Dec-08 at 11:30 pm | PermalinkJKL,
I think we're okay because we are still counting interceptions and fumbles; INTs are included in ANYAA and fumbles are going to be included in this catch-all category, anyway.
I've got another thought, though. Perhaps fumble return yardage (INT return yardage is a story for another comment) should be included as well. A fumble return for 40 yards is worth more than a fumble return for 0 yards; and we data on fumble return yards. It also solves the problem of how to value a fumble return touchdown -- we know the incremental value of a fumble returned to the one yard line and one returned into the end zone: it's about ten yards. The reason is because it's a touchdown scored on a play where the next play is going to be first down anyway (http://www.pro-football-reference.com/blog/?p=603).
On the other hand, fumble return yards are somewhat like kick return yards -- they aren't as valuable as regular yards because they don't gain incremental first downs. They're not exactly like KR yards because there are no "gimme" yards, though. I think the best way to value all fumble returns would be something like: Fumble return yards x some number less than one plus FRTDs * 10.
INT return yards and INT TDs can be put in this formula as well. I'm still not exactly sure how to measure the value of the actual fumble recovery, but that may just be 40 or 45 yards per Jim A's answer.
Posted on 18-Dec-08 at 11:16 am | PermalinkChase: Regarding your post at 6, the 2006 Vikings had an amazing (that's one way to put it) stretch of four games where they gave up something like 2, 2, 1, and 2 defense/return TDs. I know it was seven in four games, and they might have had one or two more over the course of the season. That'll throw your points allowed seriously out of whack when compared to your yards allowed.
Posted on 18-Dec-08 at 5:03 pm | PermalinkHmm... The "per attempt" part bugs me. I can see it's necessary to account for the team's offense in the numbers but I don't know if per attempt really works. For average teams, I'm betting things average out nicely over the course of a season and per-attempt works just fine. But when we're focusing on best and worst, we're probably looking at a lot of the teams where per-attempt might not just work.
After game 5, their opponents scored:
Posted on 19-Dec-08 at 4:04 am | Permalink.
Teams that are spectacularly good or bad on offense remove the opposing offense's freedom to call what they want for a large portion of the game. When you're the 2007 Patriots and you're up by 3 TD's in the second half, you can sit back in a nickel defense or pass blitz all day. That's going to make their pass defense look spectacular and the rush defense look worse than it really is. Conversely, the 2007 49ers spent most of their games down big. Their opponents ran the ball to run the clock, and they could call rush defenses with relative impunity -- So they ended up 30th in rush attempts allowed and 6th in Y/C allowed.
.
And I know why they weren't mentioned (their first 5 games were not good) but the 1976 steel curtain deserves to be mentioned in any best defense discussion.
6, 0, 0, 0, 3, 16, 3, 0, 0
Nine straight wins, three shutouts in a row, five shutouts in nine games, eight of nine holding the opp to under seven points... Yikes. I've never seen anything close to that.
This approach suffers from two fundamental methodological errors.
First, if your goal is to measure the overall effectiveness of a defense, evaluating rushing and passing separately is Just Plain Wrong. A fundamental assumption you have to make in evaluating measurement of a game-theoretical problem like play selection is that the system is approximately in a Nash equilibrium (i.e., opposing defenses are playing rationally), even if you don't understand why. Separating out running and passing is just a hidden way to play "what-if" (as in what if the opposing coaches had a different payoff matrix). The job of a defense is to be effective against the opponents they have, not some hypothetical one.
Second, the biggest challenge in evaluating performance of a defense is taking into account down and distance (and, to a lesser extent, game situation); your methodology doesn't address this. The easiest way to do this over a sample as large as a season is to use drive statistics, rather than play statistics. (Perhaps you avoid these because because they don't go back as far.)
Posted on 19-Dec-08 at 5:02 am | PermalinkMattie,
I agree that there's no perfect way to look at raw stats, but I'm not sure what alternative is better than rate statistics. I think using raw statistics would be even worse. FWIW, the Pats ranked 5th in ANYAA and 22nd in AYPCA. The '07 49ers ranked 2nd in AYPCA, but I'm not so sure they weren't a very good run defense. I understand your point, but I don't know if there's a good fix for it.
Posted on 19-Dec-08 at 8:03 am | PermalinkMr.Mr: You mention the Mid 80's Bears and that they could have won another Super Bowl had McMahon been healthy.. I think part of the problem was people started to figure out the '46' defense (spread it out). McMahon's health was an issue, but Ditka also stated that losing Wilbur Marshall had alot to do with not winning another Super Bowl as well. Personally I think they didn't win another one because the competition was just too great. Not sure the Bears would have gone into the Meadowlands in '86 and beaten the Giants. '87 the Redskins beat them (just as they had done the year prior); and I think that's a byproduct of Gibbs' system being a bad matchup for the Bears with their 3 WR set and multiple formations. And then in '88 they had a terrible loss to the 49ers considering the game was at home in Chicago weather. Bottom line: The league was stronger when the Bears were great. These numbers that teams are putting up today don't impress me.. sounds like a blog topic to me..
Posted on 19-Dec-08 at 1:15 pm | PermalinkI don't think there's any sort of elegant fix at all -- It was just an observation. Everything is so intertwined that I don't even know what "best defense" really is.
Posted on 19-Dec-08 at 8:43 pm | Permalink.
Two thoughts:
One could do away with a straight ranking system and instead measure how many standard deviations above (or below) the norm that team's numbers were. A team could potentially be narrowly edged out on one category but be so devastating on the others that they might still be considered a "best defense" even without 1,1,1,1 rankings.
One could weight the pass and rush numbers based on the average pass/rush ratio for that era -- A 60's team's rush defense is probably far more important than a modern team's, etc.
I think you're devaluing the safety by saying it's really not better than a stuff inside the one yard line. That may be true, it may be basically a wash, but on the other hand a sack in the endzone is a heck of a lot better than a completed pass for a first down.
It doesn't seem consistent with your other measures to try and guess what might have happened if the safety didn't occur, esp. since you're not weighting things like field position resulting from a pick or fumble recovery, etc.
I like the idea of considering the safety as a turnover.
Posted on 20-Dec-08 at 10:48 am | PermalinkI'm thinking the value of a safety depends greatly on the down and distance when it occurs. A safety on 4th down (on a punt attempt) would give you 2 points + worse field position when compared to a successful punt, which is what you're assuming I think. In the end, that might be close to a wash.
Posted on 20-Dec-08 at 3:11 pm | Permalink.
But a safety on 1st down is 2 points + turnover - 60 yards + removal of all chances of an opposing drive of any length whatsoever. That seems far more valuable, yes?
.
So to calculate the average value of an average safety, it seems we'd have to calculate the value on 1st, 2nd, 3rd, and 4th down, then multiply that by the likelihood of a safety happening on 1st, 2nd, 3rd, and 4th down. I'm guessing it's going to be worth slightly more.
Steelers end the season 1-1-1-1. Lions end the season 32-32-32-32.
http://forums.footballguys.com/forum/index.php?showtopic=450718
Posted on 31-Dec-08 at 2:01 pm | PermalinkBest D of all time. The giants of last year stopped the #1 offence in the history of the leauge, Hows that for a measurement?
Posted on 07-Jan-09 at 12:05 pm | PermalinkJust taking a sec to say how much I love PFR and how often I dig DEEP into the stats on all my old faves and teams. I always have to get a good trivia question for my big brother for whenever I see him, and this is where I get them - but it's tough, because he remembers guys like Claxton Welch and Gloster Richardson.
Posted on 27-Jan-09 at 9:21 am | PermalinkTHANK YOU!
Baltimore Ravens Have had the best defense for the past decade. Led by Ray Lewis they are devastating to offenses. They don't get the credit they deserve. Partly because they haven't won a super bowl in 10 years. Every year they are top 5 in least amount of yards. They are one of the most feared defenses ever.
Posted on 13-Jun-09 at 10:06 am | Permalink