SITE NEWS: We are moving all of our site and company news into a single blog for Sports-Reference.com. We'll tag all PFR content, so you can quickly and easily find the content you want.
Also, our existing PFR blog rss feed will be redirected to the new site's feed.
Pro-Football-Reference.com ยป Sports Reference
For more from Chase and Jason, check out their work at Football Perspective and The Big Lead.
ShareThisSuper Bowl Winners, SRS champs, ANY/A and AYPC
I don't think this post will shed any light for our regular PFR readers, but I propose that from now on, we rank passing offenses not by passing yards but by either adjusted net yards per attempt or by Value - the number of adjusted yards gained over the league average.
Adjusted net yards per attempt, of course, is calculated by adding 20 yards (previously 10 yards) for each passing touchdown, subtracting 45 yards for each interception, subtracting sack yards from gross passing yards, and dividing this adjusted yardage by the combined number of sacks and attempts. That gives you the adjusted net yards per attempt for every player or team. To get their Value, you would subtract the league average from that number and multiply the difference by their number of combined sacks and pass attempts.
To me, this is a much better way to rank team passing. For example,
the 2008 Rams passed for 2,947 yards while the Titans passed for 2,819 yards. But along with those numbers, the Rams had 11 TD passes, 19 INTs, 520 attempts and were sacked 45 times. The Titans had 13 TD throws, 11 INTs, 453 attempts and were only sacked twelve times. The Rams averaged 4.09 ANY/A while the Titans averaged 5.75 adjusted net yards per attempt. With a league-wide average of 5.70 ANY/A, that puts the Rams at -911 yards above average (Value) and the Titans at 23 yards of Value. So despite St. Louis ranking ahead of Tennessee in passing yards, it seems obvious to me the the Titans were the far superior passing team.
While ranking passing teams by ANY/A or by Value makes good theoretical sense, there are many ways to test whether or not our theory holds water. While this isn't the most decisive way to do so, a timely and interesting way would be to measure how past Super Bowl winners rank in passing yards, adjusted net yards per attempt, and Value.
PYD ANY/A VALUE
gnb1966 12 1 1
gnb1967 16 12 12
nyj1968 3 4 2
kan1969 16 12 12
clt1970 3 7 5
dal1971 2 1 1
mia1972 16 1 1
mia1973 21 7 10
pit1974 21 17 17
pit1975 14 5 5
rai1976 3 2 2
dal1977 4 1 1
pit1978 12 2 3
pit1979 2 3 4
rai1980 19 18 17
sfo1981 7 5 4
was1982 12 5 5
rai1983 9 11 9
sfo1984 4 2 2
chi1985 20 6 6
nyg1986 17 12 12
was1987 4 3 3
sfo1988 10 6 6
sfo1989 2 1 1
nyg1990 22 5 6
was1991 5 1 1
dal1992 5 3 3
dal1993 7 2 2
sfo1994 4 1 1
dal1995 13 4 5
gnb1996 5 1 1
den1997 9 3 3
den1998 7 3 4
ram1999 1 1 1
rav2000 22 23 23
nwe2001 22 11 11
tam2002 15 15 15
nwe2003 9 10 9
nwe2004 11 7 7
pit2005 24 7 8
clt2006 2 1 1
nyg2007 21 22 23
average 10.8 6.3 6.3
This supports the argument that ANY/A is a better at predicting future success than passing yards. The average Super Bowl champion ranked 11th in the league in raw passing yards but sixth once you start making some adjustments. The 1990 Giants and '05 Steelers are good examples of this. Those teams didn't pass often but when they did, they passed efficiently. The '90s Cowboys were good passing teams that simply didn't throw that often; but we saw in the playoffs that they could throw when they needed to. Those were great passing teams by ANY/A standards, and considering their talent, ANY/A does a much better job than raw passing yards. So we can have some confidence that for predictive purposes, we'd care a lot more about a team's ANY/A or Value than their raw passing yards.
We can also measure whether this method is better at rating team accomplishments -- i.e., a retrodictive system. I looked at the best team in the NFL each year (as measured by SRS) and checked their rankings in the same three categories. Note: While PFR hasn't run the 2008 SRS numbers yet, I suspect the Titans will come in at #1, further supporting the use of ANY/A over raw passing yards.
PYD ANY/A VALUE
nwe2007 1 1 1
nwe2006 12 9 8
clt2005 3 1 1
nwe2004 11 7 7
kan2003 2 4 4
rai2002 1 2 1
ram2001 1 1 1
rai2000 15 6 7
ram1999 1 1 1
min1998 1 1 1
den1997 9 3 3
gnb1996 5 1 1
sfo1995 1 5 4
sfo1994 4 1 1
sfo1993 2 1 1
sfo1992 3 1 1
was1991 5 1 1
buf1990 10 2 3
sfo1989 2 1 1
min1988 4 4 5
sfo1987 2 1 1
nyg1986 17 12 12
chi1985 20 6 6
sfo1984 4 2 2
was1983 7 1 1
nyj1982 7 4 4
phi1981 20 14 13
phi1980 7 1 2
pit1979 2 3 4
dal1978 3 1 1
den1977 18 7 8
pit1976 22 13 13
pit1975 14 5 5
was1974 1 1 1
ram1973 13 1 2
mia1972 16 1 1
clt1971 21 19 20
min1970 14 10 11
average 7.9 4.1 4.2
Once again, the adjusted versions come out on top.
You can do the same exact analysis for pass defense. Here are the rankings of the SB champs:
PYD ANY/A VALUE
gnb1966 1 1 1
gnb1967 1 1 1
nyj1968 7 5 6
kan1969 5 2 2
clt1970 21 4 4
dal1971 19 6 5
mia1972 5 1 2
mia1973 1 2 2
pit1974 1 1 1
pit1975 4 3 3
rai1976 23 18 18
dal1977 2 2 2
pit1978 12 4 4
pit1979 10 4 3
rai1980 19 5 5
sfo1981 3 3 3
was1982 7 11 11
rai1983 13 9 9
sfo1984 17 9 9
chi1985 3 1 1
nyg1986 19 7 5
was1987 24 9 9
sfo1988 8 8 6
sfo1989 11 6 6
nyg1990 6 3 3
was1991 9 3 3
dal1992 5 10 9
dal1993 10 10 11
sfo1994 17 5 5
dal1995 8 9 9
gnb1996 1 1 1
den1997 5 4 5
den1998 26 14 14
ram1999 20 3 3
rav2000 8 4 5
nwe2001 24 13 13
tam2002 1 1 1
nwe2003 15 1 1
nwe2004 17 8 7
pit2005 16 5 4
clt2006 2 9 12
nyg2007 11 12 12
average 10.4 5.6 5.6
And the SRS champs:
PYD ANY/A VALUE
nwe2007 6 5 6
nwe2006 12 3 3
clt2005 15 10 9
nwe2004 17 8 7
kan2003 20 4 4
rai2002 23 10 9
ram2001 11 4 4
rai2000 25 15 14
ram1999 20 3 3
min1998 19 8 7
den1997 5 4 5
gnb1996 1 1 1
sfo1995 11 1 1
sfo1994 17 5 5
sfo1993 16 8 6
sfo1992 26 16 16
was1991 9 3 3
buf1990 7 8 9
sfo1989 11 6 6
min1988 2 1 1
sfo1987 1 1 1
nyg1986 19 7 5
chi1985 3 1 1
sfo1984 17 9 9
was1983 28 14 14
nyj1982 9 4 2
phi1981 1 1 1
phi1980 5 4 2
pit1979 10 4 3
dal1978 5 5 5
den1977 25 8 6
pit1976 6 6 4
pit1975 4 3 3
was1974 5 2 2
ram1973 7 5 7
mia1972 5 1 2
clt1971 2 1 2
min1970 1 1 1
average 11.2 5.3 5.0
There is little benefit in calling a team a good passing team based on their raw passing yards -- we really should just use their Value or their ANY/A numbers. Similar, a team's pass defense must be measured by ANY/A or Value, as raw passing yards allowed doesn't tell a great story.
All that said, what do the 2008 numbers look like?
Tm ANY/A Value San Diego Chargers 8.04 1174 New Orleans Saints 7.47 1145 Arizona Cardinals 7.02 866 Miami Dolphins 7.19 768 Indianapolis Colts 6.83 677 Atlanta Falcons 7.01 589 Denver Broncos 6.58 556 Green Bay Packers 6.59 508 Carolina Panthers 6.72 443 Houston Texans 6.45 439 New York Giants 6.14 227 New England Patriots 6.00 174 Dallas Cowboys 6.00 172 Philadelphia Eagles 5.80 63 Tennessee Titans 5.75 22 Tampa Bay Buccaneers 5.71 6 Baltimore Ravens 5.55 - 70 Washington Redskins 5.54 - 91 Pittsburgh Steelers 5.42 - 160 Minnesota Vikings 5.32 - 192 Jacksonville Jaguars 5.26 - 256 San Francisco 49ers 5.22 - 273 Buffalo Bills 5.12 - 304 Chicago Bears 5.08 - 346 Kansas City Chiefs 4.96 - 428 New York Jets 4.84 - 480 Oakland Raiders 4.64 - 490 Seattle Seahawks 4.51 - 607 Detroit Lions 4.39 - 735 St. Louis Rams 4.09 - 911 Cleveland Browns 3.32 -1220 Cincinnati Bengals 3.46 -1266
I think the 2009 Jets will manage to get along just fine without Brett Favre. And it's no surprise that seven of the bottom eight teams in passing Value will be making the first seven pick in this April's draft.
I did the same analysis for team rushing but got different results. I used rushing yards, adjusted yards per carry (yards per carry but with a 20-yard bonus for a rushing TD), adjusted yards per carry over 3.5 yards per carry and AYPC over 3.0 YPC. All those formulas produced similar results - the SB champion and the SRS champion averaged between an 8 and 10 ranking on all metrics. I suspect that a better analysis would be to just use the RBs for the team, because team rushing is much more likely to be influenced inappropriately than team passing -- QB kneeldowns, QB scrambles and reverses happen relatively frequently.
That said, there's a key distinction between rushing and passing. Allow me to make some gross generalizations. When you're a good passing team (measured by ANY/A), you probably won't need to pass that often. You'll score points and you'll be winning, so you'll stop throwing. If you're a bad passing team, you'll throw some INTs, have a bunch of three and outs, and are likely to have to pass a bunch in the fourth quarter. That's exactly why raw passing yards is a bad way to measure passing production -- bad passing teams are likely to have a bunch of passing yards late in the game and good passing teams aren't going to accumulate many passing yards at the ends of games. That means passing yards doesn't accurately measure passing ability, since it penalizes good passing teams and helps bad passing teams.
The opposite happens with rushing. If you're a great rushing team, you'll keep rushing. If you're a bad rushing team, you'll stop rushing. If you're effective running the ball through three quarters, you may not average a high YPC in the fourth quarter, but you'll keep running. If you've got 14 carries for 42 yards after three quarters and are down by 10 points, you're unlikely to have a 100 yard day. So in that sense, raw rushing yards tells a pretty good story. And that's why raw rushing yards and adjusted yards per carry over 3.0 YPC won't bring out too many differences. While I might have a slight preference for AYPC over 3.5 or 3.0 yards per carry, rushing yards hits the nail pretty well (especially if you just use rushing yards by running backs).
This entry was posted on Friday, January 16th, 2009 at 8:28 am and is filed under History, Statgeekery. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.
I'm dubious about the 20 yard bonus for rushing TDs.
On passing this makes sense because passing gets tougher, so the QB's passing numbers deteriorate, in the "red zone" due to the field getting shorter and the pass D having less area to cover (with the back of the end zone effectively acting as very efficient line of safeties across the field!)
The bonus for passes completed in that last 10 yards keeps QBs from being penalized for operating in that part of the field.
But I don't know that rushing becomes similarly more difficult near the end zone, such as to signficantly distort rushing performance stats. Yeah, runs from the 1 are truncated to 1 yard, so if a team's average rush is 4 yds a bonus of 3 might be justified to avoid distortion, but how significant is that for a team's total numbers?
Just one person's unsubstantiated opinion.
I think you're underpenalizing sacks. A significant part of the problem with giving up sacks is that they result in a high number of fumbles, and those fumbles are more easily recovered by the opponent than other types of fumbles. This certainly doesn't change the conclusions in this study, but I think the adjusted yards formula should probably incorporate this (or simply include the fumbles).
For a rough calculation, the fumbles/sack rate is somewhere around 1 in 8.5, and the loss rate is above 50%. If a lost fumble hurts the same as an INT (and it should probably hurt more), then there should be at least a 3-4 yard added penalty per sack. I guess that isn't much. Still...
This is interesting. But I am thinking that these ANY/A and Value are numbers that are more of a reflection of overall team strength than they are about actual passing ability.
For instance, Tennessee ranks slightly higher than Pittsburgh here. But if you were down by a couple points late in the game, would you rather have the Tennessee passing game or the Pittsburgh passing game? I think Pittsburgh is a better passing team than Tennessee. But this year I think Tennessee had a better running game and had more "easy" games, while Pittsburgh seemed to have a lot of games go down to the wire. So while Pittsburgh was needing to use their passing game and take more chances (leading to more sacks and picks), Tennessee was cruising, and was able to use their passing game as a supplement to their running game, and a supplement to their defense which put the team in good position to be in the lead.
But, there aren't a lot of examples like I present. Most of those ranking pass the sniff test. The one oddity in there is Miami. They had lots of close games. Pennington just had an amazing year.
Maybe part of it is just my prejudice after years and years of interpreting conventional statistics like raw passing yards.
Hey Jim,
The passing bonus does help off-set the problem of the short field. However, that's not where the 20 yard bonus comes from. It's not that it's difficult to score a touchdown, but the (weighted) average value of a touchdown is about 20 yards. The link in the post provides a more detailed explanation, as does this one.
Larry,
You're certainly correct. One of the updates I'm making to my QB analysis this year will be to incorporate fumbles. While not all QB fumbles come on designed passing plays, I'd imagine the majority do. Where did you pull those numbers you're citing?
I wouldn't say Tennessee had notably more "easy" games than Pittsburgh this year. Tennessee's differential was +141, Pittsburgh's +124. Pittsburgh won games 38-17, 38-10, 23-6, 27-10, 33-10, and 31-0. Watching most of the rest with a Steelers fan friend of mine, I can credibly say that the reason a lot of the other ones were close was because of the passing game's frequent poor play, especially in taking sacks.
The perception is that Pittsburgh has a better passing game is based on the team having flashier skill players, but the offensive line play between the two teams was monumentally different.
Rich, the thing to remember here is that this is a retrodictive analysis and not a predictive one. For me, I'd still take the Colts passing game over the Cardinals or Dolphins, but the numbers don't back that up. Why? Because Manning had a down year, AP voters aside. But going forward, I'd expect Manning to be like he is most years. I think the same analysis goes for the Titans and Steelers. The Tennessee passing game was better this year, but that's just because Roethlisberger and the PIT OL weren't up to par.
Also worth noting -- these numbers are *not* adjusted for strength of schedule. They should be and they will be, at some point in the off-season.
I'd be curious what the results would look like without TDs altogether. I know there is value at looking at the number of TDs a passing offense scores, but this does nothing for all the TD drives when the team goes the length of the field through the air and it's the RB that gets the 1-2 yard TD and credit.
How difficult would it be to run the cumulative APY/A for each team over the last 3 or 4 seasons?
The fumbles/sack number and recovery probabilities are my recollection from one of the Football Outsiders' books. I think it's from the first one they published. My recollection could be wrong, though. I didn't want to just suggest something without trying to estimate the magnitude of the effect, so I went ahead and tried the calculation; I think the answer is right within a factor of two in any case.
Chase,
On your QB pages, you have total fumbles and fumbles recovered, but not fumbles lost. Is it possible to get data on fumbles lost so we can find out how many turnovers each QB had?
Now you have a way of measuring the usefulness of a statistic... If that were fleshed out a bit, incorporating more data than just SB winners, you could then do things backwards and use it to determine the importance of things like sacks. Just run it with every penalty value per sack, like -10 to 10 yards (just to be complete), and it may be that at, say, 3.8 yards per sack penalty, the numbers align most closely with what they "should be" based on your metric.
Or you could have TD bonus, INT penalty, and sack penalty as variables and somewhere in 3d space, you'll have the highest correlation with some objective measurement of a team's strength (SRS, whatever).
Did that make any sense? It made perfect sense in my head... Anyway, if nothing else, it could be used as sort of a sanity-check on the numbers you're using like 20 yards and -45 yards.
I definitely think passing ranking needs to be based on a per-attempt or efficiency standard, not a total number. However, I don't like Adjusted Yards Per Attempt because, like Passer Rating, it is a somewhat contrived number. It involves placing a created metric (adding yards for TDs, subtracting yards for INTs) that, while not arbitrary, is still a human manipulation. Adjusted Yards Per Attempt is better than Passer Rating because it accounts for sacks, but it is similar in nature.
I prefer the Net Yards Per Attempt that you provide on this database. It is better than Yards Per Attempt because it accounts for sacks (which are critical), but it does not involve a theoretical manipulation of the numbers (since as you know from writing about it, people can and do debate what value should be manipulated for TDs and INTs). It is just a simple Per Attempt number with no internal debate regarding it (though there can be external debate about its usefulness).
That said, I recognize that ANY/A is more useful than NY/A because it accounts for TDs and INTs, and NY/A does not. And perhaps this is merely my own preference for looking at a few different numbers rather than trying to boil all numbers into one total metric. I'd rather look at a team's numbers and rankings in NY/A, TDs, and INTs separately, rather than look at a single metric which accounts for all three (and which could be argued). So I guess I'm not trying to poo poo ANY/A, just stating my own preference for looking at multiple numbers. And since this post begins by suggesting one single, totalizing number for ranking be replaced by another single, totalizing number for ranking, that's not entirely relevant, I suppose. If you're going to have a single number for ranking, ANY/A is obviously superior to Total Pass Yards, preferable to Passer Rating because it accounts for sacks, and preferable to NY/A because it accounts for TDs and INTs. I think somehow while writing this response I convinced myself you were right.
Can you use your metadata to analyze for fumbles per game? I did some minor calculations myself (Warner .82 fum/game, Favre - .57 fum/game even though he's second all-time for number of fumbles) I thought you would have better access to this data. I live in St. Louis and I'm trying to find some more reasons why the Rams didn't keep Warner and it would be good to put that into context.
Pacifist Viking---You say that you like certain stats more than others because they account for sacks, and you add they are critical. Then wouldn't a QB's rushing yards be just as critical, and shouldn't they be added in also? Most QB's gain rushing yards because they have just avoided a sack, and decide to take off and run with it. The only run that shouldn't be added in is a planned QB draw. What' fair is fair.
Denny, would that mean you would not include rushing yards from QB scrambles in the team's rushing yard totals?
I can see the logic to that--those rush yards do come on a team's called pass play. But I don't think those yards indicate the quality of the team's passing game. So if you're looking for a stat to measure strictly the quality of the passing game, those rush yards would, I think, confuse the issue (for example, Michael Vick gained a lot of rushing yards, but that doesn't necessarily indicate the quality of Atlanta's ability to pass the ball during Vick's tenure). Sacks, however, seem more clearly a negative effect of a passing play, and thus are of greater necessity in assessing the quality of a team's passing game.
I think it's silly that in college football, yards lost on sacks are subtracted from the player and team rushing total (as is my understanding from video games, anyway). Clearly those lost yards shouldn't be included in an evaluation of the player's or team's rushing ability. Perhaps the same argument should, indeed, go for QB scrambles.
I can see your argument, but I'd rather not have the scramble yards confusing the evaluation of a pass offense or defense.
I think this is particularly true if I'm trying to assess a team's pass defense. I'd want sacks accounted for, since rushing the passer is a significant part of pass defense. QBs gaining yards scrambling against a defense, however, don't seem a major factor in assessing strictly defense's ability to stop the pass--it's another issue.
Mattie, that's not a bad idea. I feel pretty good with the current values (counting a sack as a pass attempt, a sack yard lost equal to one less passing yards, and -45 and +20) but I'll put that on my to-do list. You'll start to run into some predictive/retrodictive issues, but that's not the end of the world.
Greetings, I will certainly be including fumble data soon. It's going to be in the next round of updates I make to my QB ranking system.
As for the Denny/PV debate, if it's possible, I agree with both of you. By only looking at sacks and sack yardage lost you penalize scramblers. And that's bad. That's why in my QB ranking system I give credit to scramblers. That said, a QB ranking system isn't the same as a passer rating system so I see PV's side of the debate as well.
Good points being made all around, and I guess in the end it really doesn't matter one way or the other. I still think that it's really ironic that the guy that holds the NFL Record for taking sacks (Elway) is also the same guy that led his Team to more Super Bowls (5) than any other QB in the S.B. era. Just out of curiosity--does anybody know who holds the Playoff Record for taking sacks? I wouldn't doubt that the answer could be Elway again.