**SITE NEWS:**
We are moving all of our site and company news into a single blog for Sports-Reference.com. We'll tag all PFR content, so you can quickly and easily find the content you want.

Also, our existing PFR blog rss feed will be redirected to the new site's feed.

Pro-Football-Reference.com ยป Sports Reference

For more from Chase and Jason, check out their work at Football Perspective and The Big Lead.

## Breaking down average passer rating performances

I took all quarterbacks who had a roughly average passer rating performance on at least 20 attempts (game rating of 75.0 to 85.0) in the last decade (2000-2009 seasons) and broke them down by the four categories used in the passer rating formula--completion percentage, yards per attempt, touchdown percentage and interception percentage. By focusing on roughly average performances overall as judged by the passer rating formula, we should be able to see how an extreme performance in one of the categories compares to another in points scored and team won-loss record. 645 quarterback games met the criteria and were included in the study.

Here are the correlations between the various categories and both total points scored and team win-loss record:

Completion Percentage and Points Scored: -0.08

Completion Percentage and Winning Percentage: -0.04

Yards Per Attempt and Points Scored: +0.21

Yards Per Attempt and Winning Percentage: +0.05

Touchdown Percentage and Points Scored: +0.32

Touchdown Percentage and Winning Percentage: +0.06

Interception Percentage and Points Scored: +0.27

Interception Percentage and Winning Percentage: +0.03

Total Number of Pass Attempts and Points Scored: -0.02

Total Number of Pass Attempts and Winning Percentage: -0.28

Let's put those into sentence form. Among quarterbacks judged to be roughly equal by passer rating . . .

Those with a better completion percentage score fewer points and win slightly less than those with a lower completion percentage.

Those with a better yards per attempt score more points and win slightly more than those with a lower yards per attempt.

Those with a better touchdown percentage score more points and win slightly more than those with a lower touchdown percentage.

Those with a better interception percentage score fewer points and win slightly more than those with a lower interception percentage.

Those with more total pass attempts score about the same number of points and win less than those with fewer pass attempts.

I added the total pass attempts thing to show one of the quirks of using the passer rating in individual game situations. I don't think we can fault a passer who has to throw alot because his team is trailing, and the chances of going through a game without an interception or completing a high percentage decrease as the sample size increases. I think we would all agree that throwing 50 passes without an interception is more difficult than throwing 20, yet both passers get the same sub-score for interception rate. The passers who avoided interceptions and got a perfect score by not throwing as much (25 passes or less) won 69% of the time in this group. In contrast, no passer among this group who threw 45 attempts or more without an interception, the more difficult feat, actually won the game (0-16-1).

A further breakdown of each category may be even more illuminating. Here are the points scored and winning percentage breakdowns by completion percentage:

comp% | no. | pts | win pct | ||
---|---|---|---|---|---|

under 50 | 48 | 22.0 | 0.563 | ||

50-54.9 | 118 | 20.9 | 0.462 | ||

55-59.9 | 166 | 19.8 | 0.464 | ||

60-64.9 | 190 | 19.1 | 0.447 | ||

65-69.9 | 83 | 20.3 | 0.506 | ||

70 or higher | 40 | 19.4 | 0.450 |

Here we see that passers who are heavily dinged for completing a low percentage of passes score more points and win a higher percentage of games than other passers judged similar by passer rating. A quarterback completed less than 40% of passes only three times in this group of 645, and that quarterback's team won all three and they all scored at least 20 points. Each featured a yards per completion over 17.

Here is a breakdown by interception rates for this group:

int% | no. | pts/g | win pct | |
---|---|---|---|---|

0 | 224 | 17.6 | 0.471 | |

0.1 to 2.5 | 58 | 19.8 | 0.293 | |

2.6 to 5.0 | 281 | 20.7 | 0.505 | |

5.1 to 7.5 | 64 | 24.2 | 0.469 | |

7.5 and up | 18 | 24.3 | 0.500 |

Here is the breakdown by Yards per Attempt:

YPA | no. | pts | win pct | |
---|---|---|---|---|

under 5 | 45 | 16.7 | 0.400 | |

5 to 5.9 | 131 | 18.1 | 0.466 | |

6 to 6.9 | 216 | 19.9 | 0.442 | |

7 to 7.9 | 177 | 21.5 | 0.520 | |

8 to 8.9 | 76 | 22.0 | 0.487 |

And lastly, here is the breakdown by touchdown percentage:

td% | no. | pts | win pct | |
---|---|---|---|---|

0 | 119 | 17.2 | 0.521 | |

0.1 to 2.4 | 66 | 16.8 | 0.265 | |

2.5 to 5.0 | 315 | 19.7 | 0.460 | |

5.1 to 7.5 | 108 | 24.3 | 0.528 | |

7.6 or up | 37 | 24.5 | 0.595 |

This entry was posted on Friday, February 26th, 2010 at 7:30 am and is filed under General, Statgeekery. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.

Completion percentage has been badly overvalued for a long time. People would rather see their guy go 20-20 for 200 yards, "moving the chains," than go 4-20 for 200 yards on a few long TD bombs. The object of the game isn't to get first downs, it's to score, and if you can do that with short passes or long passes, it should make no difference.

It's the same thing in baseball, that if a guy alternated months hitting .250 and .350, he'd be valued as worse than a guy who hit .300 every month. People only want players to be consistent when they're doing poorly -- nobody complains about the guy hitting .350 (or the QB completing 4 long bombs for 200 yards), it's when he has his "down" times that everyone wants him to be "consistent."

Jason, your overall premise is correct--comp.% is somewhat overvalued. As a Saints fan, however, I feel that I must point out that the guy who just set a record with 70.62% completions also won the Super Bowl MVP--partly for doing just that. It's also worth noting that he is NOT regarded as a dink and dunk guy--he takes numerous shots downfield also. The stats bear this out, as: Colston's ypc=15.3, Henderson=15.8, Meachem=16.0; amongst the TE's: Shockey=11.9, Thomas 10.2; RB's: P. Thomas=7.7, Bush=7.1.

BTW, Brees RAW YP attempt were 8.54. If Chase or somebody could do his ANYA and add it here, I would appreciate it. I don't have the sack data in front of me right now.

Sorry to get off topic a bit, but as a whole, the PASSER RATING is broken--completion percentage still tells us which QB gets the ball into the hands of his playmakers better.

Joseph,

Brees also was #1 this year in ANY/A.

Can you do this analysis for good QB performances (90-100)?

Joseph, I know it seems like I hate completion percentage. I don't. I want my quarterback to complete as many as possible. The league leaders will usually have good completion percentages, but will also be really good at yards per attempt as well. If you complete alot of passes but don't gain many yards, you will get a significantly better QB rating than if you complete less passes for the same yards. That doesn't appear to be justified if we are trying to tie performance to scoring points and winning.

I do think that completion percentage has *some* predictive quality to it, though there is also alot of system and playing style wrapped into it and it is not just a measure of accuracy.

Re passing percentage, in this study ISTM there are two considerations:

1) Passing percentage itself, and its importance to scoring and winning.

2) Passing percentage in the passer rating formula, which was used to select the QBs in the sample.

As to #1, I'm a fan of the old-time long-throwers who favors high AYA over high completion pct -- but even for me, completion pct being *negative* compared to scoring and winning looks pretty odd. I'd expect it to be less positive than AYA, but actually negative looks strange. One would think that other things equal, a high completion pct indicates ability to convert first downs and keep the ball moving, e.g. compared to someone with a low completion pct and same AYA -- and that ought to be worth *something* positive.

As to #2, passing percentage is

grosslyover-weighted in the NFL formula, since completing even passes that lose *any* number of yards gives the QB a rating of 79 (10 of 10 for minus 200 yards, rating 79).As I mentioned elsewhere, Pete Palmer long ago gave the NFL passer rating formula as reducing to a yards-per-attempt formula "with a bonus of 20 yards for each completion, an additional 80 yards for each touchdown, and a 100-yard penalty for each interception. So two completions for ten yards each are worth the same as one completion for forty yards and one incompletion". ISTM that a yards-per-attempt rating with a 20-yard bonus for each completion is pretty biased!

So, putting those thoughts together... perhaps (just speculating) using passer rating to pick the selection of "average" QBs actually got some poor ones in there, and excluded some better ones, biasing the sample? That is, maybe getting *poor* QBs with low AYAs in the sample, on the basis of their high completion pcts, would actually penalize high completion pcts in the results, because poor QBs score fewer points and lose more games?

(I hate to make carping sounds like this when other people are doing all the work, and it is so interesting, and I'm just speculating with no evidence of my own having done no work myself, so I apologize ... but what's an Internet for?)

One thing that would interest me would be to compare passer rating itself with the AYA measures to see what correlates best with scoring and winning.

The rest of the results make sense to me. By their weight...

1) Scoring TDs should correlate pretty well with scoring! (Although this may be better explanatory looking back than predictive looking forward.)

2) Interceptions are the next biggest plays (but rates again may be better at explaining results than predicting them).

3) AYA I'd expect to be more positive than 4) completion pct.

5) QBs who are chucking the ball a ton generally are playing behind.

Anyhow, thanks for all this interesting material.

Great post, Jason. I agree with your comment above that completion % is heavily influenced by play calling, but I think there's another factor at work here: the way the NFL tallies passing yards.

For some bizarre reason, we give the QB full credit for the yards gained by the receiver after the catch, even though the receiver is usually far more responsible for those yards than the QB. This causes some major distortions in both comp % and YPA, because we don't know how far passes travel in the air (the QB's job), and therefore how difficult said passes are to complete. On the stat sheet, a pass that travels 30 yards in the air with no YAC looks the same as a 1 yard dumpoff with 29 YAC. The first pass is obviously far tougher to complete, but we can't tell the difference with traditional box score stats.

A good example is to compare the 2009 seasons of Jason Campbell and Kurt Warner. Their completion percentages were very similar (65% and 66%), and their yards/completion were nearly identical (11.06 and 11.07). However, anyone watching the games would tell you that Warner's pass attempts were more difficult than Campbell's. Splits from espn.com back this up:

For passes thrown 11+ yards in the air, Jason Campbell was 38 of 93 for 932 yards. Kurt Warner was 76 of 131 for 1481 yards.

Campbell: 40.9 comp %, 10.02 YPA

Warner: 58.0 comp %, 11.31 YPA

It turns out that Campbell's completion % is heavily inflated by a bunch of easy short passes and screen plays where the receivers do all the work but Campbell gets the credit. Warner, on the other hand, "earned" his stats more by attempting harder passes thrown further downfield, and not by relying on his receivers to pick up all the yards for him (which is ironic because people often say Warner is carried by his great receivers). Sorry this post ran so long, but I think the influence of receiver YAC on QB stats is an issue that needs to be examined further.

It's the same thing in baseball, that if a guy alternated months hitting .250 and .350, he'd be valued as worse than a guy who hit .300 every month. People only want players to be consistent when they're doing poorly -- nobody complains about the guy hitting .350 (or the QB completing 4 long bombs for 200 yards), it's when he has his "down" times that everyone wants him to be "consistent."The fans are entirely correct to value the consistent player more highly -- when they do so!

Take an example from football. Team A gets 3.5 yards per play, each and every play. Team B averages 8 yards per play -- more than twice as much -- but randomly among plays that get -5, -1, 0, 2, 4, 12, 16, 20. The teams play each other 100 times. Who is going to win?

Team A is going to win each and every time, 100-0, because it is an unstoppable scoring machine -- every three plays another first down. Team B, even averaging so much more per play at 8 yds each, is going to have series where it goes -5, 0, -1, punt, etc. It is going to lose every single time (assuming overtime play).

The point about "inconsistent play" is that you get *streaks* that are wasted on the upside and can be killers on the downside. If a baseball player hits one HR per game in each of six games, most will probably contribute something constructive towards winning the game. If another player who hits the same total of HRs overall hits six in just one game he may feel great about setting a record, and the fans will cheer ... but most of those HRs were probably wasted running up the score, and in another five games he contributes nothing. The first player is more valuable in winning games.

This is right out of finance. Stocks return a lot more than bonds on average in the long run, and can make you rich. But with stocks you can also get six bad years in a row -- and you may not live long enough to recover from that, your long run may not be long enough.

What's the best mix for a given objective comes from "portfolio theory" ... which is being applied to NFL play calling right now as we speak by sports economists and leading-edge coaches (for stocks say "passing" and bonds say "rushing") ... though that's another subject.

But the point is, consistency has positive value.

Take an example from football. Team A gets 3.5 yards per play, each and every play. Team B averages 8 yards per play -- more than twice as much -- but randomly among plays that get -5, -1, 0, 2, 4, 12, 16, 20. The teams play each other 100 times. Who is going to win?I could also say that the team that gains 10 yards every play will also win all the time because it's just as realistic as what Team A does.

Football is, admittedly, a little odder than baseball in that you have a set number of downs to achieve a certain goal and that, yes, 3.5 yards per carry, every carry, would make you unstoppable. But, apart from being unrealistic, it's a short-term sample.

The point about "inconsistent play" is that you get *streaks* that are wasted on the upside and can be killers on the downside. If a baseball player hits one HR per game in each of six games, most will probably contribute something constructive towards winning the game. If another player who hits the same total of HRs overall hits six in just one game he may feel great about setting a record, and the fans will cheer ... but most of those HRs were probably wasted running up the score, and in another five games he contributes nothing. The first player is more valuable in winning games.How do you know that streaks are "wasted on the upside" and "killers on the downside" without examining each game? Maybe the score in the six games for Player B (we'll assume all wins for his team) were 10-4, 11-1, 6-2, 4-3, 5-1, and 12-6. In only the 4-3 game can we say, with certainty, that his one home run had an effect. Player A, the six-home-run guy (again, unrealistic, but let's go with it), might have helped his team win 16-15 in Coors Field, needing every one of those bombs, while his team got blown out in the other five games, rendering his "downside" virtually meaningless. We can come up with scenarios to account for pretty much any outcome.

This is right out of finance. Stocks return a lot more than bonds on average in the long run, and can make you rich. But with stocks you can also get six bad years in a row -- and you may not live long enough to recover from that, your long run may not be long enough.True, but it's again short-term thinking. If stocks return 20% every other year and 0% every other year (average 10%) and bonds return 4% every year, guaranteed, you might be better off with bonds after six years than you would with stocks. Me, I've got about 30 years to retirement, so my 401(k) is fully wrapped up in stocks. Over the course of a drive, or maybe even a game, consistency might be nice, but over a season, I'd rather have the guy with the best raw numbers.

Some interesting thoughts here. My first thought about this post in general is that I'm not sure if this data is really useful when determining how completion percentage impacts overall teams precisely because it looks only at "average" passers. Like Jim Glass, I don't want to sound overly critical when I haven't done the work to look into it. But, I guess I'm not sure if this data would translate to the upper third of QBs in passer rating. It would seem that those may be the ones you'd want to look at to see which statistics most impact scoring and winning for those QBs who more closely resemble what a team wants in a QB. I guess this is one of those things that needs more data to really understand the overall impact of different aspects of the passer rating. At the very least, it definitely shows that completion percentage is overrated in the passer rating formula. I would be interested to see how these numbers compared when including all QBs, those above this average range, and those below this average range.

The question of consistency vs. big plays is interesting as well. I think that a very consistent QB/team will tend to win many games over the long haul (see Bill Cowher/Marty Schottenheimer) while a team with big play ability can always break through and win a championship. Being a Steelers fan, the 2008 Steelers come to mind. They were wildly inconsistent on offense that year but had big play potential with several players. It paid off with a ring, but then their big plays weren't enough to overcome this season. Of course, horrible 4th quarter defense in several games and just as horrible (if not worse) special teams had a lot to do with that too.

Jason,

Your conclusions are severely overstated.

"Those with a better completion percentage score fewer points and win slightly less than those with a lower completion percentage."

Incorrect. The correlations are not significantly different from 0. Completion percentage does not meaningfully predict points scored or wins according to your results.

"Those with a better yards per attempt score more points and win slightly more than those with a lower yards per attempt."

More points yes, win more no, not meaningfully more.

"Those with a better touchdown percentage score more points and win slightly more than those with a lower touchdown percentage."

Again, more points yes, win more no.

"Those with a better interception percentage score fewer points and win slightly more than those with a lower interception percentage."

See the previous 2.

"Those with more total pass attempts score about the same number of points and win less than those with fewer pass attempts."

This is actually a valid conclusion.

@bowl game anomaly

"Incorrect. The correlations are not significantly different from 0. Completion percentage does not meaningfully predict points scored or wins according to your results."

As statistical significance is a subjective term, an unsupported blanket statement of "incorrect" is, well, incorrect in this application. In terms of the correlation numbers being small, I could break down the game into a set of aspects which together comprised all events in any given but none of which had a larger correlation with winning than, say, .01. In this case, the statement "small correlations aren't meaningful" would amount to "nothing any player does on the field during a game helps his team win". I'd certainly be curious about the confidence that these correlations are greater/less than 0, but a 650 point data set isn't negligible.

The statements were a bit sloppy; they could be made bulletproof just by going to past tense (in the tense used, to be proper you'd have to say something along the lines of "the numbers suggest that there is an x% chance that the correlation between A and B is between y and z").

The only truly "incorrect" statement he gives is "Those with a better interception percentage...win slightly more than those with a lower interception percentage.", while the numbers say that the pick tossers won more than their possession-maintaining colleagues (past tense ftw).

It's worth pointing out that none of these numbers really tell us anything about the absolute value of these subfactors, because by design the other three rating factors vary in the 'opposite' direction. All any of these numbers tell us is how good a job passer rating does of accounting for the value of these aspects. If passer rating did a good job, then all of these correlations would be very close to 0*, which is fairly unintuitive.

*First order approximation. Second order, int numbers should be off, and therefore the other three categories should be slightly off to balance.

I agree with Patrick F that my language was sloppy. I should have used past tense. Using correlation coefficient in regard to the win/loss was not going to show much because of its binary nature, even if there was a relationship. A win is a win, whereas there were a range of points scored. That's why I broke it down into grouped data. We can look at the YPA table and see that the winning percentages in the two highest groups were greater than the winning percentages in the lower YPA groups when we grouped the data. Still, about half of those qb's in the higher YPA groups lost, so the correlation coefficient was small when using W/L.

Also, you are correct on the INT thing.

"As statistical significance is a subjective term, an unsupported blanket statement of "incorrect" is, well, incorrect in this application."

If I remember anything from stats class, it's that something being statistically significant is not subjective, but wholly objective. If I'm wrong on this (which I'm more than willing to accept), I'd like to be shown how/why.

The whole thing about consistent vs. boom-and-bust passers is reminiscent of how adding a consistency metric to pyth% actually makes it a better predictor of W-L%:

http://www.rawbw.com/~deano/helpscrn/corrgauss.html

http://www.rawbw.com/~deano/articles/BellCurve.html

One implication of the Correlated Gaussian Method is that between two teams with equal point differentials, the more consistent one will win more often because it's not "bunching" that differential in alternating blowouts and close losses, but instead is spreading it out more across its games. I think that's the point Jim was making about consistency -- between two passers with equal YPA, the one with the lower comp% is "bunching" his yardage on a few big plays and also contributing to non-scoring drives, while the higher-comp% guy is having fewer big plays but almost certainly getting more 1st downs. Which passer wins more games, though? That's a matter for further research.

BSK,

Statistical significance determined by applying an objective formula to the data to see if it meets a subjectively determined standard. In some fields, the subjective standard is p=.05. In others it's p=.01, or even p=.001. In football it probably should be about p=.10 or p=.20, but I've never seen anyone discuss this issue, much less or make a convincing argument about it.

However, I somewhat carelessly conflated the concepts of "statistical significance" with "meaningful difference." Not only does a correlation need to be statistically significant to be worth talking about, it also has to be strong enough to matter. Not only are some of Jason's correlation coefficients so small that they are almost certainly not significant even at p=.10, they are so small that it wouldn't matter if they were significant, because they are so weak that they don't give us any useful information. In other words, even if they are significant, all that means is that they are probably different from zero due to real population characteristics rather than random sampling error, not that they are actually much different than zero. They are still pretty damn close to zero.

BSK is quite correct in his assessment of statistical signficance, though I think a better term than subjective is arbitrary. When determining the level of significance, whether its p values of .05 or .01 etc, its based on the amount of rigor or confidence one is trying to establish. Many social science journals are leaning toward the more stringent standard. With all that being said, the correlation values cited with the review of the data are small, and if they were significant at some acceptable p value, then they still not be meaningful (sometimes a large "n" can yield statistical significance). Another problem I saw was of the restricted range of the data which tends to lower correlations. So I ask, why only average passrer ratings being sampled and them compared to outcome variables? I want to add to that I think its great Jason went to the trouble to analyze these different passing stat factors toward outcomes.

I'm sorry to cause a fuss over statistical significance. I meant those as factual statements when I said slightly more or less. Maybe if I used the word "negligible" instead of "slightly"? Did the Steelers win slightly or negligibly more than the Dolphins this year, or was there no difference? No statisticly significant difference? Statistically insignificant does not always equal zero. I agree that, in terms of statistical significance, there is no statistically significant difference in win percentage in the four categories.

As to why just average performances, 1) this is a blog post, and I'm going to try to do more things where I get rid of the file drawer problem of incomplete or partially thought out projects, and try not to make every post a full length article, and 2) I suspect that things aren't linear with passer rating, so I wanted to start with the middle or average. For example, just taking a quick look at some other ranges. For QB's that had a game rating of 100 to 120 (what we might call a pretty good rating), those who threw 3 interceptions and still got that high of a rating won just as many games as those who threw 0 interceptions to get that rating. However, looking at those with a rating of 55 to 65 (a bad performance), those who threw 3+ ints won only 7 of 77 games (9%) while those who got that rating despite 0 interceptions (and thus likely had a lower ypa, comp or td) went 26-41 (39%). So, in that limited and incomplete look, throwing interceptions is more costly for the below average performers than the above average ones.

... between two teams with equal point differentials, the more consistent one will win more often because it's not "bunching" that differential in alternating blowouts and close losses, but instead is spreading it out more across its games.I think that's the point Jim was making about consistency -- between two passers with equal YPA, the one with the lower comp% is "bunching" his yardage on a few big plays and also contributing to non-scoring drives, while the higher-comp% guy is having fewer big plays but almost certainly getting more 1st downs.Which passer wins more games, though? That's a matter for further research.Right, that's the idea. The problem with higher variance around a given average return is that it can cause a lot of "loss events"

in a row, without time enough left to recover from them and meet your objective. And these will be offset on the other end at other times by a bunch "gains" strung together that run up beyond your objective, wasting them (such as on running up a score).With lower variance around that same average return you reduce the risk of both those extremes.

(All this assumes your "average return" is good enough to meeet your objective, of course. OTOH, if it isn't, and the only thing that matters is whether you meet your objective or not -- not by how much -- then you

dowant to go "high variance". That increases your chance of getting lucky and stringing together enough big gains to meet your objective in an "upset". Of course it also increases the risk of stringing together a lot more loss events too -- but if losing by 30 is no worse than losing by 10, who cares?)As a crude, extreme example for football, a QB who throws 3 TDs a game on average would be pretty good. A QB who throws exactly 3 TDs in every game would give his team a 21 point start in every game, pretty darn good! But another QB who alternates between throwing 0 TDs in half of his games and 6 each in the other half would leave his team to score on its own without him during half the season, while probably wasting a couple TDs per game getting the score up over 40 pts in eachgame of the other half. To win the most games over the season, you want the first guy.

So *other things equal* a high completion percentage should be modestly beneficial, as providing the most stable yards per play.

I'd be willing to place a very small wager that: while AYA is a much better predictor of passing success than completion pct, for any *given* AYA a higher completion pct would be found to be modestly correlated with more scoring and winning.

If so, the explanation for the negative value for completion pct in the results above would be: completion pct is *hugely* overvalued in the NFL passing rating formula, which resulted in some QBs with poor AYAs but good completion pcts being promoted to the studied group of "average" QBs selected via the rating formula -- which caused their high completion pcts to look like they had negative value, when actually it was their low AYAs that did.

So what kind of wager are we talking? I don't think it will prove to be true at any given AYA, though it will at some. Here's just a very quick and entirely incomplete glance. These are the 10 pairs of teams that had the same AYA in the last 5 years, but at least a 5% difference in completion percentage. The high comp team is listed first, followed by points scored and wins, then the identical AYA low comp team.

2009 7.9 NOR 510 13 DAL 361 11

2008 7.2 ARI 427 9 ATL 391 11

2008 5.8 BUF 336 7 OTI 375 13

2007 6 NOR 379 7 CLE 402 10

2007 5 RAV 275 5 NYG 373 10

2006 7.7 CLT 427 12 PHI 398 10

2006 5.4 HTX 267 6 CAR 270 8

2005 6.7 CIN 421 11 CAR 391 11

2005 5.8 RAM 363 6 NYG 422 11

2005 5.6 TAM 300 11 ATL 351 8

High comps averaged 370.5 pts and 8.7 wins. Low comps averaged 373.4 and 10.3 wins. The largest difference in comp% between these pairs was the 2005 NY Giants (53%) and the 2005 Rams (65%).

@Jason: "High comps averaged 370.5 pts and 8.7 wins. Low comps averaged 373.4 and 10.3 wins. The largest difference in comp% between these pairs was the 2005 NY Giants (53%) and the 2005 Rams (65%)."

Interesting stuff. As a group, it does look like there's not much of a difference with cmp% (pretending the wins difference falls short of significance), but I kind of see two different populations here that have very different cmp% vs. points relationships.

For the teams that averaged more than 6 YPA, the 'high completion %' teams averaged 446.25 points, vs. 385.25 for the low. On the other hand, for those teams with 6 YPA and under, the high completion % group averaged just 320 points vs. 365.5 for the low completion % group.

In words, this would suggest that 'good passing games' are much more effective when they are consistent, where 'bad passing games' are worse when they are consistent. This feels like an result that could have reasonably been predicted; I seem to remember some posts/articles making similar arguments about the variable value of consistency on other topics.

"I'd be willing to place a very small wager that: while AYA is a much better predictor of passing success than completion pct, for any *given* AYA a higher completion pct would be found to be modestly correlated with more scoring and winning."So what kind of wager are we talking?...A bigger one than before, as I just ran a multiple regression of points scored, AYA and completion pct for all teams in the 2009 seasons, and that's what it found.

Correlation with points scored was .83 for AYA and .752 for completion pct. That seemed reasonable, but doesn't separate the effects of AYA and C_pct (since a higher pct implies higher AYA and vice versa). Thus the multiple regression to do that job.

To put AYA and C_pct in a comparable metric I normed each team's numbers by the league average, so a team with an AYA 105% of league average had an AYAn of 105, and one with a completion pct 95% of league average had a C_pctn of 95.

The multiple regression solves to Points Scored = -24.15, + 2.56 x AYAn, + 1.29 x C_pctn.

For example the Giants had an AYA 112.5% of average and C_Pct 102.46% of average. Thus, (2.56 x 112.5) + (1.29 x 102.46) - 24.16 = predicted points scored of 396. They actually scored 402.

(For the record, the team that scored the most points more than predicted was the Jets, +79, having the league's best defense helps. The one that most fell most on the downside was the Redskins, -93.)

These numbers look quite reasonable to me, AYA is twice as important as completion pct, but higher completion pct is worth something too.

OTOH, a general finding that lower completion pct is systematically associated with more scoring and winning really requires some explanation. An incomplete pass is a bad thing. When is it not bad? How do bad things systematically help teams score and win more? When a run of numbers turns out something like that one must either come up with the counter-intuitive explanation that gives the "Ah, ha!" moment, or failing that, keep looking at the numbers to figure out why they are reporting that bad is good.

This is a great study, but none of these correlations is particularily strong in either direction. It would seem that no single component of passer rating gives any real indication of whether or not a QB will win. These numbers point out very little. Perhaps a combination of two or more components?

Similar to others above, I think the way that the data set is constructed makes it impossible to draw the conclusions that were drawn. By definition a quarterback with a low completion percentage must have been good on other measures for that game in order to get into the sample set. If the low completion percentage quarterback was at the average of the sample set on other measures, they wouldn't have gotten into the sample of average quarterbacks.

What the analysis demonstrates is how well the formula works rather than how important any factor is specifically in winning (due to the issue above.) If the formula worked well (if it was a good predictor of points scored or win percentage) then you would expect that analysis of the residuals would show low or no correlation with the underlying factors. Based upon a quick look, what I would take away from the above is that it does a decent job with completion percentage, but the other factors need more weight/larger coefficients.

Playoffs are a whole new season. Certain types of coverage and physical play are allowed more so percentages change. Teams with the big play are probably going to produce along the same trend line. Consistency teams will actually produce a bit less against evenly matched comp.

Chuck Knox teams come to mind.

To everyone,

A lot of these arguments and debates can easily be settled by looking up correlation charts which show how each stat relates to winning percentages. Brian Burke has an excellent one in his April 8, 2007, blog. I have one (I think I got it from Fein Sports) in my "favorites" menu which shows "differentials" for various stats within games and how those "differentials" relate to winning percentage. A "differential" simply means that if team A has a higher passer rating than team B in a game, the chances of team A winning that game is .81 (correlation). Other differential correlations to winning percentage are: Yards per Pass Attempt .77; Yards per Play .72; Passing TD % .71; Total Yards .70; Rushing TDs .68; First Downs .63; Yards per Completion .60; and, finally, completion Percentage .56.

The Indy Colts web site Stampede Blue has run a remarkable research series on differential correlations and how they relate to winning percentages. The results are astounding.

Best, Clark