Pro Football Reference Blog

What happens when a quarterback changes teams? Which performance stats remain most consistent, suggesting they are more the responsibility of the quarterback himself, and which are least consistent, suggesting that outside forces (such as teammates, game situation, and random luck) play a larger role?

To examine this, I took all quarterbacks since the merger, between ages 25 and 35, who threw 14 or more attempts per team game in consecutive seasons, but did so for a different team in year two. The result was 48 different quarterback seasons (a handful of players appear on the list more than once). I examined the five basic performance rate stats: yards per attempt, completion percentage, touchdown percentage, interception percentage, and sack percentage. Other stats, such as passer rating or net adjusted yards per attempt, are derivative stats that rely on some combination of these underlying performance measures. Oh, and I used the advanced passing ranking in those five categories, rather than the raw rate stats, to avoid any era bias affecting the results, and to be able to compare between statistics (so we can compare a change of 5% in completion percentage to a 2% drop in sack rate).

I've tried to look at the data from several different angles. First, I looked at the correlation coefficient for our group of 48 passers, for the year N advanced passing score compared to the year N+1 advanced passing score in each category. This should tell us whether the passers who were good in a performance area (or bad) tended to be the ones who remained good in that performance area the following season, even with the uncertainty of team changes (some positive, some negative for the quarterback).

Sack Percentage:  +0.31
Completion Percentage: +0.25
Yards Per Attempt:  +0.20
Touchdown Percentage: +0.12
Interception Percentage: +0.10

Sack percentage checks out on top, primarily because those quarterbacks who were good at avoiding sacks tended to remain good at avoiding sacks. Of the top 21 in sack rate in year N (the top 20 and ties), 17 of them were above average in sack rate the following year on a different team. This probably understates the difference, because consider that we have three relatively rare event stats (sacks, touchdowns, and interceptions) that each occur infrequently. Even the most careless type typically throws an interception on about 3% more of the passes than his counterparts (over a large season sample) and the most prolific touchdown passer is only a few percentage points above league average. On the other hand, completion percentage and yards per attempt are not defined by rare events--one dropped pass is not going to impact completion percentage or yards per attempt nearly as much as an unlucky tip will influence interception rate.

Next, I looked at the absolute value difference in the advanced passing score from year N to year N+1. This measure is not concerned with direction, but magnitude of change. Here are the results in ascending order (remember that here, the smaller the number, the less average change in that category) when looking at change in advanced passing score in each category.

Completion Percentage: 13.44
Sack Percentage:  13.63
Yards Per Attempt:  14.15
Touchdown Percentage: 14.35
Interception Percentage: 15.90

Here, we see roughly the same order, except completion percentage moves just ahead of sack rate at the top. Interception percentage is still dead last, and moves even further away. The difference between interception percentage and touchdown percentage is now larger than the difference between the other four.

Next, to evaluate where each quarterback changed the least or most, I assigned an ordinal ranking of 1 to 5 to each of the five categories, with "1" representing the category where the quarterback was most similar to the previous year, and "5" representing the category with the largest change. Individual results varied greatly. Here are the composite scores (lowest means most consistent).

Completion Percentage: 2.79
Sack Percentage:  2.92
Touchdown Percentage: 2.98
Yards Per Attempt:  3.08
Interception Percentage: 3.22

Finally, I looked at the total change in a quarterback's performance with a new team (as measured by the sum of the absolute value differences in each category), and divided each category to assign a percentage of change. This last summary lists the number of times that each category represented 20% or less of the total change in a quarterback's performance from the previous year, after a team change.

Sack Percentage:  31 of 48
Completion Percentage: 30 of 48
Touchdown Percentage: 29 of 48
Yards Per Attempt:  28 of 48
Interception Percentage: 22 of 48

It's pretty clear which performance category is least consistent from year to year and probably belongs to the individual quarterback the least. It's probably the one the general public uses to judge a quarterback the most--interceptions. You rarely hear after a game about how the receivers caused the interceptions, or the bad luck of it all, or the game context dictated the interceptions.

At the other end of the spectrum, two performance measures stand out at the top--completion percentage and sack rate. One of those, of course, is the one passer performance measure that is not currently included in the NFL's passer rating formula.

Back in the 1985 Bill James Baseball Abstract, James talked about how the old sources never included walk information for hitters, and that a walk was generally thought of as "something that the pitcher did; the batter was just the guy who was standing there when he did it." I think that walks in baseball are a nice parallel to sacks in football some twenty years later. The common wisdom of sacks is that they are the fault of the offensive line or the accomplishment of the defensive player. The quarterback is "just the guy who was standing there".

While sacks can be the fault of the offensive line or the accomplishment of the defensive player, the evidence is pretty clear that the quarterback is at least as responsible for his team's sack rate as other passing performance measures that we readily attribute primarily to the quarterback. It's time that the NFL passer rating reflect as much. I suppose the argument against change would be that the passer rating is only measuring quarterbacks as passers, and a sack isn't a pass. This, to me, makes about as much sense as having a hitter rating in baseball that ignores walks and called third strikes, because, well, the guy wasn't swinging at the ball.

I recognize that I'm not the first the make an observation that quarterbacks might be more responsible for sack rate than we believe. In an article written in 2003, Michael David Smith, formerly of Football Outsiders, observed that quarterbacks on the same team showed different sack rates, when looking at a three year period. Those numbers weren't put into context with other statistics (to compare with changes at, say, yards per attempt or interception rate when teams change quarterbacks), so I'm going to do a post in the near future to look at quarterbacks on the same team, in the same season, and look at the same five categories I looked at here.