## What quarterback rate stats stay most consistent when a team changes quarterbacks?

Posted by Jason Lisk on October 25, 2009

Three weeks ago, I asked the question "what quarterback rate stats stay most consistent when a quarterback changes teams?" Today I'm going to follow up with what happens when the opposite occurs, and a team changes quarterbacks (through ineffectiveness, injury, or because the team "needs a change" because the defense is giving up 30 points a game). I pulled all teams that had two quarterbacks each throw 200 or more passes, since the merger, and just like last time, used the advanced passing rating in each of five categories (completion percentage, yards per attempt, touchdown percentage, interception percentage, and sack percentage) to compare. There are 73 pairs of quarterbacks who qualify, ranging from notable names like DeBerg and Montana for the 1980 San Fransisco 49ers, to notorious names like the Craig Whelihan/Ryan Leaf pairing for the 1998 Chargers. One pair of quarterbacks, Jay Schroeder and Steve Beuerlein, appear on the list twice, for different franchises, six years apart.

Some teams' numbers stayed fairly consistent when they changed, like when the 1995 Saint Louis Rams used Rypien in place of Chris Miller, or when the 1998 Giants exchanged Kent Graham for Danny Kanell. Other switches resulted in wild swings in performance, like when a young Dave Krieg replaced Jim Zorn for the 1983 Seahawks, or an elderly Dave Krieg replaced a struggling Scott Mitchell for the 1994 Lions.

Running the correlation coefficients for this group was a little dicey. I'll go ahead and report the numbers, since I used it in the last post. But unlke before, where I'm comparing what the exact same quarterbacks did a year later, here, I had to decide which quarterbacks to include in group 1, and which in group 2. I settled on putting the first quarterback to play that season in the first slot. Here are the correlation coefficients between starter A (first QB to play) and starter B for this group of two quarterback teams.

Yards Per Attempt: +0.26 Completion Percentage: +0.25 Touchdown Percentage: +0.07 Sack Percentage: +0.00 Interception Percentage: -0.08

There were enough cases where the backup played significantly worse or better than the original starter that the correlations among the group are not high. YPA and completion percentage do remain most constant. Sack percentage and Interception percentage are the least consistent.

Next, the absolute value differences between the two quarterbacks' advanced passing score, sorted from smallest difference to largest.

Yards Per Attempt: 16.90 Completion Percentage: 17.00 Touchdown Percentage: 18.40 Interception Percentage: 19.90 Sack Percentage: 20.00

Same story here, yards per attempt and completion percentage have the smallest change; interceptions and sacks the largest. Next, just like last time, I computed the ordinal rankings for each team to see what category was the least and most consistent.

Completion Percentage: 2.89 Yards Per Attempt: 2.92 Touchdown Percentage: 2.94 Sack Percentage: 3.01 Interception Percentage: 3.25

Same story here. Finally, I looked at the total change in a quarterback's performance with a new team (as measured by the sum of the absolute value differences in each category), and divided the change in each category by the overall change to assign a percentage of change. This last summary lists the number of times that each category represented 20% or less of the total change in a team's performance after a quarterback change.

Yards Per Attempt: 48 of 73 Completion Percentage: 40 of 73 Touchdown Percentage: 40 of 73 Sack Percentage: 39 of 73 Interception Percentage: 37 of 73

Yards per attempt stands out here. Again, sack and interception percentage were the least consistent and accounted for the most change in performance.

Several commenters had excellent points in the previous post. So let me say that I'm not at this point measuring the exact contribution of the quarterback versus that of his offensive line or skill position players in terms of percentages. That's above and beyond the scope of what these numbers show. What they do show, though, is some general information about the relative effects of luck or other factors, such as game situation. Let's do some general summarizing of the information from this and the previous post.

Yards per attempt and Completion Percentage are both somewhat consistent when a team changes quarterbacks, and are also somewhat consistent when a quarterback changes teams. This tells me that with a large enough sample size of throws, luck plays a much smaller role in these two stats. The quarterback himself has something to do with both of these, and the teammates (and I'll lump things like offensive scheme and playcalling here as well) have something to do with these two. Is it 30/70? 40/60? I don't think we can answer that from just this information.

Touchdown percentage is somewhat inconsistent when a team changes quarterbacks, and also somewhat inconsistent when a quarterback changes teams. Interception percentage even moreso. It is the most inconsistent both when a team changes quarterbacks and a quarterback changes teams. This tells me that luck and game context play a large role in determining touchdown rates, and an even larger role in interception rates. What percentage of the remainder belongs to the quarterback versus his teammates? Again, that's beyond the scope of what I've looked at so far.

Which brings us to sack rate. It is one of the most consistent things when a quarterback changes teams. It is one of the least consistent things when a team changes quarterbacks. This tells me that the quarterback plays a larger role than people think in determining a team's sack rate. Again, I don't know what exact percentage is attributable to the quarterback versus the line, and I certainly don't think the line is irrelevant in determining a quarterback taking hits. I just think we measure line play by the wrong stat if we focus solely on sack rate. Sack rate does seem to have a lot to do with a quarterback's style, decision making, and willingness (or unwillingness) to gamble with a throw before ready. A quarterback with a tendency to take fewer sacks is going to get rid of the ball; it's his yards per attempt and completion percentage that are going to reflect whether the line did a good job. Was he throwing the ball when he wanted to, or before he wanted to?

To me, it makes sense that sack rate would most belong to the quarterback. It is the simplest statistic, and the one that the quarterback can exercise the most control over. It is simply "to release the ball, or not release the ball". What happens after the releasing of the ball brings in a lot of other factors--teammates, the opponent, luck.

Jason,

Like many people around here, I actually have been working on improving the QB rating formula. My current formula is currently running in the Fifth Down Blog every wednesday (https://fifthdown.blogs.nytimes.com/2009/10/21/building-a-better-passer-rating/).

The reason I bring this up is because (and I'm not the first one to think of this) the formula includes sacks as part of the Y/A calculation. And, this series of articles is what convinced me to include sacks against a QB. Very compelling work.

If you get a chance, swing by https://www.nflstatanalysis.net and look at some of the stuff I've done. One of the things I did was dig deeper into Aaron Rodgers' season. Why is his rating (in my calculation)low? How are sacks hurting him? What his he doing to be so effective on third down.....

Awsome post.

...awesome, that is.