By the end of the 2010 regular season, Matt Ryan had become a media and fan favorite. A fellow blogger of mine at the Fifth Down argued for Ryan as MVP. When the Falcons beat the visiting Packers late in the season, Atlanta ran its home record to 19-1 with Ryan as the starter. Pro Football Focus named Ryan the most valuable player in football at the end of the season. Ryan's clutch play gave him more Win Probability Added than any other player in the league, according to Brian Burke. The Falcons ended the season with the best record in the NFC, and Ryan seemed on the verge of being anointed the game's next great quarterback.
But Ryan's Falcons were one-and-done in the playoffs, eliminated in no small part thanks to Ryan's ugly play in Atlanta's home loss to the Packers. Ryan's errant throw -- caught by Green Bay's Tramon Williams -- was one of the worst interceptions of the season. Was Ryan's season as magical as some would have you believe, or was his playoff performance more than just a bad game at the wrong time?
My favorite predictive stat for future success is net yards per attempt, i.e., yards per attempt with sack yards subtracted from the numerator and sacks added to the denominator. My favorite retrodictive or explanatory statistic is Adjusted Net yards per Attempt, which formed the foundation for how I graded quarterbacks in my Greatest QB of All-Time series. Adjusted Net Yards per Attempt starts with the basic NY/A formula, but adjusts the numerator with touchdown bonuses and interception penalties. But because those metrics are very random events -- important in determining winners, but not useful in determining future winners -- I prefer NY/A to ANY/A when judging a quarterback's true ability.
Why is that important? Because while Ryan performed well in ANY/A, that was largely because of his strong touchdown and interception rates. Coupled with his legitimately strong performances in clutch situations, it's easy to see why the Falcons won 13 games last year. But touchdowns, interceptions, and performance in high-leverage situations are factors that don't lend themselves well to repeatability. They tend to determine games, but they don't reflect skill or ability particularly well. And, in fact, while Ryan performed really well in certain areas, he was below the league average in net yards per attempt. Ryan ranked 25th out of 32 quarterbacks in NY/A in 2010, in a cluster behind Chad Henne, Matt Hasselbeck and ahead of Kerry Collins and Shaun Hill.
Not surprisingly, wins and net yards per attempt are strongly correlated. How rare is it for a quarterback to have the sort of success Ryan's team had with Ryan's individual statistics? I looked at all quarterback seasons from 1970 to 2009 which met the following three criteria:
- The quarterback won at least 10 games as a starter
- The quarterback won at least 70% of his games
- The quarterback had at least 10 times as many passing yards as rushing yards
138 quarterbacks met those three criteria, or roughly four players per season. In 2009, there were five such players -- Philip Rivers, Drew Brees, Peyton Manning, Brett Favre and Donovan McNabb: all were well above league average in NY/A. Only three did so in 2010: Tom Brady, Matt Ryan and Joe Flacco. Brady ranked 3rd in the league in NY/A with a 7.2 average. While Flacco only averaged 0.3 more NY/A than Ryan, Flacco finished 13th overall while Ryan was at the bottom of a crowded pack at 25th.
Of the 138 quarterbacks from 1970 to 2009, only 11 ranked below league average in NY/A. The full list, below, sorted according to the ratio of their NY/A average to the league average NY/A rate:
Now what the heck does all this mean? Wouldn't you rather have a quarterback that goes 12-4 but is below average in NY/A than a quarterback who is 11-5 but is above average in NY/A? Sure. But we only know that after the fact. Who would you rather have next year, the 12-4 "winner" or the 11-win QB who also put up good stats?
Of the 138 quarterbacks mentioned above, 9 of them were out of the league or played on different teams in the next season. Another 18 failed to start at least 8 games in the next season. That leaves 111 quarterbacks on whom we can perform some interesting analysis. I broke those quarterbacks down into five groups, based on where they fell along the scale of net yards per attempt relative to league average:
|NY/A Rat||#QBs||Ratio||Win%||Next Yr|
For those fans of using correlation coefficients, the CC between winning percentage in year N and winning percentage in year N+1 was 0.09. The CC between NY/A ratio in Year N and winning percentage in year N+1 was 0.20. Consider what that means -- looking at NY/A doesn't tell you anything about the quality of the quarterback's defense or running game or coaching or special teams, something a team's record does take into account. But just using raw winning percentage from year to year was less helpful, in large part because of all of the random, fluky things that can happen in a football game that impacts the final result of the game. Net yards per attempt is much less skewed by outliers; because of that, it's a much better way to get a true understanding of a quarterback's ability than winning percentage. That's why it holds up better over the course of time.
What does this mean for Ryan? Remember, it's not like Ryan has been Trent Dilfer his whole career; Ryan ranked 3rd in the league in NY/A as a rookie in 2008. In '09, he regressed to a 6.0 average, a number he duplicated in 2010. Ryan has great pedigree and -- largely on the basis of his strong rookie season -- is still one of the most efficient passers in modern history through three seasons. I don't claim to know what changed for Ryan between 2008 and 2010, and I surely don't claim to know whether or not he can get back to that level of efficiency.
But I do think he's overrated. Quite heavily so, perhaps. I e-mailed PFR friend Brian Burke for his thoughts. Paraphrasing his response below, he also noted that Ryan was fortunate to come away with some wins this year, and might be overrated because:
1) Against the 49ers, Ryan threw a game-killing pick but Roddy White chased down the defender, forced a fumble and got the ball back. Ryan went on to get a win for that game. If White doesn't make that play, Ryan's Win Probability Added (WPA) takes a *huge* hit.
2) Roddy White in any other situation. White's WPA is 2.60, more than half of Ryan's WPA. When not throwing to White, Ryan isn't the same guy. He's good, but not anywhere close to what he is with White in the game.
3) Random variation. Ryan just happens to have his better moments in high leverage situations and his ugly moments when the game isn't on the line.
4) Factors beyond his control. Against the Ravens, Ryan threw incomplete on a 3rd down on the final drive of the game, forcing an improbable 4th and long. Inexplicably, Suggs was called for a facemask when the replay showed it was actually Suggs who was the victim of a facemask. At the very least, the call should have been for offsetting penalties. The refs' confusion gave the Falcons a critical first down late in the game, and three plays later, White catches huge touchdown. Another example: if NO doesn't miss that FG in overtime, there's no following drive for Ryan to mount, which was worth huge WPA. There are probably a few other examples like those. Ryan was charmed in 2010.
5) Fumbles. He rarely does it, and NY/A doesn't capture that.
6) He's clutch. No, I don't believe in clutch, or at least pro-level QBs are the kind of guys significantly affected by pressure, but I can't disprove it. Maybe he really is 'clutch.' We've got to at least admit the possibility. Perhaps he's just smart in terms of 'situational' football. He knows when to take risks and when to throw the ball away. Most likely, however, this is just the same as #3 above.
He's an above average quarterback but he was very lucky in 2010.
Ultimately, it's hard for me to put Ryan down anywhere near where his NY/A ratio says he should be. He's a young guy who should be penciled in as the starter for his team for a long time: only a handful of teams have one of those. Drew Brees averaged 6.5 NY/A and finished 10th; Ryan averaged 6.0 NY/A and ranked 25th. There isn't a significant difference between those numbers, but the ranking disparity looks enormous. I don't think Ryan's a top-10 quarterback, but I think he's probably in that next tier behind them.
But that's as far as I'll go. Keep in mind that his "mediocre" passing numbers also came while playing 10 of his 16 games in domes and going up against a weak schedule. If anything, with some weather and schedule adjustments, Ryan might come out looking quite a bit farther outside of the top ten. He was a winner in 2010; for him to be a winner in 2011, he's likely going to have to play a whole lot better.
This entry was posted on Monday, April 18th, 2011 at 9:00 am and is filed under Quarterbacks. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.