SITE NEWS: We are moving all of our site and company news into a single blog for Sports-Reference.com. We'll tag all PFR content, so you can quickly and easily find the content you want.

Also, our existing PFR blog rss feed will be redirected to the new site's feed.

Pro-Football-Reference.com » Sports Reference

For more from Chase and Jason, check out their work at Football Perspective and The Big Lead.

ShareThis

Pro Football Reference Blog

Winning vs. Stats: Matt Ryan

Posted by Chase Stuart on April 18, 2011
«
»

By the end of the 2010 regular season, Matt Ryan had become a media and fan favorite. A fellow blogger of mine at the Fifth Down argued for Ryan as MVP. When the Falcons beat the visiting Packers late in the season, Atlanta ran its home record to 19-1 with Ryan as the starter. Pro Football Focus named Ryan the most valuable player in football at the end of the season. Ryan's clutch play gave him more Win Probability Added than any other player in the league, according to Brian Burke. The Falcons ended the season with the best record in the NFC, and Ryan seemed on the verge of being anointed the game's next great quarterback.

But Ryan's Falcons were one-and-done in the playoffs, eliminated in no small part thanks to Ryan's ugly play in Atlanta's home loss to the Packers. Ryan's errant throw -- caught by Green Bay's Tramon Williams -- was one of the worst interceptions of the season. Was Ryan's season as magical as some would have you believe, or was his playoff performance more than just a bad game at the wrong time?


My favorite predictive stat for future success is net yards per attempt, i.e., yards per attempt with sack yards subtracted from the numerator and sacks added to the denominator. My favorite retrodictive or explanatory statistic is Adjusted Net yards per Attempt, which formed the foundation for how I graded quarterbacks in my Greatest QB of All-Time series. Adjusted Net Yards per Attempt starts with the basic NY/A formula, but adjusts the numerator with touchdown bonuses and interception penalties. But because those metrics are very random events -- important in determining winners, but not useful in determining future winners -- I prefer NY/A to ANY/A when judging a quarterback's true ability.

Why is that important? Because while Ryan performed well in ANY/A, that was largely because of his strong touchdown and interception rates. Coupled with his legitimately strong performances in clutch situations, it's easy to see why the Falcons won 13 games last year. But touchdowns, interceptions, and performance in high-leverage situations are factors that don't lend themselves well to repeatability. They tend to determine games, but they don't reflect skill or ability particularly well. And, in fact, while Ryan performed really well in certain areas, he was below the league average in net yards per attempt. Ryan ranked 25th out of 32 quarterbacks in NY/A in 2010, in a cluster behind Chad Henne, Matt Hasselbeck and ahead of Kerry Collins and Shaun Hill.

Not surprisingly, wins and net yards per attempt are strongly correlated. How rare is it for a quarterback to have the sort of success Ryan's team had with Ryan's individual statistics? I looked at all quarterback seasons from 1970 to 2009 which met the following three criteria:

  • The quarterback won at least 10 games as a starter
  • The quarterback won at least 70% of his games
  • The quarterback had at least 10 times as many passing yards as rushing yards

138 quarterbacks met those three criteria, or roughly four players per season. In 2009, there were five such players -- Philip Rivers, Drew Brees, Peyton Manning, Brett Favre and Donovan McNabb: all were well above league average in NY/A. Only three did so in 2010: Tom Brady, Matt Ryan and Joe Flacco. Brady ranked 3rd in the league in NY/A with a 7.2 average. While Flacco only averaged 0.3 more NY/A than Ryan, Flacco finished 13th overall while Ryan was at the bottom of a crowded pack at 25th.

Of the 138 quarterbacks from 1970 to 2009, only 11 ranked below league average in NY/A. The full list, below, sorted according to the ratio of their NY/A average to the league average NY/A rate:

QB year           age team           w win% ny/a lgavg ratio
Ken Stabler 1976 31 OAK 11 0.917 8.17 5.28 155%
Dan Marino 1984 23 MIA 14 0.875 8.60 5.90 146%
Bert Jones 1976 25 BAL 11 0.786 7.58 5.28 144%
Peyton Manning 2004 28 IND 12 0.750 8.74 6.14 142%
Mark Rypien 1991 29 WAS 14 0.875 8.19 5.97 137%
Randall Cunningham 1998 35 MIN 13 0.929 8.03 5.89 136%
John Brodie 1970 35 SFO 10 0.750 7.45 5.51 135%
Kurt Warner 1999 28 STL 13 0.813 7.86 5.85 135%
Boomer Esiason 1988 27 CIN 12 0.750 7.96 5.93 134%
Kurt Warner 2001 30 STL 14 0.875 7.87 5.87 134%
Ken Stabler 1974 29 OAK 11 0.846 7.10 5.35 133%
Peyton Manning 2005 29 IND 14 0.875 7.80 5.90 132%
Jim Hart 1976 32 STL 10 0.714 6.95 5.28 132%
Ken Anderson 1975 26 CIN 10 0.769 7.14 5.45 131%
Chris Chandler 1998 33 ATL 13 0.929 7.72 5.89 131%
Joe Montana 1984 28 SFO 14 0.933 7.69 5.90 130%
Philip Rivers 2009 28 SDG 13 0.813 8.00 6.17 130%
Joe Montana 1989 33 SFO 11 0.846 7.93 6.12 130%
John Hadl 1973 33 RAM 12 0.857 6.84 5.29 129%
Steve Young 1994 33 SFO 13 0.813 7.74 5.98 129%
Tom Brady 2007 30 NWE 16 1.000 7.81 6.05 129%
Drew Brees 2009 30 NOR 13 0.867 7.96 6.17 129%
Jim Hart 1975 31 STL 11 0.786 7.02 5.45 129%
Steve McNair 2003 30 TEN 10 0.714 7.42 5.84 127%
Ken Stabler 1977 32 OAK 10 0.769 6.56 5.18 127%
Peyton Manning 2006 30 IND 12 0.750 7.55 5.97 126%
Steve Young 1997 36 SFO 12 0.800 7.18 5.70 126%
Peyton Manning 1999 23 IND 13 0.813 7.35 5.85 126%
Bobby Hebert 1992 32 NOR 12 0.750 7.25 5.78 125%
Terry Bradshaw 1978 30 PIT 14 0.875 6.92 5.54 125%
Terry Bradshaw 1979 31 PIT 12 0.750 7.11 5.76 124%
Peyton Manning 2009 33 IND 14 0.875 7.62 6.17 124%
Trent Green 2003 33 KAN 13 0.813 7.20 5.84 123%
Ken Stabler 1975 30 OAK 10 0.769 6.71 5.45 123%
Tony Romo 2007 27 DAL 13 0.813 7.42 6.05 123%
Brett Favre 2007 38 GNB 13 0.813 7.39 6.05 122%
Jay Schroeder 1990 29 RAI 12 0.750 7.31 5.98 122%
Peyton Manning 2003 27 IND 12 0.750 7.12 5.84 122%
Jim Kelly 1990 30 BUF 12 0.857 7.30 5.98 122%
Ken Anderson 1973 24 CIN 10 0.714 6.42 5.29 121%
Troy Aikman 1993 27 DAL 11 0.786 7.05 5.81 121%
John Elway 1998 38 DEN 10 0.833 7.14 5.89 121%
Dan Fouts 1979 28 SDG 12 0.750 6.97 5.76 121%
Troy Aikman 1995 29 DAL 12 0.750 7.21 5.96 121%
Peyton Manning 2007 31 IND 13 0.813 7.31 6.05 121%
Brett Favre 2001 32 GNB 12 0.750 7.09 5.87 121%
Ben Roethlisberger 2004 22 PIT 13 1.000 7.41 6.14 121%
Roger Staubach 1978 36 DAL 11 0.733 6.68 5.54 121%
Brett Favre 1997 28 GNB 13 0.813 6.86 5.70 120%
Vinny Testaverde 1998 35 NYJ 12 0.923 7.08 5.89 120%
Jim Kelly 1991 31 BUF 13 0.867 7.16 5.97 120%
Donovan McNabb 2004 28 PHI 13 0.867 7.35 6.14 120%
Ken Anderson 1981 32 CIN 12 0.750 7.17 6.02 119%
Roger Staubach 1976 34 DAL 11 0.786 6.28 5.28 119%
Fran Tarkenton 1976 36 MIN 10 0.808 6.27 5.28 119%
Roger Staubach 1977 35 DAL 12 0.857 6.14 5.18 119%
Matt Hasselbeck 2005 30 SEA 13 0.813 6.99 5.90 118%
Vince Ferragamo 1980 26 RAM 11 0.733 7.04 5.96 118%
Jake Delhomme 2008 33 CAR 12 0.750 7.28 6.16 118%
James Harris 1975 28 RAM 11 0.846 6.43 5.45 118%
Drew Brees 2004 25 SDG 11 0.733 7.24 6.14 118%
Bob Griese 1974 29 MIA 10 0.769 6.31 5.35 118%
Joe Montana 1987 31 SFO 10 0.909 6.90 5.87 118%
Joe Theismann 1983 34 WAS 14 0.875 7.04 6.01 117%
Dan Marino 1985 24 MIA 12 0.750 6.80 5.83 117%
Troy Aikman 1994 28 DAL 10 0.714 6.98 5.98 117%
Ron Jaworski 1980 29 PHI 12 0.750 6.94 5.96 116%
Troy Aikman 1992 26 DAL 13 0.813 6.72 5.78 116%
Jay Schroeder 1986 25 WAS 12 0.750 6.80 5.88 116%
Joe Montana 1990 34 SFO 14 0.933 6.91 5.98 115%
Tom Brady 2004 27 NWE 14 0.875 7.06 6.14 115%
Jake Plummer 2005 31 DEN 13 0.813 6.76 5.90 115%
Fran Tarkenton 1973 33 MIN 12 0.857 6.04 5.29 114%
Bob Griese 1970 25 MIA 10 0.714 6.29 5.51 114%
Bert Jones 1977 26 BAL 10 0.714 5.88 5.18 114%
Brett Favre 2009 40 MIN 12 0.750 7.00 6.17 113%
Danny White 1981 29 DAL 11 0.733 6.80 6.02 113%
Stan Humphries 1992 27 SDG 11 0.733 6.51 5.78 113%
John Elway 1997 37 DEN 12 0.750 6.40 5.70 112%
Peyton Manning 2008 32 IND 12 0.750 6.88 6.16 112%
Fran Tarkenton 1975 35 MIN 12 0.857 6.08 5.45 112%
Jim Kelly 1993 33 BUF 12 0.750 6.49 5.81 112%
Donovan McNabb 2009 33 PHI 10 0.714 6.88 6.17 112%
Philip Rivers 2006 25 SDG 14 0.875 6.66 5.97 112%
Daryle Lamonica 1972 31 OAK 10 0.808 6.31 5.66 111%
Bob Griese 1973 28 MIA 12 0.923 5.88 5.29 111%
Jim Kelly 1988 28 BUF 12 0.750 6.54 5.93 110%
Phil Simms 1990 36 NYG 11 0.786 6.59 5.98 110%
Jeff Garcia 2001 31 SFO 12 0.750 6.46 5.87 110%
Johnny Unitas 1970 37 BAL 10 0.808 6.04 5.51 110%
Bob Griese 1977 32 MIA 10 0.714 5.68 5.18 110%
Dave Krieg 1986 28 SEA 10 0.714 6.44 5.88 110%
Dave Krieg 1984 26 SEA 12 0.750 6.46 5.90 109%
Danny White 1980 28 DAL 12 0.750 6.51 5.96 109%
John Elway 1996 36 DEN 13 0.867 6.37 5.83 109%
Joe Montana 1981 25 SFO 13 0.813 6.56 6.02 109%
Bobby Hebert 1987 27 NOR 10 0.833 6.37 5.87 109%
Steve Bartkowski 1980 28 ATL 12 0.750 6.47 5.96 109%
Ken Anderson 1976 27 CIN 10 0.714 5.73 5.28 109%
Craig Morton 1972 29 DAL 10 0.714 6.12 5.66 108%
Mark Brunell 1998 28 JAX 10 0.769 6.36 5.89 108%
Brett Favre 1996 27 GNB 13 0.813 6.27 5.83 108%
Marc Bulger 2003 26 STL 12 0.800 6.25 5.84 107%
Danny White 1983 31 DAL 12 0.750 6.43 6.01 107%
Bernie Kosar 1986 23 CLE 12 0.750 6.28 5.88 107%
Dan Marino 1990 29 MIA 12 0.750 6.36 5.98 106%
Pat Haden 1978 25 RAM 12 0.750 5.84 5.54 105%
Brad Johnson 2002 34 TAM 10 0.769 6.20 5.88 105%
Jim Hart 1974 30 STL 10 0.714 5.64 5.35 105%
Mark Brunell 1999 29 JAX 13 0.867 6.14 5.85 105%
Stan Humphries 1994 29 SDG 11 0.733 6.25 5.98 104%
Tom Brady 2003 26 NWE 14 0.875 6.08 5.84 104%
Phil Simms 1986 32 NYG 14 0.875 6.10 5.88 104%
Danny White 1985 33 DAL 10 0.714 6.04 5.83 104%
Tony Eason 1986 27 NWE 10 0.714 6.09 5.88 104%
Tom Brady 2006 29 NWE 12 0.750 6.19 5.97 104%
Phil Simms 1989 35 NYG 11 0.733 6.33 6.12 103%
Craig Morton 1977 34 DEN 12 0.857 5.36 5.18 103%
Kerry Collins 2000 28 NYG 12 0.750 6.04 5.85 103%
Steve McNair 2006 33 BAL 13 0.813 6.15 5.97 103%
John Elway 1984 24 DEN 12 0.857 6.04 5.90 102%
Brett Favre 2002 33 GNB 12 0.750 6.01 5.88 102%
Rex Grossman 2006 26 CHI 13 0.813 6.09 5.97 102%
Jim Plunkett 1983 36 RAI 10 0.769 6.11 6.01 102%
Warren Moon 1993 37 HOU 10 0.714 5.90 5.81 102%
Gary Cuozzo 1970 29 MIN 10 0.833 5.57 5.51 101%
Kerry Collins 2008 36 TEN 12 0.800 6.18 6.16 100%
John Elway 1991 31 DEN 12 0.750 5.94 5.97 100%
Marc Wilson 1985 28 RAI 11 0.846 5.80 5.83 100%
Tom Brady 2001 24 NWE 11 0.786 5.79 5.87 99%
Eli Manning 2008 27 NYG 12 0.750 6.06 6.16 98%
Scott Hunter 1972 25 GNB 10 0.714 5.50 5.66 97%
Dieter Brock 1985 34 RAM 11 0.733 5.55 5.83 95%
Ben Roethlisberger 2008 26 PIT 12 0.750 5.86 6.16 95%
Jim Miller 2001 30 CHI 11 0.846 5.49 5.87 93%
Dan Pastorini 1975 26 HOU 10 0.714 5.04 5.45 92%
Steve Bono 1995 33 KAN 13 0.813 5.48 5.96 92%
Neil O'Donnell 1994 28 PIT 10 0.714 5.41 5.98 90%

Now what the heck does all this mean? Wouldn't you rather have a quarterback that goes 12-4 but is below average in NY/A than a quarterback who is 11-5 but is above average in NY/A? Sure. But we only know that after the fact. Who would you rather have next year, the 12-4 "winner" or the 11-win QB who also put up good stats?

Of the 138 quarterbacks mentioned above, 9 of them were out of the league or played on different teams in the next season. Another 18 failed to start at least 8 games in the next season. That leaves 111 quarterbacks on whom we can perform some interesting analysis. I broke those quarterbacks down into five groups, based on where they fell along the scale of net yards per attempt relative to league average:

NY/A Rat #QBs Ratio Win% Next Yr
Over 130 16 136% 0.828 0.667
120-129 25 124% 0.794 0.678
110-119 32 115% 0.785 0.661
100-109 26 106% 0.787 0.628
Below 100 9 95% 0.774 0.552

For those fans of using correlation coefficients, the CC between winning percentage in year N and winning percentage in year N+1 was 0.09. The CC between NY/A ratio in Year N and winning percentage in year N+1 was 0.20. Consider what that means -- looking at NY/A doesn't tell you anything about the quality of the quarterback's defense or running game or coaching or special teams, something a team's record does take into account. But just using raw winning percentage from year to year was less helpful, in large part because of all of the random, fluky things that can happen in a football game that impacts the final result of the game. Net yards per attempt is much less skewed by outliers; because of that, it's a much better way to get a true understanding of a quarterback's ability than winning percentage. That's why it holds up better over the course of time.

What does this mean for Ryan? Remember, it's not like Ryan has been Trent Dilfer his whole career; Ryan ranked 3rd in the league in NY/A as a rookie in 2008. In '09, he regressed to a 6.0 average, a number he duplicated in 2010. Ryan has great pedigree and -- largely on the basis of his strong rookie season -- is still one of the most efficient passers in modern history through three seasons. I don't claim to know what changed for Ryan between 2008 and 2010, and I surely don't claim to know whether or not he can get back to that level of efficiency.

But I do think he's overrated. Quite heavily so, perhaps. I e-mailed PFR friend Brian Burke for his thoughts. Paraphrasing his response below, he also noted that Ryan was fortunate to come away with some wins this year, and might be overrated because:

1) Against the 49ers, Ryan threw a game-killing pick but Roddy White chased down the defender, forced a fumble and got the ball back. Ryan went on to get a win for that game. If White doesn't make that play, Ryan's Win Probability Added (WPA) takes a *huge* hit.

2) Roddy White in any other situation. White's WPA is 2.60, more than half of Ryan's WPA. When not throwing to White, Ryan isn't the same guy. He's good, but not anywhere close to what he is with White in the game.

3) Random variation. Ryan just happens to have his better moments in high leverage situations and his ugly moments when the game isn't on the line.

4) Factors beyond his control. Against the Ravens, Ryan threw incomplete on a 3rd down on the final drive of the game, forcing an improbable 4th and long. Inexplicably, Suggs was called for a facemask when the replay showed it was actually Suggs who was the victim of a facemask. At the very least, the call should have been for offsetting penalties. The refs' confusion gave the Falcons a critical first down late in the game, and three plays later, White catches huge touchdown. Another example: if NO doesn't miss that FG in overtime, there's no following drive for Ryan to mount, which was worth huge WPA. There are probably a few other examples like those. Ryan was charmed in 2010.

5) Fumbles. He rarely does it, and NY/A doesn't capture that.

6) He's clutch. No, I don't believe in clutch, or at least pro-level QBs are the kind of guys significantly affected by pressure, but I can't disprove it. Maybe he really is 'clutch.' We've got to at least admit the possibility. Perhaps he's just smart in terms of 'situational' football. He knows when to take risks and when to throw the ball away. Most likely, however, this is just the same as #3 above.

He's an above average quarterback but he was very lucky in 2010.

Ultimately, it's hard for me to put Ryan down anywhere near where his NY/A ratio says he should be. He's a young guy who should be penciled in as the starter for his team for a long time: only a handful of teams have one of those. Drew Brees averaged 6.5 NY/A and finished 10th; Ryan averaged 6.0 NY/A and ranked 25th. There isn't a significant difference between those numbers, but the ranking disparity looks enormous. I don't think Ryan's a top-10 quarterback, but I think he's probably in that next tier behind them.

But that's as far as I'll go. Keep in mind that his "mediocre" passing numbers also came while playing 10 of his 16 games in domes and going up against a weak schedule. If anything, with some weather and schedule adjustments, Ryan might come out looking quite a bit farther outside of the top ten. He was a winner in 2010; for him to be a winner in 2011, he's likely going to have to play a whole lot better.

Related posts:

    Checkdowns: Matt Ryan vs. Jake Long
    Matt Ryan, rookie extraordinaire
    Matt Ryan, Brian Brohm, and the right to choose
    How many Super Bowl winning quarterbacks are there?
    Quarterbacks: Career Playoff Drive Stats

This entry was posted on Monday, April 18th, 2011 at 9:00 am and is filed under Quarterbacks. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.

«
»