I am now trying to reproduce PFR's AV methodology in order to take a view on its usefulness (first rule of scientific method is that a model must be such as to be reproduced by other observers). I've created a spreadsheet and, as a starter, am trying to reproduce 2009 and some "test" QB's.

I have successfully reproduced the 2009 AV for Brady (16.7 vs. PFR Actual of 17), P. Manning (16.9 vs. Actual of 17), Rivers (18.9 vs. 19) and Roethlisberger (13.6 vs. 14), but I can't get what seems to be my properly functioning model to produce the same results as PFR for Drew Brees.

Can someone tell me where I am going wrong or whether PFR is wrong? I have taken the following inputs from PFR and made the listed calculations:

League 2009

Rushing TD's 429

Passing TD's 710

Field Goals Made 756

Field Goal Attempts 930

Total Turnovers 872

Interceptions thrown 525

Fumbles Lost 347

Punts 2451

Rushing Yards 59739

Passing Yards 111851

Total Yards 171590

Saints 2009

Rushing TD's 21

Passing TD's 34

Field Goals Made 22

Field Goal Attempts 28

Total Turnovers 28

Interceptions thrown 12

Fumbles Lost 16

Punts 58

Rushing Yards 2106

Passing Yards 4355

Total Yards 6461

Brees

Passing Yards 4388

Passing TD's 34

Pass Attempts 514

Interceptions thrown 10

Sacks 20

Sack Yards 35

I am using the following factors as given in the "AV Methodology" section:

OLine: .455

Rusher: .220

Rush/Total Yds Avg since 1970: .370

Passer Factor: .260

TD Multiplier: 20

INT Multiplier: 45

I end up with the following, using the formulas provided:

League Average

Offensive Points per Drive 1.90

Offense Points 100

Team Points for o-line 45.5

Team Points for skill positions 55.5

Team Points for Rushers 11.3

Team Points for Passers 11.2

Saints

Offensive Points per Drive 2.67

Offense Points 140.5

Team Points for o-line 63.9

Team Points for skill positions 76.6

Team Points for Rushers 14.9

Team Points for Passers 16.1

Brees Base Score 16.2

Bonus 1.4

Total 17.6

Actual Brees for 2009 per PFR: "16"

League

Adjusted Net Yards Per Pass Attempt 6.04

Pass TD Multiplier 20

INT Multiplier 45

Brees

Adjusted Net Yards Per Pass Attempt 8.82

Pass TD Multiplier 20

INT Multiplier 45

I appreciate the help. This model is getting some attention and, before I evaluate it, I want to be sure (a) that I know how it works and (b) that it works consistently on the web page.

]]>1) The increase is based on the contextual difference between TD passes and yardage in non-TD passes. That there is such an increase is unsurprising, since goal-to-go can be a high leverage situation. However, a corresponding hit for *non-TD* passes in goal-to-go situations (whether incomplete or not) is not assessed against the QB. The result is that quarterbacks who pass more often in goal-to-go situations will systematically be overvalued relative to other quarterbacks, even if their success rate in those situations is exactly the same.

2) The same contextual adjustments are not applied to the other weights. +10 for TDs and -45 for interceptions are appropriate when compared to changes in first down position for an offense as a whole. But for the actual interception itself, I think you'll find that the value of that play works out to average something like -45 for first down interceptions, -40 for second down interceptions, and -35 for third down interceptions -- interceptions are less costly as a play when the likelihood of making the first down goes down. On 4th down, the cost of an interception is usually no worse than an incompletion, and sometimes better. If we're willing to give an extra bonus to the touchdowns for context, shouldn't the interception context reduce that penalty? And perhaps the sacks should be examined to see if negative sack yardage deserves a negative or positive adjustment, given context?

]]>To account for this difference, we must multiply the value of a passing touchdown (19.3) by 55% to find how much better scoring a touchdown was compared to . This equals 10.6 yards, which is the actual value of a touchdown.

]]>On the other hand, QBs with high TD totals often lack a good red zone/short yardage RB, thus giving them more opportunities to throw from point-blank range. One can still wonder how many more TD passes Troy Aikman would have had without Emmitt Smith running it in on the ground.

]]>But if you're calculating the value of a TD and the values of pass TDs on "and-goal" are all inflated, then the average will be inflated too, wouldn't it? Perhaps 19.3 yards/td is maybe a bit over-generous, which could give too much advantage to QBs who threw more TD's per attempt. The difference would probably be lost in the noise after all the other issues with quantifying QB performance are taken into account (the quality of the opponents, defense, WRs, running game, blah blah), and good QB's probably throw a lot more TD's/attempt anyway, but... 🙂

Hmm, so are you going to recalculate the average value of an INT now? 😉

Your other point is also a fair one. Yes, a 4th-and-G TD is going to be more worth more than a regular TD. I understand that my system says 19 yard completion, INC, INC, INC, TD is worth more than 20 yard completion for a TD. But it's actually not. Why?

Because I'm giving 19.3 for all TDs. So a 20 yard TD on 1 attempt is worth 39.3 yards/attempt. A 19 yard completion, INC, INC, INC, TD is worth 39.3 yards, but on five attempts. That's a significant difference.

]]>Those are two different points so let me address them separately.

Your first point is correct. It's something I've spent a little bit of time thinking about, and I'm not sure what to do. Your XP example is a good one -- the problem with my system is it doesn't address difficulty level. Value and difficulty level often coincide, but not always, and this is probably a good example of that.

On the other hand, I think it's not a serious problem as long as our comparisons are intraposition. If we're comparing kickers to each other, it's okay if they all get 1.00 points for XPs instead of .02 points, as long as we're looking at rate numbers and not totals.

For QBs, the value of a touchdown, on average, is worth about 19.3 yards. Sometimes those touchdowns will be easy, sometimes hard. Sometimes a 10 yard pass is easy, and sometimes it will be hard. I think the only way to do it given the limited data available on older players is to look just at value creaeted. Keep in mind that when we do the best QB ever rankings, we are comparing all QBs to all other QBs, and that handles a lot of these concerns. At least, I think.

]]>In reality, every kicker makes what, 98% of his extra points? More? So we're saying 98% of that value is already there in the "and goal" situations, and a made extra point is really worth the value 1.0 minus the average value of an extra point attempt (0.98, 0.99?) so it's worth 0.01 or 0.02 points and a missed XP would be worth -0.99 or -0.98. So why arent' we doing this for passing plays? To my mind, a TD on 4th and goal is worth 3.2 and an incompletion would be worth -1.65. The difference between those two results is still 4.85 but now it's not all weighted on the positive end.

.

Lets say 1st and 10 from the 20 is worth 3.0 points. (arbitrary)

If we throw a TD, we just incresed our value from 3.0 to 6.4, so the value of that play is 3.4. That's partly for the TD and partly for the 20 yards. Your analysis suggests 2.55 for the 19 yards, then 0.85 for the TD+1 yard.

If we throw a 19 yard pass, then throw a TD on 1st and 1, we've "scored" 3.4 across two plays instead of 1, so they still need to add up to 3.4, right?

2.55 + 0.85 (if it was 1.07, a 19 yard and 1 yard pass would be worth more than a 20 yard pass)

If we throw a 19 yard pass, three incompletions, and a TD pass, it's still worth exactly 3.4 split up over 4 plays:

2.55 - 0.22 - 0.58 - 1.55 + 3.20

If a TD on 4th and goal was worth 4.85, that series of plays would be more valuable than either of the above, which makes no sense to me at all. ]]>

What you have to do is not compare the value of the touchdown to the value of that down, but rather to the value of the *next* down. That is, we're using as theory the comparison of a touchdown or a play down to the one.

Think of a fourth down play. It's 4th and goal at the one. QB passes, incomplete. His team is now in a +1.55 sitution. Why? Because for the *offense*, being at your own 1 yard line on 1st and 10 is worth -1.55 points. So the QB's fourth down incompletion puts his team at +1.55; a fourth down TD would have put his team at +6.4. So the value of his fourth down *touchdown* is 4.85.

Keep in mind that the starting field position on a play is irrelevant. Consider fourth and goal from the 10. A play to the 1 puts the team in a +1.55 situation; a play to the end zone puts the team in a +6.4 situation. The value of the fourth down touchdown is still 4.85.

So you're always comparing the value of a touchdown on down X to the value of being in down X+1. By definition, a long TD must be worth more than a TD on first and goal. Why? Because we're comparing to a play that gets you down to the one yard line. In the former example, you're in 1st-and-G from the one; in the latter, it's 2nd-and-G from the one.

]]>