Pro Football Reference Blog

A few weeks ago, footballoutsiders linked to my ten thousand seasons series. A general theme among the comments was that the simulation was inaccurate because it was based on season-long power ratings instead of last-few-weeks power ratings. Because teams' true strengths vary so much during the course of a season, I should have used a smaller but more recent sample instead of using all the data. Less is more. I thought about that for awhile and pondered the possibility that those folks might have a good point.

That caused me to try to build a power rating system based on at-the-time strength of schedule, which I was unable to do. But a by-product of the effort was this post about at-the-time strength of schedule. Interestingly, the majority of the respondents to it felt that taking a five-week slice of data introduced too much variability into the numbers. Use all the data. More is more. I thought about that for awhile and pondered the possibility that those folks might have a good point too.

So I decided to do a quick check. I looked at all games in weeks 10--13 during the years 1990--2005. For each game, I recorded the following information:

• the difference between the two teams' full-season at-the-time ratings according to the simple rating system.


• the difference between the two teams' last-5-weeks at-the-time ratings according to the same system.

So if it's week 12 of 2005 and San Francisco is playing Tennessee, we look at their week 1--11 ratings (which rate the Titans as about 5 points better) and their week 7--11 ratings (which rate the 49ers a couple of points better). I chose to look only at weeks 10 through 13 because week 10 is late enough to show some differentiation between the full-season and at-the-time ratings, and week 13 is early enough that most teams haven't given up or started to rest their regulars or whatever.

Now that we've got all the data collected, we run a logit regression to build a formula that will predict the winner of each game. Result: the at-the-time rating was not significant (in the official statistical sense). That means: if you know the full-season ratings, then there is not sufficient evidence to conclude that knowing the last-5-weeks ratings helps you predict the winners of this week's games.

If you build a formula that uses just at-the-time ratings, it will predict about 62% of the games correctly. If you build a formula that uses just full-season ratings, it will predict about 66.4% of the games correctly. If you build a formula that incoporates both, it will predict about 66.6% of the games correctly.

Interesting.

One problem here is that the simple rating system does not take home field advantage into account. It could be modified to do so, but I've never bothered because NFL teams always play the same number of home and road games during the course of a season. But that's not true in a 5-week stretch, so the last-5-weeks ratings have a bit of noise included in them. I'm not sure how much of a difference that makes, but it might make some.

Assuming the above paragraph doesn't invalidate the study, this looks like pretty clear evidence that, in this case, less is not more.