SITE NEWS: We are moving all of our site and company news into a single blog for Sports-Reference.com. We'll tag all PFR content, so you can quickly and easily find the content you want.
Also, our existing PFR blog rss feed will be redirected to the new site's feed.
Pro-Football-Reference.com » Sports Reference
For more from Chase and Jason, check out their work at Football Perspective and The Big Lead.
ShareThisAt-the-time strength of schedule
The Steelers and the Saints had the same strength of schedule last year. The Steelers' opponents' record, after throwing out the games against Pittsburgh itself, was 121-119. The Saints' opponents' record was the same.
But it seems that Pittsburgh's opponents were playing a lot better at the time than New Orleans' were. If you look at each opponent's record in just the two weeks before and the two weeks after they played the team in question, you get a different picture of their strength of schedule.
For instance, here is Pittsburgh's:
1 ten beat bal, lost to stl
2 hou lost to buf, lost to cin
3 nwe beat oak, lost to car, lost to sdg, beat atl
5 sdg beat nyg, beat nwe, beat oak, lost to phi
6 jax lost to den, beat cin, lost to stl
7 cin lost to jax, beat ten, beat gnb, beat bal
8 bal beat cle, lost to chi, lost to cin, lost to jax
9 gnb lost to min, lost to cin, beat atl, lost to min
10 cle lost to hou, beat ten, beat mia, lost to min
11 bal lost to cin, lost to jax, lost to cin, beat hou
12 ind beat hou, beat cin, beat ten, beat jax
13 cin lost to ind, beat bal, beat cle, beat det
14 chi beat tam, beat gnb, beat atl, beat gnb
15 min beat det, beat stl, lost to bal, beat chi
16 cle lost to cin, beat oak, beat bal
17 det lost to cin, beat nor
Here's how you read that mess:
In week 1, Pittsburgh played Tennessee. In the weeks surrounding the Pittsburgh game, Tennessee beat Baltimore and lost to St. Lous. In week 14, Pittsburgh played Chicago. In the surrounding weeks, Chicago beat Tampa, Green Bay, Atlanta, and Green Bay again.
If you tally it all up, you'll find that Pittsburgh's opponents were 32-24 in the two weeks before and after playing the Steelers. Note in particular the last weeks of the season. From week 12 on, Pittsburgh's opponents were beating almost everyone except Pittsburgh (and other teams that Pittsburgh played during that stretch).
New Orleans' opponents, on the other hand, were not doing as well around the time they played the Saints.
1 car beat nwe, lost to mia
2 nyg beat ari, lost to sdg, beat stl
3 min lost to tam, lost to cin, lost to atl
4 buf lost to tam, lost to atl, beat mia, beat nyj
5 gnb lost to tam, lost to car, lost to min
6 atl beat min, lost to nwe, beat nyj
7 stl lost to sea, lost to ind, beat jax
8 mia lost to tam, lost to kan, lost to atl, lost to nwe
9 chi beat bal, beat det, beat sfo, beat car
11 nwe lost to ind, beat mia, lost to kan, beat nyj
12 nyj lost to car, lost to den, lost to nwe, beat oak
13 tam beat atl, lost to chi, beat car, lost to nwe
14 atl beat det, lost to car, lost to chi, lost to tam
15 car beat atl, lost to tam, lost to dal, beat atl
16 det lost to gnb, lost to cin, lost to pit
17 tam lost to nwe, beat atl
When you add it up, Pittsburgh's at-the-time strength of schedule was 32-24 and New Orleans' was 21-33. Even though their overall strengths of schedule were identical, it might be the case that the Steelers were actually playing a tougher slate.
Here is the full list.
TM YR Local Overall Diff
=========================================
sfo 2005 35- 20- 0 126-114- 0 +0.111
nwe 2005 33- 21- 0 124-116- 0 +0.094
pit 2005 32- 24- 0 121-119- 0 +0.067
dal 2005 31- 21- 0 127-113- 0 +0.067
jax 2005 30- 25- 0 115-125- 0 +0.066
nyj 2005 29- 22- 0 123-117- 0 +0.056
buf 2005 28- 24- 0 117-123- 0 +0.051
ind 2005 28- 25- 0 115-125- 0 +0.049
min 2005 29- 26- 0 117-123- 0 +0.040
nyg 2005 31- 26- 0 121-119- 0 +0.040
bal 2005 29- 25- 0 124-116- 0 +0.020
cle 2005 29- 27- 0 120-120- 0 +0.018
kan 2005 29- 26- 0 123-117- 0 +0.015
gnb 2005 28- 25- 0 124-116- 0 +0.012
sea 2005 25- 30- 0 107-133- 0 +0.009
det 2005 27- 27- 0 118-122- 0 +0.008
ten 2005 27- 28- 0 119-121- 0 -0.005
phi 2005 28- 26- 0 126-114- 0 -0.006
sdg 2005 29- 23- 0 136-104- 0 -0.009
den 2005 27- 26- 0 125-115- 0 -0.011
chi 2005 24- 29- 0 112-128- 0 -0.014
oak 2005 28- 27- 0 126-114- 0 -0.016
hou 2005 26- 27- 0 123-117- 0 -0.022
car 2005 23- 30- 0 110-130- 0 -0.024
stl 2005 23- 31- 0 114-126- 0 -0.049
mia 2005 22- 32- 0 110-130- 0 -0.051
was 2005 27- 28- 0 132-108- 0 -0.059
tam 2005 21- 33- 0 110-130- 0 -0.069
cin 2005 22- 32- 0 117-123- 0 -0.080
atl 2005 21- 34- 0 118-122- 0 -0.110
nor 2005 21- 33- 0 121-119- 0 -0.115
ari 2005 20- 33- 0 119-121- 0 -0.118
My original intent in looking into this was to develop a rating system that would use this kind of strength-of-schedule number instead of the full season number. I ran into some technical troubles, but I'll get them sorted out sooner or later.
This entry was posted on Monday, July 3rd, 2006 at 4:11 am and is filed under General. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.
I like the concept. I've often argued that the 2002 Patriots played a really tough schedule because of how teams were playing at the time of the contest. That year, the Pats seemed to meet a lot of teams early (San Diego, Miami, Denver) who got off to fast starts (and would stumble later), and meet a lot of teams later (Raiders, Titans, Jets) who finished the season on hot streaks.
One comment: your full list seems to be missing some teams eg. Cincinnati.
Thanks Ryan. I corrected the table to include the missing teams.
The 2002 Pats played a tough schedule by any measure, but it doesn't appear that it was noticeably tougher looking at it this way:
1 pit lost to oak
2 nyj beat buf, lost to mia, lost to jax
3 kan beat cle, lost to jax, beat mia, beat nyj
4 sdg beat hou, beat ari, lost to den, beat kan
5 mia beat nyj, lost to kan, beat den, lost to buf
6 gnb beat car, beat chi, beat was
8 den lost to mia, beat kan, lost to oak
9 buf beat mia, beat det, lost to kan
10 chi lost to min, lost to phi, lost to stl, beat det
11 oak lost to sfo, beat den, beat ari, beat nyj
12 min lost to nyg, beat gnb, lost to atl, lost to gnb
13 det lost to nyj, lost to chi, lost to ari, lost to tam
14 buf lost to nyj, beat mia, beat sdg, lost to gnb
15 ten beat nyg, beat ind, beat jax, beat hou
16 nyj beat den, lost to chi, beat gnb
17 mia beat oak, lost to min
nwe 2002 29- 25- 0 127-112- 1 +0.006
This is pretty neat Doug, but do you have any way of knowing if this isn't just due to random variation? After all, as you said, "splits happen". I know that some of these are rather obvious, like philadelphia sucking at the end of the year because of injuries to Mcnabb and the T.O. situation, but what about the others? Could random variation explain these streaks away, or can we be sure that something else is at work here?
I often wondered that my strength of schedule in my fantasy league seemed low because I am so much better than the competition. Partly true, of course, but I just realized its because I don't have to play me! Boy, I feel sorry for those other 11 poor suckers. GLS.
Interesting. I think a good corollary to this line of thought would be "At-the-time Adjusted Records". I've always thought the ebb and flow of power sometimes changes during the football season. Take the Colts in the late 80s. They started the season 0-13 and ended up 3-13. A 3-win season sounds pathetic, but ending the season with 3 wins in a row is very respectible. The temptation is to say they weren't a good team, but they were 3-0 over a 3 week stretch. Nobody had a better record in that time. In my opinion, a W-L record has exactly 1 purpose: to determine who goes to the playoffs.
Look at the Bills in 2004. In the midst of their dominating 6-0 stretch near the end of the season in which they outscored everybody by 20+ points, a common sentiment was "if this team was any good, they would have beaten Jacksonville week 1". How good they were in games 10-15 has nothing to do with their efforts in game 1.
I believe Sagarin counts all games equally in his ratings. I fail to see how that can be a true measure of a team's quality.
Very true, Alex. That's why I was careful to say "Even though their overall strengths of schedule were identical, it might be the case that the Steelers were actually playing a tougher slate."
It's the usual sample size tradeoff. If you want to know how strong Chicago was when they played Pittsburgh, is it better to use a small-but-more-relevant sample or a big-but-possibly-less-relevant sample? The bigger sample has the advantage of decreasing the random variation, but it also is incapable of recognizing when a team truly does change strength in mid-year.
I don't know where the right balance is.
This is an interesting idea, and I think Doug went about it the right way. And on a general level we want to capture whether or not Chad Pennington was playing for the Jets, TO for the Eagles, or Marc Bulger for the Rams. There are problems of course: this won't catch all the problems and it will create a few as well.
If Marc Bulger is going to miss one game, the Rams could win the prior two and next two but lose the game against Team Y (probably the Patriots), while unjustifiably inflating Team Y's rating.
The second problem is something we've discussed a lot recently: team streaks. (You can get there by clicking my name). Maybe there isn't such a thing as a hot team. If Team X goes 6-6, and then wins three games, they may be no more likely to win the last game of the season then Team C that goes 6-6, and loses their last three games. And if that's true, the two games (or whatever number you want to use) before the game against your opponent may not be very meaningful.
The third problem is the flip side. In 2002 the Chargers were 6-1, and the Jets were 2-5. Then Herm Edwards gave his "you play to win the game" speech, and the Jets went to San Diego and won 44-13. The Chargers finished the season 8-8. Maybe the Jets loss didn't propel the Chargers into a second half collapse, but maybe they did. And if they did, we're going to discredit the Jets for beating the Chargers, when we would really want to credit them.
I totally agree with Chase here. Teams should be given no credit at all for beating the Jets while Pennington was quarterbacking them.
I think two games is a too much of a microscopic view of the season. Having only a 16 game season produces enough noise as it is. I don't know if looking at the record of the team two games prior really tells you anything.
Plus at least they way I interpreted it you didn't factor in if the team's oppoenent was playing at home/away or if the oppopents oppoents wereplaying good/bad teams at the time
The way sports work in this day and age is that teams are rewarded for winning and penalized for losing. It has little to do with who's better than who; just who won more than who. Strength of schedule is a big factor, even on the basis of a given week. Suppose in a given week, the strongest team was playing against the 2nd strongest team, and meanwhile, the worst team was playing against the 2nd worst team. One of the top 2 teams would sustain a loss, while one of the worst 2 teams would get a win. Nobody in the top 2 really deserves to lose, and nobody in the bottom 2 really deserves to win. However, the junk team's win counts just as much as any of the best team's wins, and the best team's loss counts just as much as any of the junk team's losses. (Unless the Colts lose to Seattle, in which case you're not supposed to count the loss
)
---
Where am I going with this? Wins and losses, while adequate for determining standings, are a poor indicator of relative strength, even when diluted by any strength of schedule adjustments or any "at the time" considerations.
---
What's the solution? Instead of thinking of the results for a given week as 16 teams getting a win and 16 teams getting a loss, maybe each week's results should be ranked from 1 to 32 somehow. Suppose the Saints beat the Texans the same week the Panthers beat the Bears 21-17 in a game for the ages. No way should the Saints win count as more of an accomplishment than a game Bears effort against another top echelon team in terms of who's better than who.
Dave makes two very good points.
The premise underlying at-the-time SOS is that a given team's intrinsic strength has a significant trend to it. I am at least skeptical about the idea.
To test the idea, I used my power ratings to predict the result of every NFL game from 2005, then compared those predictions to the actual game outcomes. My power ratings consist of an intrinsic strength of each team, and a league wide HFA. Any outcomes that differ from the predictions are assumed to be due to a random game-to-game variance. The power rankings correctly predicted the outcome of about 75% of the games. I then looked at the upsets and looked for strings of games meeting the following criteria:
a) the string of games had to have at least 3 upsets and all the upsets had to go in the same direction (either for or against the team of interest)
b) games that went as expected were allowed in the string as long as
c) there were no more than two expected results between expected results. (Rule c only eliminated Philadelphia, as explained below).
I'm not entirely happy with the above test, but here are the teams that seemed to have a distinct changes in their intrinsic strength in 2005:
Miami showed the clearest trends. I'll use them as an example to show my analysis:
Week 1: Defeated Denver, unexpected
Week 2: Lost to the Jets, unexpected
Week 3: Defeated Carilolina, expected
Week 5: Lost to Buffalo, unexpected
Week 6: Lost to Tampa Bay, expected
Week 7: Lost to KC, unexpected
Week 8: Defeated NO, expected
Week 9: Lost to Atlanta, unexpected
Week 10: Lost to NE, toss up
Week 11: Lost to Cleveland, expected
Week 12: Defeated Oakland, expected
Week 13: Defeated Buffalo, expected
Week 14: Defeated SD, unexpected
Week 15: Defeated Jets, expected
Week 16: Defeated Tennessee, expected
Week 17: Defeated NE, unexpected
The significant streak began in week 2 with the unexpected loss to the Jets and spanned to week 9. In between were two more unexpected losses and three expected results. Late in the season they had another streak bookened by unexpected wins over SD and NE, but since this streak did not have 3 unexpected results all going the same direction, I didn't count it.
Tampa Bay had a mid-season slump, being upset in games 5, 8, 9, and 12 by the Jets, San Francisco, Carolina and Chicago (weeks 9 and 12 were weak upsets).
Oakland had a slight surge in the middle of the season having unexpected wins in weeks 4, 8, and 11 against Dallas, Tennessee, and Washington. All the other results during that stretch were expected.
Many people talk about how the TO situation and McNabb's injury made Phili a much worse team by the end of the season than they were at the beginning of the season. This did not show up clearly in my analysis. They had a strong upset of KC in week 4 and another upset of SD in week 7. Finally they had a mild upset of St.Louis in week 15. The rest of Philadelphia's games went as expected from their season-average power rating. I didn't count this as a significant trend because of the huge gap between week 7 and 15.
Every other team had at most two upsets going in the same direction in a row. While this analysis is far from definitive, it seems to argue against the idea that team's intrinsic strength varies as a clear trend as the season progresses.
dang typos.
"c)there were no more than two expected results between [i] unexpected[/i] results. (Rule c only eliminated Philadelphia, as explained below).
Why are some teams better than others?
Hey Doug, I've been wondering about your "simple rating system": how difficult is it for you to find the ratings for a certain year? If it's not too hard, would it be possible to include these ratings for each team each year with the team's statistics?
2004 Panthers. Everyone talked about how slow they started and how well they finished, but in truth, they just played to their schedule. They lost to one 5-11 team (Raiders) and beat one 10-6 team (Seahawks). The Raiders were in one of the stronger divisions and the Seahawks were in the weakest division.
Essentially, the Panthers beat all their really bad opponents (six wins or less) and lost to all their good opponents (ten wins or more). The split against average teams (between seven and nine wins inclusive, plus Raiders and Seahawks). If it hadn't been for the way that their schedule fell out (most of the tough teams early and easy teams in the late middle), then they wouldn't necessarily have had those exciting streaks.
It would also be interesting to do the cdcox analysis on a team like the 2005 Steelers. How much did Batch/Maddox affect them? They were 7-5 with Roethlisberger in and out and then 8-0 with him healthy (or 11-1 with Ben healthy and 4-4 not; 13-1 in games started by healthy Ben or Batch; 2-4 in games started by Maddox or injured Ben -- both wins by Ben following the first injury).
Finally, does this do better or worse than season strength of schedule at predicting the results of games? I find it a little troubling for this measure that the Steelers opponents who won had a local strength of schedule of .579 (11-8) and those who lost had a .568 (21-26). Especially since the season record differential was about .640 to .442 (throwing out the game being counted but keeping any other games; consistent with the 121-119) or .658 to .441 (throwing out all games played against the Steelers; total would be 119-115). Dropping splits (Ravens/Bengals) and counting two wins as one (Browns), this goes to .733 to .420 (83-81).
If that (negligible difference between records in wins and losses) holds for other teams and seasons, then I think that this is one of those curiousity stats more than a useful stat.
Hey Matt,
Not sure who you mean but the 2004 Panthers lost to the 9-7 Seahawks anyway. As far as I can tell, Carolina didn't beat a single team with a winning record that year.
Natt:
The break down of the Steelers game by starting QBs is interesting. The Steelers had 6 unexpected results last year. Batch didn't play in any of those games.
Roethlisberger was 2-2 in games with unexpected results. He upset SD and Cincinnati (road) in weeks 5 and 7 and lost to NE and Cincinnati (home) in weeks 3 and 13.
Maddox was 0-2 in games with unexpected results, losing to Jacksonville and Baltimore.
The Steelers run through the playoffs was extrordinary. They were all unexpected results. I do think the playoffs is the one period of the season where teams often perform differently than predicted by their intrinsic strength ratings.