SITE NEWS: We are moving all of our site and company news into a single blog for Sports-Reference.com. We'll tag all PFR content, so you can quickly and easily find the content you want.

Also, our existing PFR blog rss feed will be redirected to the new site's feed.

Pro-Football-Reference.com » Sports Reference

For more from Chase and Jason, check out their work at Football Perspective and The Big Lead.

Coaching changes and lurking variables

Posted by Doug on June 12, 2006

Lots of teams changed head coaches this year.

I have a crazy theory that changing coaches just for the sake of changing coaches is usually a good thing. Maybe not every year, but unless my team's record is good and getting better, I'd be inclined to fire the coach and start afresh every two or three years. I realize this is contrary to the coaching-continuity-is-everything line of thinking that the Steelers' Super Bowl title has wrought, so I probabaly should take a full blog post to explain it more clearly sometime. But for today, let's just do a quick investigation of how teams do after changing coaches.

Since 1990, we have seen 80 teams change head coaches between seasons. On 53 of those occasions, the team improved its record, and another four times the team's record stayed the same. If we count those four as half improve and half decline, then we get that 55 of the 80 teams improved. That's 68.8%. During the same span, teams that did not change coaches improved their record only 45.7% of the time. [Technical note: I have thrown out teams that changed coaches in mid-season in either year N or year N+1. My intent was to focus on situations where a conscious between-seasons choice was made between continuity and a new direction.]

Despite the fact that it appears to support my point, that is an extremely misleading statistic. While there is unquestionably an association between changing coaches and improving your record, there may or may not be a causal relationship. There is a lurking variable here, and its name is team quality. In the NFL, bad teams tend, as a group, to improve their records more often than good teams do. Bad teams also tend to change coaches more than good teams do. These two facts might lead to the statistic we saw above even if there is no causal relationship at all between changing coaches and improving your record.

One simple way to remove this bias would be to group the teams into groups of roughly equal quality. So let's do that.


YrN === changed coaches ===+=== did not change ======
Wins Imp Same Dec Imp% | Imp Same Dec Imp%
=====+========================+=========================
0-3 | 14 1 0 96.7 | 9 1 1 86.4
4-5 | 15 0 1 93.8 | 27 5 9 72.0
6-7 | 14 0 8 63.6 | 42 6 23 63.4
8-9 | 8 1 9 47.2 | 34 10 38 47.6
10+ | 2 2 5 33.3 | 23 17 93 23.7
=====+========================+=========================
53 4 23 68.8 | 135 39 164 45.7

From 6 wins on up, there is no difference between the improvement rate of the teams that changed coaches and those that didn't. For the 10+ win group, the percentages appear to be different, but a switch of just one team from the improved to the declined column would make them almost identical.

For the really bad teams, though, the story might be different. I'm too lazy to run the appropriate statistical tests, but it is notable that bad teams that change coaches improve "almost always" while bad teams that don't change coaches only improve "usually." Obviously, what's needed now is to see if that improvement sticks. Perhaps all the gains from the first year are given back in the next year. I'll put an investigation of that on the to-do list.

This entry was posted on Monday, June 12th, 2006 at 4:10 am and is filed under General. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.