Pro Football Reference Blog

Over a week ago, the San Diego Chargers became the first team to fire a head coach following a a fourteen-win season. Marty Schottenheimer's team lost its first playoff game, which seems less punishable when you remember what happened the previous two years. In 2004, a 15-1 Steelers team was a Doug Brien field goal away from losing its first playoff game, and got blown out the next week at home; the following year, Bill Cowher brought the city of Pittsburgh its fifth Super Bowl Championship. In 2005, a 14-2 Colts team lost its first playoff game; the following year, Tony Dungy brought the city of Indianapolis its first professional sports title ever (discounting the three ABA titles won in the early 1970s).

Marty Schottenheimer won't get a chance to bring the city of San Diego its first professional sports title (discounting the AFL title in 1963), and you'll hear lots of reasons why. If Schottenheimer was Bill Belichick, we know he wouldn't have been fired. But Belichick has a past history of post-season success, and Schottenheimer has a horrible history of playoff failure. Almost assuredly, if Schottenheimer did not have a poor career record in the playoffs, he would have been retained. While the loss of both assistant coaches was significant, it is my opinion that the overriding factor was the thought that "Marty won't win in the playoffs." This can only make sense if past post-season success is indicative of future post-season success. To make my bias clear, before I conducted this study I believed that statement to be false. Let's see what happens. (Note: I don't care to turn this into a debate on the reasons Schottenheimer was fired. There's currently an 847 post thread on that at our other site.)

From 1970-2005, there were 346 playoff games played in the NFL. To figure out if past playoff prowess is correlated with future post-season success, we need to isolate two factors: regular season record and home field advantage. Because regular season record is highly correlated with home field advantage (the team with the better record has usually been the home team), we're going to leave HFA out for now to make this a bit more palatable.

I hate having to write keys for charts, because that usually means the data isn't presented in its simplest format. But this was the best I could do. Every playoff game has a "clutch" coach and a "choke" coach. The clutch coach is simply the coach with the better career post-season record prior to that game ("better" will be explained in a bit).

RWD = Regular Season Win Differential
N = Number of times two teams met in the playoffs with that differential
Cl Win% = Winning percentage of the "clutch" coaches when they were X games better in the regular season than the opponent.
Ch Win% = Winning percentage of the "choke" coaches when they were X games better in the regular season than the opponent.
Cl Gm = Number of times the "clutch" coach had the better record
Ch Gm = Number of times the "choke" coach had the better record
Ev Gm = The number of times teams with that RWD met and the two coaches had "equivalent" prior post-season records. Equivalent here means both coaches were the same number of games above, at, or below .500. This is only to be complete, since we won't care about these games.

RWD	  N	Cl Win%	Ch Win%	Cl Gm	Ch Gm	Ev Gm

6	  2	1.000	1.000	  1	  1	 0

5	  9	0.667	1.000	  3	  5	 1

4	 20	1.000	0.625	  9	  8	 3

3.5	  3	1.000	1.000	  1	  1	 1

3	 35	0.900	0.833	 20	 12	 3

2.5	  4	1.000	0.500	  1	  2	 1

2	 74	0.857	0.657	 28	 35	11

1.5	 12	0.600	1.000	  5	  3	 4

1	112	0.682	0.623	 44	 53	15

0.5	 15	0.429	0.500	  7	  2	 6

0	120	0.538	0.462	 52	 52	16

First, a quick example. When the 1998 (15-1) Vikings played the 1998 (9-7) Cardinals in the playoffs, Dennis Green had a career 1-4 post-season record and Vince Tobin was 1-0 in the playoffs. Green's Vikings won, so that game is filed under RWD as 6, under Ch Gm as 1 (this was the only time the "choke" coach ever had six more regular season wins than his opponent) and under Ch Wins (not presented above) as 1. Then I divided Ch Wins by Ch Gm to get the Ch Win%, which is presented above. Whew.

Let's summarize the table. When two teams face off in the post-season where one team has won five or six more games than the other, the team with the better record (regardless of coaching history) is 11-1. The one loss was when Jerry Burns (1-0) beat mighty Bill Walsh (7-3) in the playoffs, so that's a "clutch" loss.

At four wins better than the opponent, "clutch" coaches are 9-0 but "choke" coaches are only 5-3. Tom Coughlin's upset of Mike Shanahan (1996), Ted Marchibroda's upset of Marty Schottenheimer (1995) and Chuck Noll's upset of Dan Reeves (1984) were the three surprises. All three were by seven points or less. Note that Shanahan (0-0) was considered the "choke" coach and Coughlin (1-0) the "clutch" coach by only the thinnest of margins. We'll address this later today and more thoroughly tomorrow.

At 3/3.5 games better, clutch coaches are 19-2 (John Robinson over Tom Landry in 1984, Chuck Knox over Don Shula in 1983), while choke coaches are 11-2 (Bill Cowher over Tony Dungy, 2005, and Bill Belichick over Mike Martz in SBXXXVI). This illustrates one of the drawbacks of such an approach. Robinson and Knox were both six games behind Landry and Shula (in terms of career post-season records) when they faced, and were clearly big underdogs with respect to playoff success. Martz and Dungy were only one and two games behind Belichick and Cowher at the time, so they had nearly identical playoff records when they faced. So the two clutch losses were much more extreme than the two choke losses.

A wide gap emerges, however, at 2/2.5 games better. Clutch coaches are 25-4, a very respectable winning percentage. Choke coaches are 24-13, which is decidedly more average. Interestingly, in the most extreme discrepancies in games where the choke coach was on a team with two more regular season wins, the choke coach followed history and lost. Dennis Green lost his post-season debut to Joe Gibbs, whose 15-4 record in the playoffs may have mattered more than his team's 9-7 regular season record in 1992. Additionally, Bill Belichick (10-1) beat Jack Del Rio in his post-season debut, but then again, that game was in Foxboro.

The two sets of data converge again when the two regular season teams were within 1.5 games of each other. Both the clutch coach (36-20) and the choke coach (37-21) won 64% of their games when they coached a team with a slightly better regular season record.

When two teams have the same regular season record, clutch coaches have a slight edge, winning 28 of the 52 games. If we had no other data to analyze, this would be what I'd be most curious to see. When the teams are even, who wins? There could be several factors affecting this, so 28/52 isn't conclusive of much.

When coaching a much stronger team, measured by regular season record, both clutch and choke coaches dominate in the post-season. When coaching teams that are a significant but not large amount better, clutch coaches have been noticeably more successful. When coaching teams that are slightly better, clutch and choke coaches appear identical.

As hinted at earlier, we may not be comparing apples to apples. If Coach A has a 1-0 post-season record, and he faces Coach B who owns a 0-1 post-season record, Coach A will be labeled clutch and Coach B will be labeled choke. If Coach A is 10-0 and Coach B is 0-10, the same labels -- clutch and choke -- will apply. But presumably we'd want to focus more heavily on games where there is a large difference in the post-season records. Otherwise, it would be like writing the difference between a 15-1 team and 9-7 team is the same as the difference between a 10-6 team and 9-7 team. Labeling them "good" and "bad" isn't very precise.

We just looked at how the "good" team in every post-season game did (good meaning has X many more regular season wins than the opponent) depending on whether the coach was previously clutch or a choker. Now we're going to look at "clutch" coaches in every post-season game, and see how they fare depending on whether they're coaching a "good" team or a "bad" team. This is susceptible to the same problems, of course, but gives us another way to look at the data. The only reason we talk about clutch coaches in the sense of prior post-season success is because we assume that a clutch coach can beat a choke coach with a bad team. When a good team beats a bad team, we aren't surprised. But how often do "clutch" coaches lead inferior teams to post-season success, and vice versa, how often do "choke" coaches hamper superior teams?

In this chart, we'll need a third column -- even games. Before we dismissed even games because we were analyzing clutch coaching, and if neither coach is clutch, we don't care about the game. Now we might care most about the even games, because that features two teams with the same records.

CF = Clutchness Factor. How many more career post-season wins above .500 the clutch coach had.
N = Number of games where the CF differential was X.
G Win % = Winning percentage by the clutch coach when he had the "good" team (better regular season record)
E Win% = Winning percentage by the clutch coach when the two teams were "even" (same regular season record)
L Win % = Winning percentage by the clutch coach when he had the "bad" team (worse regular season record)
G Gm = Number of games where the clutch coach was on the good team
E Gm = Number of games where the two teams were even
B Gm = Number of games where the clutch coach was on the bad team

CF	N	G Win%	E Win%	B Win%	G Gm	Ev Gm	B Gm

10+	 4	---	0.500	0.500	 0	 2	 2

9	 5	---	---	0.400	 0	 0	 5

8	 6	1.000	1.000	0.500	 3	 1	 2

7	 4	1.000	---	0.500	 2	 0	 2

6	22	0.857	0.333	0.083	 7	 3	12

5	39	0.588	0.600	0.176	17	 5	17

4	24	0.615	0.500	0.400	13	 6	 5

3	50	0.788	0.600	0.250	33	 5	12

2	61	0.900	0.786	0.370	20	14	27

1	79	0.792	0.313	0.385	24	16	39

Once again, let's go through a quick explanation and a summary. I'm measuring a coach's record by how many games over .500 he is. If you're 10-5, you're at 5 games over. If you're at 6-13, you're at 8 games under. If those two coaches met, I'd record the difference as +12. This formula works well enough, and the most important thing is that we all know what the formula is, rather than finding the perfect formula.

When Mike Holmgren (9-8) met Joe Gibbs (17-5) in the second round of the 2005 NFC playoffs, Gibbs would be the clutch coach and filed under 10+ wins (since he's actually at +11). Gibbs lost, and he was on the "bad" team since Seattle had won three more games than Washington that year. But when Joe Gibbs (15-4) beat Dennis Green (0-0) thirteen years earlier, he also coached the worse team. Those are the only two times a coach with a 10+ advantage over his opponent coached the team with the worse record in the playoffs (and such a coach has never coached the better team).

On to the summary. When a playoff game features a clutch coach with a large advantage (7 games or more), the clutch coach is 5-0 when coaching the better team and 2-1 when the teams are even. When coaching the worse team, the clutch coach is 5-6. These numbers are more significant than you might initially realize; this means the choke coach is 0-5 when coaching the worse team (compared to 5-6 when the clutch coach has the worse team) and just 6-5 (vs. 5-0) when coaching the better team. Curiously, though, in the most extreme example, the choke coach won. Don Coryell (2-5) met Chuck Noll (14-4) in the playoffs in Pittsburgh, and both teams had gone 6-3 in the regular season. But Coryell's Chargers edged Noll's Steelers, 31-28.

The evidence goes the other way, however, when we look at times when the clutch coach had a 5 or 6 game edge on his opponent. When coaching good teams, he was 16-8; when coaching the worse team he was just 4-25. The converse means while coaching bad teams the choke coach still won 33% of his games, and the "choke" coach won nearly all of the games when he had the better team. The four losses? Chuck Noll (7-2) over Ted Marchibroda (0-1), in 1976; Dan Reeves (9-7) over Dennis Green (2-5) in the 1998 NFC Championship Game, Mike Shanahan (1-1) over Marty Schottenheimer (5-10) in 1997, and Herm Edwards (1-2) over Schottenheimer (5-11) in 2004. Outside of those games, the evidence strongly points to clutch coaches doing worse than choke coaches for this stretch. There are many more games like Bill Parcells losing to John Fox than Herm Edwards beating Schottenheimer.

When we look at coaches with 3 or 4 game advantages, we see a very small edge going to the "clutch" coaches. In those games clutch coaches are 6-5 against choke coaches when both teams have the same regular season record. When the clutch coach is on the good team, his record is 34-12 (74%), and when the clutch coach is on the bad team, his record is 5-12 (29%). Conversely, choke coaches win 71% of the time and 26% of the time when on the good, and bad teams, respectively.

This effect is magnified even more when we look at coaches with just slight advantages, a one or two game lead. I'm not sure this is conclusive of anything, because if there is something to this clutch ability, it shouldn't increase as we get to the least clutch coaches. Anyway, clutch coaches are 37-7 when they coach the good team, while choke coaches are just 41-25 when they're on the good team. Clutch coaches are also 16-14 against the chokers when the teams are even.

So where does that leave us? None of the above methods are perfect, since there are some drawbacks to those tools. In both examples, we made team strength (good/bad) and coaching history (clutch/choke) into binary categories, when of course they are not. As a result, some effects could be hidden. The best way to solve this is to use a regression analysis. I didn't do that today because regression analysis is useless to people who don't understand regression. The tables presented at least bring the numbers to life. Tomorrow, though, we'll sacrifice simplicity for precision, and the results are pretty interesting.