SITE NEWS: We are moving all of our site and company news into a single blog for Sports-Reference.com. We'll tag all PFR content, so you can quickly and easily find the content you want.
Also, our existing PFR blog rss feed will be redirected to the new site's feed.
Pro-Football-Reference.com » Sports Reference
For more from Chase and Jason, check out their work at Football Perspective and The Big Lead.
ShareThisAlgorithmicizing the human polls
I still have not finalized the details of my attempt to algorithmicize college football's human polls. Since this project is too silly to post on any day other than Friday, and since this is the last Friday before the season starts, I guess I'd better nail it down right here.
In the above-linked post, I laid out a rough draft of an algorithm for ranking college football teams that would match up with the human poll at the end of the year. I knew that that formula in that post probably wouldn't end up being the finished product, and was hoping to get some suggestions from the readership (that's you). As usual, I did get good suggestions. In particular, both JKL and Pat noted that there needs to be a mechanism for vaulting teams up in the rankings when they beat a team ranked ahead of them, especially early in the season. They're right.
So here is the new system:
STEP 1: Build the preseason rankings - this will be done just as described in the earlier post:
1a. First rank all the BCS conference schools
1a(i). put them in order of last year's final poll (including the "others receiving votes" part)
1a(ii). order the non-vote-receiving teams by their 2006 winning percentage, with ties broken by perceived conference superiority: SEC > Big 10 > Big 12 > Pac 10 > ACC > Big East (note: since Notre Dame was "ranked" at the end of 2006, there is no need to make a special rule for them. Should the need arise in future years (yeah, like this thing is going to survive past week 3, much less into future years), they will be treated as a Big 10 team.)
1b. rank the non-BCS teams, first using last year's final poll, then ranking the rest by record, ties broken alphabetically (A's first)
1c. put all the non-BCS teams behind all the BCS teams to create the preseason rankings.
STEP 2: weekly adjustments
2a. deal with the losers first
2a(i). teams that lose by 9 or fewer points to a team ranked higher drop 2 spots.
2a(ii). teams that lose by 10 or more points to a team ranked lower drop 10 spots.
2a(iii). all other losing teams drop 5 slots.
2a(iv). all losing teams drop an additional 5 slots if they lost their previous game as well.
2a(v). all ties broken by the previous week's rankings. [for example, if the #1 team loses by 3 points, they should drop to #6. Meanwhile, if the #4 team loses to the #2 team by 3 points, they should also drop to #6. In this case, the former #1 would be #6 and the former #4 would be #7.]At this point all the losers have a ranking. We need only order the rest of the teams, then fill them in to the empty slots in order.
2b. ordering the non-losers. (All references to "teams" below refer only to the non-losing teams.)
2b(i). starting with the lowest-ranked (worst) team, we look at each team in turn. If they beat a top 25 team, then they leapfrog int((.5 - .02r_1)r_2) teams, where r_1 is the previous rank of the beaten team, r_2 is the previous rank of the winning team, and int() is the greatest integer function (i.e. round down).
That's it. I'd like to point out that, even though it took a fair amount of verbiage, this isn't as complicated as it seems. The basic idea is: drop the losers down, move up the big winners, and fill everyone else in in the same order as the previous week.
I honestly have no idea how this is going to work out. I hope I'll be able to keep up with it throughout the season. Here is the preseason poll:
1. Florida
2. Ohio State
3. LSU
4. USC
5. Wisconsin
6. Louisville
7. Auburn
8. Michigan
9. West Virginia
10. Oklahoma
11. Rutgers
12. Texas
13. Cal
14. Arkansas
15. Wake Forest
16. Virginia Tech
17. Notre Dame
18. Boston College
19. Oregon State
20. Tennessee
21. Penn State
22. Georgia
23. Nebraska
24. Texas A&M
25. Georgia Tech
26. South Carolina
27. Maryland
28. Texas Tech
29. Kentucky
30. South Florida
31. Missouri
32. Clemson
33. Cincinnati
34. Purdue
35. Kansas State
36. Oklahoma State
37. UCLA
38. Oregon
39. Arizona State
40. Florida State
41. Miami (FL)
42. Kansas
43. Washington State
44. Arizona
45. Pittsburgh
46. Alabama
47. Minnesota
48. Iowa
49. Indiana
50. Washington
51. Virginia
52. Ole Miss
53. Vanderbilt
54. Northwestern
55. Michigan State
56. Baylor
57. Iowa State
58. UConn
59. Syracuse
60. Mississippi State
61. NC State
62. North Carolina
63. Illinois
64. Colorado
65. Stanford
66. Duke
67. Boise
68. BYU
69. TCU
70. Hawaii
71. Houston
72. Central Michigan
73. Navy
74. San Jose State
75. Ohio
76. Southern Miss
77. Nevada
78. Troy
79. Tulsa
80. Utah
81. Western Michigan
82. East Carolina
83. Middle Tennessee
84. Northern Illinois
85. Rice
86. Arkansas State
87. Kent State
88. La-Lafayette
89. SMU
90. Wyoming
91. New Mexico
92. Akron
93. Ball State
94. Florida Atlantic
95. Marshall
96. Toledo
97. UTEP
98. Air Force
99. Bowling Green
100. Fresno State
101. Idaho
102. La-Monroe
103. New Mexico State
104. Tulane
105. UCF
106. Colorado State
107. Army
108. La Tech
109. North Texas
110. San Diego State
111. UAB
112. Buffalo
113. Memphis
114. Miami (OH)
115. UNLV
116. Eastern Michigan
117. Temple
118. Utah State
119. Florida International
This entry was posted on Friday, August 24th, 2007 at 4:35 am and is filed under BCS, College. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.
I've always wondered how much the computer component of the BCS ratings is judged by how close its results come to what is expected.
It will be fun to see the results. My gut in looking at your starting list is that Boise, BYU, TCU and Hawaii (and probably only those 4) start out in the minds of pollsters higher than where you are starting them.
Boston College got very lucky to squeek by Navy in a bowl game last year. BC gets an 18 but Navy is down at 73. Ouch. What conference equivalent did you use for Navy? I'd recommend Big East.
Beat Army!
Keep in mind, Brian, that this is in no way intended to be a good ranking system. It's simple and arbitrary and there isn't much logic behind it. That's kind of the point.
Am I correct in assuming that Navy (and Army and Temple) are non-BCS teams? That is, they don't have the special deal that Notre Dame has and must get in via the Boise State route. Is that right?
I believe Temple is in the MAC now for football, but yeah the others are Boise State as to getting into the BCS. I think BYU and Boise State will get a higher rank from the polls, but I also don't think they will move up for wins as quickly as the other schools, so I think they are treated fairly. Going 10-2 or 11-1 should put them up in the top 20 I would think.
smu is too high
How does the accuracy work out against the last couple years of play?
I have no idea, Nick.
Since my premise is that the human polls are so simplistic that they can be closely approximated by writing a simple algorithm that required less than an hour of thought, it would be cheating to look at past data.
Well, I guess it wouldn't be cheating if I did it now, but it would have been cheating had I looked at the data beforehand.
I gotcha. Missed the BCS rule.
It's as if you've baked in the prevalent bias into the algorithm. Nicely done! I'm betting your system matches the human polls very well. Please keep up with it.
"I believe Temple is in the MAC now for football, but yeah the others are Boise State as to getting into the BCS."
Actually, the teams from the MAC also have to get in through the "Boise State" rule. The MAC is not a BCS conference, so Temple still needs the "Boise State" rule to get in.
Btw, Doug, I think this is awesome, and I would be interested to know how well these rankings predict the actual winners of games, and how that compares to the other ranking systems. For instance, does it have a better predictive accuracy than the BCS computer rankings, Sagarin, etc.
Better, but still a bit too simplistic. Vaulting a fixed number of teams means that late in the season (but not too late) when there are several no-loss teams, a one-loss team which beats a no-loss team will vault past several no-loss teams.
I’ve always wondered how much the computer component of the BCS ratings is judged by how close its results come to what is expected.
I hate the term 'computer component'. It is not a 'computer component'. It is a statistical component, and most of those statistical methods have been around hundreds of years. You don't need a computer, and it's not a complicated algorithm. You could do it on a piece of paper with a pencil and some time.
As for whether they're judged by how close their results come? Not really. The algorithms are judged by how retrodictive and predictive they are.
For instance, does it have a better predictive accuracy than the BCS computer rankings, Sagarin, etc.
Way too much is made of predictivity of ranking systems. The game isn't *that* predictable, and for teams in the top 25, the strength distribution is tight enough that you probably couldn't predict the games to better than ~55% or so.
The big problem with the BCS is that they don't state what the goal of each ranking system is supposed to be. What makes a good statistical ranking system? You can mathematically state this, and they don't. What is the human ranking system supposed to do? It could do something useful - it could determine strength-of-victory better than a statistical analysis could, because there's no unbiasable game output function for football.
But right now there are just no clear guidelines for the human voters, and that just leads to a wash of crap.
Doug:
Forgot to expound on the shortcomings of this algorithm, in my opinion. One easy tweak would be to say that teams with equal numbers of losses or less count as 1 team for teams to vault. Teams with 1 fewer loss count as 2 teams. Teams with 2 fewer losses count as 5 teams, etc. That'd essentially allow teams to "vault" but hit a "soft wall."
Is a Week 2 list going to make it out? I'd love to see how this works throughout the year.
Andrew, I absolutely will be tracking this throughout the year. It'd be a bear to keep up to date by hand, so I'm going to have to write a program to do it. That will happen eventually, but I won't have time to get it done for probably a few weeks.
I'd guess by late September, I'll be providing regular updates.
This seems just way too biased to the BCS. Duke should be ahead of Boise State and Hawaii because they are in the ACC. I know it is very simple and that's how you are trying to keep it, but the best system has no pre-season rankings at all, or it treats all teams fairly based on rated performance, which doesn't mean putting an obvious divide between bad BCS schools and great non-BCS schools.
Kevin, you're missing the point. This is not supposed to be a good rating system. It is, in fact, supposed to be a bad rating system.
But the problem is, the computers in place now aren't bad. The way it is all put together is bad. That's where the insanity lies.