Since we started doing Power Rankings in February 2009, we’ve found there’s one interesting constant during our various derby travels — at least one member or fan from every league will buttonhole us at an after party looking to have a sincere discussion about why DNN’s rankings are so totally biased against their league. Our stock answer for the last three years has been “Uh â€¦ dude â€¦ seriously?” However, we have a lot of data in spreadsheets going back to when we first started doing this â€¦ and now that it’s been 3 full seasons, we have enough data to break down the rankings history by team and and objectively pick out some trends. Maybe we really were secretly biased all this time and did not know! Perhaps Science could tell us!
We figured that the most effective way of determining who DNN truly hated / disrespected / was biased against the most was not just by determining overall prediction accuracy, but also by measuring how often we under ranked a team (that is, predicted them to lose a game that they won) against how often we over ranked them (predicted them to win a game that they lost).
After a LOT of data crunching to break down the numbers for the teams currently in DNN’s Top 25, here’s what we came up with:
Most Accurately Ranked
Steel City, with 41 hits on 43 games (95% accuracy). Both DNN misses here were in 2011 wins against Montreal. We were also almost always right about Gotham (33 of 35 games picked correctly, 94% accuracy) and Windy City (40 out of 43 games picked correctly, 93% accuracy).
Least Accurately Ranked
Arch Rival, with 9 misses in 29 games for 68% accuracy. (This one is in large part due to the Chicago Outfit — a third of those misses came during Arch Rival’s back and forth rivalry with the Outfit this year, with each team winning 2 of their 4 games.) We got 27 out of Rose City’s 38 games right for 71% accuracy, and did almost exactly the same on Rat City (30 of 42 games, 71% accuracy). Notably, we over ranked / under ranked these teams almost exactly evenly: 6/6 for Rat City, 6/5 for Rose City, and 5/4 for Arch Rival.
Most Often Over Ranked
Here’s where it gets interesting for us, as we were particularly interested in finding out which teams we were consistently making incorrect assumptions about. It turned out to be Cincinnati who was most likely to lose a game DNN had predicted them to win, over ranked 7 times and under ranked just once (against Atlanta in 2010). Texas was in a similar position, over ranked 7 times and under ranked twice. Philly was third in this category, over ranked 6 times and under ranked twice.
Most Often Under Ranked
This one is probably the closest one to “What team does DNN consistently give the least credit to?” This hotly contested category ends up being a tie between Minnesota, over ranked just once in a 2009 loss to North Star and under ranked 5 times since, and Oly, also over ranked once and under ranked 5 times. However, all but one of Oly’s under-ranks came during their explosive debut season in 2009, whereas Minnesota’s were spread out almost evenly over the past three seasons.
Bay Area (over ranked 3 times, under ranked 6 times) and Naptown (never over ranked, under ranked 3 times) also have good cases for being historically underestimated — again, though, Naptown’s under-ranks all came in 2011, while Bay Area has a more consistent record of posting DNN upsets.
Here’s a full breakdown of current top 25 teams sorted by DNN’s accuracy.
Some more general trivia we picked up in the course of this project:
Most Likely to Upset an Opponent
Rat City and Bay Area are tied for having dealt the most Power Rankings upsets, with 6 each.
Biggest One-Month Jumps
Oly went from unranked to #10 in April 2009. London went from unranked to #15 in May 2010. Minnesota went from unranked to #14 in October 2010.
Among teams that were already ranked at the time of their big jump, Montreal went 12 slots from #24 to #12 in March 2011, and Rocky Mountain went 10 spaces from #17 to #7 in October 2009.
Biggest One-Month Slides
Rat City dropped 7 spaces from #6 to #13 in March 2011 following an upset from then- #24 Montreal; Bay Area also dropped 7 spaces from #5 to #12 in December 2010 after losing to Texas at Championships.
Biggest One-Season Jumps
Oly went from unranked to #1 in 2009. Bay Area went from #22 to #5 over the course of 2010 (but finished at #12).
Biggest One-Season Slides
Carolina went down 15 slots from #6 to #21 over the course of 2009. Boston went down 12 slots from #13 to #25 over the course of 2011 (although they finished the year at #20).
Number One Runs
We’ve had four teams ranked #1. Gotham has spent a total of 15 months on top, Oly has been at #1 for 11 total months, Rocky Mountain comes in with 8 months, and Philly has had one month on top. Gotham and Rocky are tied for most consecutive months at #1 — each of them has an 8 month run.
In three years, the only time we’ve had Gotham projected to lose a game was when they met Rocky Mountain at the 2010 Championships. (Though Gotham hasn’t always been #1, that was the only time they weren’t ranked above their opponent at the time of the game.)
Most Active Against Top 25
Charm City and Philly are tied for most games against other teams in the Power Rankings since we started them in February 2009 (52 games each).
DNN Accuracy By Month
Somewhat surprisingly, we tend to be most accurate during the months of May and August. Our May accuracy percentages for the last three years were 92%, 93% and 93%; our August percentages were 84%, 90% and 96%. October is generally a bloodbath in the rankings (57%, 68% and 81% over the last three years) but the Championships picture in November has been fairly well projected in the last two years (66% in 2009 during the sudden rise of the West region, but 92% in 2010 and 88% in 2011).