This is part of a two-part series designed to help Fantasy Baseball fans determine on what fantasy rankings and projections to rely. The first part will cover Rankings. The second part will cover Projections. The methodology for the test relies on comparing Razzball Commenter League team drafts (576 teams in 2012 across 48 12-Team MLB leagues using ESPN’s default 5×5 format) and their end of season point totals. Background on the methodology can be found here.
Everyone who participates in fantasy baseball snake drafts has their own approach to pre-draft rankings and analysis. This approach usually results in a customized ranking of players that is a combination of trusted ranking sources, recent fantasy baseball news, and personal player preferences.
This is our annual attempt to help you determine which ranking sources are worth using.
The methodology behind testing the various ranking sources works as such:
- Convert every player for each source’s rankings to a $ figure using our Point Share dollar figures per draft pick – e.g., the #1 draft pick is usually ~$40, the next pick might be $38, etc. (If the source produces $ figures, use those.)
- If the service provides an option to adjust league settings, set it to the Razzball Commenter League format (12 team MLB, C, 1B, 2B, SS, 3B, 5 OF, CI, MI, UTIL, 9P).
- Use these player $ figures to create team $ values for all Razzball Commenter League teams (576 teams across 48 leagues in 2012) based on their drafted roster.
- See how these team $ figures correlate with that team’s final standings points
In 2011, I found that a team’s draft was responsible for about 64% of its success. In 2012, this number went down to 53.3% – meaning that in-season player moves had a greater impact on team standings. Of course, this was a year where the top hitter (Mike Trout) started the year in the minors and the top-rated pitcher (RA Dickey) went undrafted. If I credited Mike Trout’s stats to the 48 teams that received the most Trout AB during the year, this would change the % to 58.7%. (Note: Trout was drafted in 16 of 48 RCLs and 11 of those teams dropped him or traded him early so that he had more ABs on another team in the league. Ouch. Two more Trout stats: 1) The average RCL team with Trout had 84.3 points, the average team without Trout had 63.1 points. 2) According to RCL Commissioner VinWins, RCL teams with 100+ Trout ABs won 15 of the 48 leagues.)
So with 53% correlation as the proverbial ceiling for the ranking sources, let’s see how well the various ranking sources predicted success.
|2012 Fantasy Baseball Rankings – Ranked By How Well They Predict Razzball Commenter League Success*|
|Rankings Source||$ or Rankings||Calc’d or Curated||Avg Correlation Vs Other Sources||Published Before March 1st||Correlation of RCL Post-Draft Team Value with Total Team Standings Pts|
|End of Season Stats||$||Calc||—||—||53.3%||63.7%|
|Last Player Picked||$||Calc||92.7%||N||15.0%||(5.8%)|
|Razzball Point Shares||$||Calc||91.5%||N||14.7%||8.0%|
|Razzball – Grey||Rank||Curated||92.4%||Y||6.1%||7.7%|
|ESPN Custom Dollar Value Calc||$||Calc||93.0%||N||3.5%||—|
|ESPN-Tristan||Rank||Curated||94.8%||N (Y but updated in March)||1.8%||—|
* The following players were removed from the rankings as significant changes happened to their value post-publication of some of the rankings: Injuries (Carl Crawford, Chase Utley, Chris Carpenter, Ryan Madson, Joakim Soria), Increase in Value Because of Injuries (Sean Marshall, Greg Holland, Jonathon Broxton)
- The ranking sources did much better in 2012 vs. 2011 despite the major impact that Trout and Dickey had on team results.
- While Grey and I did not dominate the rankings like we did last year, we still fared well with both of us finishing in the top half. Also, Grey finishing #1 for sources who posted their rankings prior to March 1st (which is very helpful for early drafts). I’m planning on having Point Shares out in mid-Feb for 2013 with 1-2 updates in March to help all drafters.
- Aside from ESPN’s custom value calculator, the sources that calculated the rankings vs. curated the rankings performed much better. The only possible exception is KFFL which I suspect has some calculations behind it.
- KFFL was the most unique in its player values vs the other ranking sources with a 90% average correlation. They were also the most unique vs. FantasyPros’ aggregation (92.6%). My Point Shares were the 2nd most unique (91.5% avg correlation vs. other sources, 93.5% with FPro).
- Not surprisingly, FantasyPros’s aggregation of rankings were the least unique amongst the sources. Honestly, I see little value in aggregated rankings. How different is it from just using ADP? That said, the safest play performed well this year – at least relative to the other curated ranking sources.
- While KFFL’s rankings performed best, I caution using their $ values in 12-team auctions as they are too high. While the average team value was $255 across all the sources (very close to $260), KFFL’s average was $304.
- For those wondering why our friends at Yahoo! are not in this study, it is because Yahoo! only publishes a top 100 and the minimum needed for this study is Top 200.
- ESPN’s Top 300 finished last for the 2nd straight year. As with last year, I think it is possible that because our RCLs were hosted on ESPN, the ESPN Top 300 is very close to the default ESPN rankings which are used by bad/auto-drafted teams.