Oh, sure, there was a pretender to the title. That guy who pointed to the stands and delivered HIS home run. But, hey, it was just ONE prediction. Sure, a bunch of you can make one prediction. One projection. But I, Simply Fred, thought we should put the pundits to the test. All the fantasy baseball ‘perts — or at least the ones whose preseason predictions are still readily available. Why not measure the ACCURACY of their preseason PREDICTIONS to the actual, end-of-year RESULTS? FanGraphs provided us with the projections of the contenders: FANGRAPHS, BILL JAMES, MARCEL, ROTOCHAMPS, ZIPS. We threw in our own dark horse, Grey.
Methodology (reliability and accountability a requirement):
Refined our list of targeted players for which all had provided projections (an even playing field) — 177 hitters. (Pitchers not evaluated, because I’m doing this in my own free time. You want to know the pitchers; I say to you gently and encouragingly, you do it in your free time).
Needed an impartial, hopefully fair, and representative standard for comparison. The formula for a player’s value:
Rx1 + (HR x 3.5) + RBIx1 + (SB x 2) + AVG x 290 (Gets at a player’s relative worth.) — (You may have a different formula, but remember our task is to measure end results relative to beginning projections. And I can only go on my own formula, cause that’s the one I know.)
So, we establish a player’s preseason ‘value’ by applying the formula to a given set of projections. Then we apply the SAME formula to the end of year actual results.
Example: Ichiro
Fangraphs: (110+ (8 x 3.5) + 69 + (36 x 2) + (.329 x .290) = 374 which is your preseason projection. End of the year production for Ichiro came out to 303, so Fangraphs’ percent difference was 1.23 between the two. Here’s others:
Rotochamps: Preseason: 342; End of the Year: 303; Percent difference: 1.13
James: Preseason: 310; End of the Year: 303; Percent difference: 1.02
Marcel: Preseason: 290; End of the Year: 303; Percent difference: 0.96
Zips: Preseason: 303; End of the Year: 303; Percent difference: 1.00
Grey: Preseason: 299; End of the Year: 303; Percent difference: 0.99
Ideally, you want to come closest to 1.00. For this player, ZIPs was spot on! (Grey 2nd closest, and so on.)
We then measure the average of all percent differences for each prognosticator. The initial results for all 177 common players:
Marcel: 1.17
Zips: 1.23
Rotochamps: 1.34
James: 1.36
Grey: 1.36
Fangraphs: 1.41
Wow! Our Swami seems to be sinking in swampville. Though all of the projections were far too high. Hold it. What’s that we see lurking in the mud? Injuries!
Let’s limit our pool to those players that had 525 AB or more (to reduce the impact of unforeseen injuries). If you do that, you get:
Grey: 1.00
Rotochamps: 1.01
James: 1.02
Zips: 0.97
Fangraphs: 1.05
Marcel: 0.88
What does it mean that Grey had a ratio of 1.00? It means that his overall projections, for players with 525 AB or more, came out at the exact, overall end-of-year production. It does not mean perfection. Some projections a little low. Some a little high. However, overall, of all the ‘perts, he came closest to end-of-year production. Okay, now feel free to poke holes in this.
