Fantasy baseball brings out the competitive bastard in me. Expert leagues, NFBC, and DFS sites like DraftKings are the standard targets of my affliction but I also get obsessive about our daily projections. Short of maybe gambling and winning big bucks in DFS (not going to happen with my limited risk tolerance), it is hard to find ways to test the quality of our daily projections.
It is not for lack of trying. I compare our DFS hitter and pitcher point projections against DFS site salaries in our DFSBot on our Ombotsman page and – going back to May 2014 – there has yet to be a month where a DFS site’s salary has outperformed us. I did a 2 month test last year against ESPN Fantasy Forecaster that showed our starting pitcher projections were superior. I tested last year’s projections against hitter and pitcher recent game data to find areas that could use improvement (highlighted pitcher IP should leverage recent starts more).
As I was contemplating new targets to compare against, I was IM’ing with our lead DFS writer (Sky) and he noted that he looks at Vegas Over/Under estimates for Runs when considering teams to stack. That got me thinking…I could go for some pancakes. After eating those pancakes, I thought, “What if I test our projections against Vegas!”. Then I napped.
Yada yada yada, below are the (favorable) test results versus Las Vegas MLB lines. See bottom of post for some testing notes. As an added bonus (until someone pays me to take them down), DFS Premium subscribers can see these team run projections updated every day (for today + at least the next 6 days) on our Teamonator page.
Please visit our Subscription page for more information on our daily projection subscription packages. We offer daily, monthly, and season long packages – catering both to the Roto and DFS player (if you play both, even better).
Also, stay tuned tomorrow for an awesome addition to DFSBot!!!
| Games Included: 169 Non-Double Header Games (338 GS) between May 26th and June 7th | ||||
| Razzball Opposing Team ERA Projection | Vegas Team Run Projection | Razzball Hitter Run Projection | Razzball Hitter RBI Projection | |
| Average | 3.860 | 3.818 | 3.966 | 3.920 |
| Correlation to Team Runs (r) | 0.231 | 0.168 | 0.182 | |
| AAE to Team Runs | 2.067 | 2.120 | 2.104 | |
Summary: Razzball ERA projections (for the opposing team) had both a higher correlation and a lower average absolute error to a team’s runs scored than Vegas’ team run projection. This difference is not statistically significant based on this sample size but it is a very good sign that our projections are at least statistically comparable to Vegas. As for the R/RBI projections, they are slightly worse but with the same statistical significance caveats.
| Games Included: 169 Non-Double Header Games (338 GS) between May 26th and June7th | ||||
| Razzball Both Team ERA Projections | Vegas O/U | Razzball Total Hitter Run Projection | Razzball Total Hitter RBI Projection | |
| Average | 7.719 | 7.637 | 7.932 | 7.840 |
| Correlation to Team Runs (r) | 0.240 | 0.238 | 0.200 | 0.222 |
| AAE to Team Runs | 2.937 | 2.949 | 2.960 | 2.944 |
Summary: Same general story as with Team Runs – our ERA projections slightly ahead of Vegas for the test but not statistically significant. Positive sign nonetheless!
| Picking Winners (% of Time The Team w/ More Projected Runs or Vegas Odds-on Favorite Won) | ||||
| Razzball ERA | Vegas Line | Razzball Total Hitter Run Projection | Razzball Total Hitter RBI Projection | |
| Winning % | 57.4% | 52.4% | 52.1% | 50.9% |
Summary: This is a surprising one. The average ‘favorite’ for Vegas was 55.7% but they only won 52.4% of the time meaning there was a slight run on underdogs during the test (netting out to a yawn-inducing 1.8% ROI if you invested in every underdog). The average ERA favorite was only about 53% – I am chalking the difference vs the 57.4% to some good luck. But, again, still a positive sign that our projections are comparable in quality to Vegas.
Test Notes:
- Since my projections are made before lineups are announced, I estimate a percent chance for each hitter to start (%Start). For Team Run/RBI totals, I summed Runs, RBIs, and ‘%Start’ estimates. I then multiplied each team’s Run and RBI totals by the Sum of % Start / 8.59. So teams where I overprojected playing time have their Runs/RBIs brought down and vice versa.
- The Hitter Run and RBI totals correlate (r) at 0.906 and 0.867 with the ERA projections. Reasons for differences include: 1) Caught a bug in my archiving of R/RBI projections that did not update if I re-ran during the day based on starting pitcher chance or hitter activation/DL, 2) ERA doesn’t account for unearned runs, 3) There are R instances without RBIs, and 4) While all three projections use the same park factors, they are calculated independently based on Steamer ROS ERA and Steamer R/RBI to batting event ratios. To get these three stats to optimally align (prior to posted lineups) would require a megaton of development at very minor gain. Still TBD how much re-running against posted lineups would improve accuracy.
- I used Covers.com for Vegas lines and O/U. To estimate Vegas team run projections, I did the following calculation:
Underdog Runs = 100/(Line+100)*O/U run projection. Favorite equals O/U minus Underdog runs.-3.15224+5.43206*Vegas projected win % + 0.550223*O/U runs. This calculation was based on a regression of various scores and their Pythagorean Winning Percentage.
