Don't be shellfish...Share on FacebookTweet about this on TwitterShare on Google+

One of the biggest challenges facing the fantasy baseball fanatic is how to value and rank players.  This is felt most acutely during draft season when nearly every fantasy sports site/expert has their own Top 200/300 rankings and each manager has to decide which source(s) to believe.  This challenge is also felt – albeit to a lesser extent – during the season when managers are looking for a ‘player rater’ to determine trade values.

When we started Razzball a couple years back, I decided to leverage whatever limited math and Excel skills to come up with the best source of player ranking/valuation.  This eventually led me to create Point Shares which estimate the difference in an average team’s points if they were to substitute a given player for the average player at his position.  This differs from the Standings Gain Point (SGP) concept which is more akin to VORP/WAR-methodology and uses something close to the replacement player for valuing players.  I prefer using the ‘average player’ as a benchmark vs. a ‘replacement player’ but I am not going to delve into methodology in this post (You can reference this post if this topic tickles your curiosity).

There are three major components when it comes to developing preseason rankings:

  1. Playing Time Estimates (PA/AB and IP)
  2. Statistical Projections (HRs per AB, K/9, etc.)
  3. Methodology For Converting #1 and #2 Into Player Rankings/Value

I assume the majority of ‘expert’ rankings (Grey’s included) do not split out each of these three components.  Instead, they leverage their past fantasy baseball experience plus playing time/statistical expectations (likely based on third-party projections and, increasingly, component stats like BABIP and FIP) and create what I would call a ‘curated’ rankings list and/or auction $ values.  The initial reaction of mathy types might dismiss this for being non-scientific but, in my eyes, a well-curated player rankings beats a poorly-architected quantitative system every day of the week.

Testing curated rankings, though, is a challenge.  Unlike with projection systems where each statistic can be broken out and measured (here is a recent test done on FanGraphs), one would need to design a test that used only the order in which the players were ranked (note: $ estimates are easier).  A test design based on simulating draft results creates both logistical and methodological challenges .  For the past three years, I have taken part in a ‘Forecaster’s Challenge‘ run by Tom Tango (co-author of  The Book, creator of the Marcel projection system, and prolific blogger on InsideTheBook.com) which simulated drafts based on submitted rankings and credited victories based on total points (like in a ‘points’ league, each statistic is worth a certain amount of points).  I think Tom did a great job at creating an impartial test but the conceits with any test of this type are difficult to overcome (e.g., simulated snake draft vs. actual, reducing 2B/SS/3B to one position, 20 teams vs. the standard 12 teams, inability to factor how well a team could overcome certain draft disappointments vs. others (say, Dunn vs. Hanley) through FA replacements, etc.).

Note:  I believe I finished around the middle of the pack in each challenge.  My dissatisfaction with some of the test conceits is unrelated to my performance.

When my brain awoke this February/March from my annual winter hibernation from baseball, I hit upon the construct for a test that I believe can remove many of the conceits of past tests.  It could be used to test all of the following:

  1. Player Ranking/Value Methodologies – aka ‘Player Raters’ (component #3)
  2. Pre-Season Rankings (baking in components #1, #2, #3)
  3. Statistical Projections (component #2 – specifically, how well do projection systems project stats relevant for fantasy baseball)
  4. Playing Time Estimates (component #1)

In the process, it could also determine how much the final standings are impacted by one’s draft selections as well as the reliability (or lack thereof) of pre-season standings (as in using projections to determine who looks best in the pre-season).

Here is the test:

  1. Take the draft results by team from the 38 Razzball Commenter Leagues in 2011 (hosted on ESPN, 12 team, MLB, 5×5, C/1B/2B/SS/3B/CI/MI/UTIL/9P/3 Bench/1 DL, 180 Games Started, Daily Roster Changes, Unlimited FA/Waiver pickups).  This amounts to 456 teams’ worth of draft data.
  2. Create a team total based on ‘expert’ rankings/$ totals/other arbitrary metric (like ESPN Player Rater Total Points)
  3. See how these team totals correlate with each team’s final Total Standings Points

Notes:

  1. For testing individual components, the other two components must be kept constant (e.g., to test Playing Time Estimates, use the same Statistical Projections and Player Ranking/Value Methodology).
  2. Rankings need to be converted into $ because the difference in value between picks progressively gets smaller as the draft progresses.

The key benefits of such a test vs. a simulated test are:

  • These are ACTUAL draft results based on real fantasy baseball manager behavior.
  • The team standings points reflect ACTUAL in-season fantasy baseball manager behavior such as replacing poor-performing draft picks, using replacement players when players are injured, etc.

I’m going to focus my first test on Player Ranking/Value Methodologies (aka Player Raters) because it is the easiest one to do.  Why?  Because there is a sure-fire, uncontroversial source for Playing Time Estimates and Statistical Projections to act as the ‘constant’ – 2011 Final Season Statistics.

I tested the following free public sources as part of the test:

  1. Razzball Point Shares – 12 Team ESPN
  2. ESPN Player Rater (It’s one player rater so no way to customize for league format.  Note:  This link will likely be overwritten with 2012 data once the season starts.  I have archived the results.)
  3. Last Player Picked – 12 Team League, $260, ‘Optimal Hitter/Pitcher Mix’, using same roster format as listed above with 6 SP/3 RP (most representative split of pitchers based on league behavior)

In addition, I tested a total points formula that’s primarily based on the one that Tom Tango created for the Forecaster Challenge:  HR+SB+(R+RBI)/3 + (H-(0.27*AB)) + 2*W + 1.5*SV + K/5 + (IP-(H+BB+ER)/2).  The one difference was to multiply Saves by 1.5 vs. 1 to better reflect RP value.

Since these leagues are ostensibly populated by Razzball readers, I first wanted to test to see if there might be any bias in draft behavior.  Below are the correlation percentages between the Average Draft Positions (ADP) for players in the Razzball Commenter Leagues (RCL) vs. Grey’s pre-season 2011 rankings, the pre-season 2011 Point Share rankings, and the ESPN Top 300 for 12-Team leagues.   I included all players drafted in at least 30 of the 38 leagues.  I broke out the ADP for the top 100 teams vs. all 456 teams to see if there might be a ‘Razzball’ bias amongst only the top teams.

Correlation (%)
RCL ADP-Top 100 Teams RCL ADP- All Teams Grey’s Rankings Point Shares (3/8) Point Shares (Late March) ESPN Top 300 (12-Team)
RCL ADP – Top 100 Teams 99.5 91.8 79.5 82.7 96.7
RCL ADP – All Teams 99.5 92.6 80.4 81.5 96.7
Grey’s Rankings 91.8 92.6 72.5 72.9 85.5
Point Shares (3/8) 79.5 80.4 72.5 96.9 78.3
Point Shares (Late March) 82.7 81.5 72.9 96.9 79.9
ESPN Top 300 (12-Team) 96.7 96.7 85.5 78.3 79.9

I assume the extremely high correlation with ESPN’s Top 300 for 2011 (96.7%)  is driven by the default ADP used in the draft software.  Interestingly, Grey’s rankings and my Point Share rankings differ from ESPN’s (78-85% correlation) but differ more from each other (~73% correlation).  Given these correlations, I think it’s fair to assume that the Razzball Commenter League (RCL) draft results are fairly indicative of standard ESPN drafts.

Here are the correlation % results for the Player Ranking/Value Methodologies (links to each were provided above, here is an aggregated view).  I tested both my actual Point Shares as well as my conversion to dollars.  ESPN Player Rater is based on their Total Points in their Player Rater.  Last Player Picked is based on their $ estimates.

Other notes:

  • Players not found in a player rater (usually based on injuries/missed playing time) are set at $0 for Point Shares/LPP and zero for ESPN Player Rater.
  • Any player with < $0 in Point Shares/LPP is capped at $0 as players that bad (or missed that much time) were likely excised from a team roster before they could do a full season’s worth of damage (and, remember, that ‘replacement value’ is at $0).  For instance, Brian Matusz was drafted in every league.  His $ estimate in Point Shares was -$31 in Point Shares, -$25 in LPP, and -7.08 in ESPN Player Rater points.  All are now set to zero.  For Point Shares, I capped it at -2.64 which is the equivalent of $0.
 Source Correlation With Team Standing Points
Point Shares 63.8%
Point Shares (converted to $) 63.7%
ESPN Player Rater 56.7%
Last Player Picked 55.2%
Points Formula 49.7%

Based on the above results, I would answer the question of “How much are the final standings impacted by one’s draft selections?” as probably somewhere in the 60-65% range.  I can’t say for sure since it’s unclear what the actual ceiling for player rater accuracy.  Please note that this is a wholly different question than “How much are the final standings impacted by one’s draft selections as valued by preseason rankings/projections?”  That will be answered in my next test.

With assistance from Jared Cross (co-creator of Steamer Projections), I tested to see whether these differences vs. Point Shares are statistically relevant at a 95% confidence interval (this is the typical standard confidence interval used in research).  Here were the findings (worksheet found here):

Source Confidence of Difference vs. Point Shares
Point Shares (converted to $) 53.98% (z-score of -0.1)
ESPN Player Rater 99.90% (z-score of -3.1)
Last Player Picked 99.9+% (z-score of -4.2)
Points Formula 99.9+% (z-score of -5.0)

The minute difference between Point Shares and my $ values isn’t that surprising since my dollar conversion formula is just a calculation from the Point Shares.  If it resulted in significantly different results, it would be a sign that my calculation was flawed.

I cannot say for sure why Point Shares beats ESPN and Last Player Picked as I do not know all the details behind their methodology.  They correlate at 93.5% which isn’t markedly higher than their correlation vs. Point Shares (89.9% for ESPN, 92.4% for LPP).  Last Player Picked is the more transparent of the two in terms of methodology and it’s clear that Mays @ LPP uses ‘replacement level’ as the foundation of his analysis (vs. me using ‘average player level’).  No idea if that really plays a role here.  If I had to guess what drives ESPN’s Player Rater, I’d venture some application of Z-Scores per category.

I also really don’t care to spend too much energy researching why my Point Shares methodology appears to be superior.  One variable I can say for sure is that my position factors (e.g. a catcher w/ same stats as an OF is worth more) have no measurable impact.  I ran Point Shares with no positional adjustments and got a 63.73% correlation instead of 63.78% (z-score 0f -0.1).

If you compare each of the three rankings/$ estimates, you could potentially deduce some of the methodology differences.  For instance, it probably comes as no shock to anyone familiar with the ESPN Player Rater that – when comparing it to Point Shares – some of the largest differences come into play with players whose primary value comes from stolen bases.  Here are some examples (Point Share Rank / ESPN Rank):  Michael Bourn (47/14), Elvis Andrus (77/51), Coco Crisp (109/60), and Brett Gardner (118/66).

If Bourn was really the 14th most valuable player, you’d think that teams who drafted him received great value (ESPN had him ranked 89th in the pre-season) and performed disproportionately better vs. teams that did not draft him in the Razzball Commenter Leagues.  As you can see in this spreadsheet, teams who drafted Bourn finished almost exactly in the middle of the pack.  While this test isn’t perfectly conclusive of player value (e.g., Granderson only ranked #89), the results seem to correlate fairly well with expectations.  The following are in the top 10%:  Kemp, Ellsbury, Weaver, Verlander, and Bautista.  The following are in the bottom 10%: Hanley Ramirez, Carl Crawford, Jayson Werth, Joe Nathan, Kendrys Morales, Chase Utley.  I wonder if this potential issue with the ESPN Player Rater is driving Matthew Berry’s 2012 love for Michael Bourn (note:  even at Point Shares #47 rank, you could make an argument if you felt confident that Bourn could repeat his 2011 stats that he warrants a 3rd round pick.  I’d consider him a 5th round pick at best.)

It also should be noted that both Last Player Picked and ESPN Player Rater have significant usability advantages vs. my Point Shares.  Last Player Picked can customize $ estimates based on just about any league permutation imaginable.  While ESPN Player Rater doesn’t allow for league customization, it is updated throughout the season which is a huge advantage vs. Point Shares/LPP.

I tried to be as transparent and unbiased as possible with this analysis.  The one piece of information that I didn’t link to is the actual draft selections per team.  I will provide that once I’ve completed my next analysis.  Please feel free to comment with questions and/or to point out ways I may have screwed up the analysis.

My next test will be testing 2011 Player Rankings against team results.  I will only use free, publicly available rankings unless authorized by someone at the company behind the subscription-based rankings.  All player rankings must have a date stamp prior to the beginning of the 2011 season.   If you see a notable omission below, please provide me with a link to the rankings.  Thanks to FantasyPros.com who helped me gather some of the below rankings:

  • Razzball – Point Shares (1 version on March 8th, one done around end of March)
  • Razzball – Grey’s Rankings
  • ESPN – Matthew Berry’s Top 200
  • ESPN – Pre-Season Top 300
  • FantasyPros.com Aggregated Top 300
  • FoxSports Top 300
  • Hardball Times (Jeffrey Gross) – not public but permission-provided
  • KFFL Top 200
  • Last Player Picked – using 2011 Composite Stats
  • RotoChamp Top 300
  • RotoExperts Top 300
  • SI.com Top 300
  • USAToday.com Top 200

Note:  CBSSports.com uses a static link for its free pre-season guide so the link now points to 2012 rankings (if someone has a saved download of the 2011 PDF, please e-mail it to me at rudy@razzball.com).  Our pals at Yahoo! (perhaps wisely) do not publish pre-season rankings. 

  1. Jake says:
    (link)

    14 team, 5×5 weekly roto. We get 4 keepers, and mine weren’t particularly good ones. No legitimate top 20 players.

    C-J Montero
    1B-Konerko
    2B-Altuve
    3B-Lawrie
    SS-Andrus
    CI-Betemit
    MI-Cozart
    OF-McCutchen
    OF-Pence
    OF-Heyward
    OF-Rios
    UT-Heisey
    UT-Thames
    BN-Utley
    BN-Doumit

    SP-Verlander
    SP-Darvish
    SP-Beachy
    SP-Kuroda
    RP-Kimbrel
    RP-Street
    RP-League
    BN-T Hudson
    BN-Milone
    BN-Thornton
    BN-Crain
    BN-Adams

    Got Thornton/Crain for super cheap and I figured the extra depth/trade bait certainly couldn’t hurt if 1 of em ran with the job.

    What do you think of the team? Thanks.

    • Grey

      Grey says:
      (link)

      Because you went for such a strong pitching staff, it hurt your hitting… You’re low on power, and it might be worse if Konerko falls off a cliff… Don’t love Darvish either, but he could work out for you…

      • Jake says:
        (link)

        @Grey,

        Thanks, I have a similar outlook on the team. Most of the “elite” bats were already kept, so the best I could do was adding Konerko. HR/RBI will probably be a season long issue, though I’m expecting to do very well on the pitching side. Maybe I can trade a non-Kimbrel closer for a bat, when someone gets desperate.

  2. Tony says:
    (link)

    wow, i hate math

  3. Jake says:
    (link)

    Also, I really enjoy your in-depth number crunching, Rudy. Gives a bit of a different perspective from the usual (also great) content on here. Much appreciated.

    • @Jake, You’re welcome. Appreciate your appreciation.

  4. Sweeney says:
    (link)

    Rudy, that is exceptional work, thanks.

    • @Sweeney, you’re welcome appreciate the kind words!

  5. BigA says:
    (link)

    A heapin’ helpin’ of statistical analysis. Good job, very interesting read.

    One last piece of the puzzle: Are you going to have an opportunity to post AL-only pointshares? I’m using the NL only ones for my NL-only league and they are FANTASTIC.

    Would very much appreciate the AL only ones being posted (before my Friday auction, he says selfishly).

    BigA

    • @BigA, My AL-only draft is tomorrow so planning on posting it before Friday.

      • BigA says:
        (link)

        @Rudy Gamble,
        Ahem…patiently waiting for those AL only pointshares. Hope your league went well. Need them for my Friday auction…

  6. chata says:
    (link)

    gotcha .

  7. beardcrabs says:
    (link)

    RCL team/ What do you think Grey?

    C- G. Soto
    1B- J. Votto
    2B- H. Kendrick
    3B- P. Sandoval
    SS- A. Escobar
    MI- J. Altuve
    CI- M. Reynolds
    OF- A. McCutchen
    OF- B.J. Upton
    OF- A. Jones
    OF- B. Boesch
    OF- P. Bourjos
    Ut- A. Presley
    B- S. Rodriguez
    P- Z. Grienke
    P- M. Garza
    P- B. Beachy
    P- J. Garcia
    P- C. Sale
    P- V. Worley
    P- J. Haranhan
    P- S. Santos
    B- M. Adams
    B- L. Gregorson

    • beardcrabs says:
      (link)

      @Rudy as well…

  8. beardcrabs says:
    (link)

    Also, Tom Millone or Josh Collmentor as a 6th SP in an NFBC…

  9. uncdrew says:
    (link)

    Wow, looks like a ton of work and smarts went into that article.

    But, uh, I’m stoopid and am not sure what the main points were. Was it saying Grey is bad at pre-season rankings? ESPN is good? Us lemmings tend to follow ESPN’s rankings?

    Basically, what do I need to do to win? :)

    • @uncdrew, This test was just focused on testing ESPN vs. Last Player Picked vs. my Point Shares for valuing players based on the same stats (in this case, end of year stats). Next test will focus on pre-season rankings but the early conclusion is use the Point Shares + Grey’s rankings.

      • RandomItalicizedVoice says:
        (link)

        @Rudy Gamble, For all my drafts so far this year, I have 2 computers running. On my main machine, I have the draft window open, along with Rotoworld minimized to check between pick for any last second breaking news. On the laptop, I have 2 windows open….the relevant Point Shares listing and Grey’s rankings. So I guess I’m doing it right.

        Thanks for all the work, as usual Rudy. I have a love/hate relationship with math myself, but I do really appreciate it’s power and purity. Thanks again!

        • VinWins

          VinWins says:
          (link)

          @RandomItalicizedVoice, That’s not fair. Next year I think we should institute a 1-computer rule.

      • uncdrew says:
        (link)

        @Rudy Gamble,

        Thank you for the reply. It’s good for you guys (but bad for me) in that everyone in my league reads Razzball. So I don’t have much of an advantage.

        Hell, I almost want to use ESPN’s rankings just so I’m being contrarian. I’m happy being contrarian (and actually enjoy it), but being wrong is no fun.

  10. mindicohn says:
    (link)

    Had my 10 tm 5×5 roto auction last night…Went heavy on offense:
    C – Doumit $1
    1b – Fielder $45
    2b – Cano $21 (keeper)
    3b – Moustakas $1
    SS – Castro $10 (keeper)
    OF – Gordon $7 (keeper), Kendrick $10, Rios $1
    Util – Pedroia $41 and Hanley $40
    P – Price $15 (keeper), Lester $39, Scherzer $11, Liriano $1, Sale $1, Lilly $1, Kimbrel $8 (keeper ) Rivera $8
    bench – gamel $1,hardy $1, Alexei Ramirez $2,Sean Marshall $1 and Capps $1
    Should I drop LIlly or Hardy for Chacin, Niese, Stauffer or Vance Worley – to name a few?

    Thx

  11. Long Bawls says:
    (link)

    Just got offered Carlos Santana for my Brett Lawrie (I have Ramos at C). Considering his Inspector Clouseau series of misfortunes, I feel like Brett might actually be more liable to lose time to the DL, but still, seems kinda meh to me.

    • Former Skeptic says:
      (link)

      @Long Bawls, Do it!

      • Long Bawls says:
        (link)

        @Former Skeptic, Interesting! Thanks!

        • @Long Bawls, I’d do it in a heartbeat too.

    • Sweeney says:
      (link)

      @Long Bawls, I drafted Santana reluctantly this year, because he dropped well past where I expected him to be taken. I’m generally in the “punt catcher” camp, in a one-catcher league. If I were you, I’d keep Lawrie for now, assuming a very decent chance that you can “sell high” on him somewhere in the first half of the season if he ever catches a hot streak.

  12. AL KOHOLIC says:
    (link)

    great job Rudy

  13. ritzer says:
    (link)

    Just drafted this 14 team h2h first year keeper team.
    Batting; runs, doubles, triples, HR,RBI, steals, OBP ( no average)
    Pitching W Era, K, WHIP, Saves, QS, Holds

    C Miguel Montero
    1B Prince Fielder
    2B Brandon Phillips
    3B Evan Longoria
    SS Jhonny Peralta
    LF Mike Carp or JD Martinez or Marcus Thames
    CF B.J. Upton
    RF Jay Bruce
    U Drew Stubbs

    SP Zack Greinke , Daniel Hudson, Gio Gonzalez , Shaun Marcum , Jhoulys Chacin ,Mike Minor , Luke Hochevar
    RP Jason Motte, Jonathan Broxton, Brett Myers, Matt Capps, Mark Melancon, Joel Peralta

    Bench Mike Aviles and whoever I don’t start in LF.
    A few questions.
    Who do I go for in LF?
    How bad is my RP?
    Any other general observations?

  14. RK says:
    (link)

    I’ve been playing FBL for 15 years. I’m terrible.

    So I went back to the drawing board, and became more analytical. Solely examining the draft, I figured out that I would’ve finished third, yet ended the year in 6th; obviously it isn’t the draft that is my problem.

    I started doing something like the above, too, because I wanted to see what the actual value of the 2012 projections were. Using ESPN’s player rater and 2011 data, I fit the data into a normal distribution (3 factors was enough). Figuring out the averages points (ERA, WHIP, Avg) was trickier, though. Also, it helps to know if I skip one player who is being overvalued in one round, there’s a comparable value coming up that I can steal.

    This way, I can compare apples to apples, or HR vs. SB, or pitchers vs. hitters. Position scarcity is one factor that I haven’t factored in, yet, but I was using Last Player Picked as one proxy. The last OF picked has a PR of 4.28 (and there are plenty of similarly-valued players around him) while the 10th catcher is 2.80.

    Also, you can take ADP vs. projected PR and see who is being undervalued (Michael Brantley is ranked #173 in PR but is lucky to get drafted).

  15. Ben says:
    (link)

    Hey Grey could you give me some feedback on my team? Thanks a bunch.

    12 team h2h points mixed league

    1. (2) Albert Pujols (LAA – 1B)
    2. (23) Adrian Beltre (Tex – 3B)
    3. (26) Andrew McCutchen (Pit – OF)
    4. (47) Elvis Andrus (Tex – SS)
    5. (50) Jay Bruce (Cin – OF)
    6. (71) Howie Kendrick (LAA – 1B,2B,OF)
    7. (74) Jon Lester (Bos – SP)
    8. (95) Gio Gonzalez (Was – SP)
    9. (98) Mat Latos (Cin – SP)
    10. (119) Alex Avila (Det – C)
    11. (122) Jordan Zimmermann (Was – SP)
    12. (143) Corey Hart (Mil – OF)
    13. (146) Anibal Sanchez (Mia – SP)
    14. (167) Jason Motte (StL – RP)
    15. (170) Huston Street (SD – RP)
    16. (191) Kenley Jansen (LAD – RP)
    17. (194) Trevor Cahill (Ari – SP)
    18. (215) Colby Rasmus (Tor – OF)
    19. (218) Scott Baker (Min – SP)
    20. (239) Brett Myers (Hou – SP)
    21. (242) Austin Jackson (Det – OF)
    22. (263) Juan Nicasio (Col – SP)
    23. (266) Bryce Harper (Was – OF)

  16. Sweeney says:
    (link)

    Rudy,

    One thing I’m working on is how to incorporate both “value to my team” and “market value” concepts into my auction strategy. I’m thinking about punting RBI and HR in my 4×4 NL only league. So a player’s value to my team will be WAY different than his value to the rest of the league. Roughly speaking, I’m assuming a player’s value to the rest of the league is alot like his 4×4 NL-only pointshares value. His value to me is like a 2×4 (SB,AVG,W,K,ERA,SV) NL-only pointshare. This however sets up some tough questions:

    For example, Michael Bourn might be worth $40 to me and $20 to everyone else (on pointshares), but in the auction he gets bid up to $30. This is still $10 of “value” to my team in the categories I’m not punting, but also $10 more than I was expecting to have to pay for him when I made up my list of targetted players for my strategy. Should I be willing to pay $30 for him?

    I think there is a quantitative way to approach this question, but I’m not quite sure what it is. Any thoughts?

    Any other advice on how to best execute this double-punt strategy would also be well appreciated. Don’t try to talk me out of it, decision’s already been made. Long story short, it’s in a deep dynasty league and cirumstances leave me no choice.

    • @Sweeney, I don’t agree with punting 2 categories in 4×4 but, if I had to, I’d avoid going too far above market price on any hitter to hoard your $ to dominate pitching.

  17. BWC says:
    (link)

    So the analysis run in this test is a) the end of the year performance (measured by total roto score) vs b) the total of that team’s original drafted players estimated value using the different methodologies (Point Share, LPP, Player Rater) running the actual 2011 data through as the inputs to those methods? Just want to confirm that I have it correct. (I might have follow-up questions, but wanted to make sure I understood the setup).

    • @BWC, Yes, you’ve got it right.

      • BWC says:
        (link)

        @Rudy Gamble, And you’d say the other 35% of final points success is coming from in-league management (picking up good hitters/closers, streaming pitchers, trades, and luck?) I can’t think of any biases that would benefit point shares in correleating with those factors, so that’s good. So far, I like the way this analysis is going, I think the next step will be interesting (I’m assuming you’ll analyze the success of the different methods and then the various projection systems being run through those methods to rank both method and projection systems when using pre-season projections). I’m looking forward to it.

        • @BWC, I’d have to think through the remaining ~35%. I don’t think luck is a main component – I think the next analysis on rankings will show that a lot of luck falls under this 60-65%. Trades have some value but that also taps into the 65% – e.g., easier to trade if you drafted a lot of good assets. But if you can change junk to jewels, then totally. I think FA pickups play a big component but not sure how to estimate it.

          I’ll be testing the pre-season rankings next…after that, i’d probably look to test projections through Point Shares (with the playing time estimations I had as of late March 2011). Will have to ask around though about getting those projections..

          • BWC says:
            (link)

            @Rudy Gamble, I’m sure someone at insidethebook will help provide projections. This method is good and better than the challenges that Tango devised (in large part because you have access to so many leagues for data)

            • @BWC, Yes, I know a few people who might be helpful in this regard. Only going to focus on those that released their projections publicly, though. Figure the most relevant testing for Razzball readers is projections that are readily available.

              This is definitely a different type of test than I’ve seen done before. Conceptually, it could be better. We’ll see. Biggest challenge is how statistically confident the differences are b/w the sources i compare. This first test worked out well (in terms of a clear ‘winner’). We’ll see going forward….

  18. James Redacted

    James says:
    (link)

    Rudy – I’m not big on Veto’s but this one’s got me riled up a bit.

    10 team Keeper league:
    Lawrie (2nd round) for Mat Latos (8th round) + Derek Holland (18th round).

    It happened soon after the draft this year and it’s before the season (obv.) so it’s essentially an 8th + 18th for a 2nd round pick. Let it go or make some noise? Thanks!

    • hawk says:
      (link)

      @James,

      I normally base vetos on the owners (do they know what they’re doing, are they newbies etc…). If the two owners know what they’re doing and they want it, I let it go.

      This one seems close enough that I’d let it go but I’d ask the owner why he didn’t come to me first as I’d probably have beaten that offer!

    • @James, i’m fine w/ the trade too. Lawrie in unkeepable at that price. Point Shares has Latos > Lawrie so there’s definitely justification for this trade even if the majority opinion is that Lawrie urinates liquid gold and poops platinum.

  19. hmmmmm says:
    (link)

    Hey Rudy-

    Nice work. What’s the correlation between Grey’s rankings and Team standing points?

    Thanks.

    • @hmmmmm, will be part of the next analysis. this first test just focused on any system that converts player stats into one composite value (like $)

  20. Steve Stevenson says:
    (link)

    Sitting on the #8 pick in a roto league where we start 3OF, 1 util and no corners. I usually go 1B in spots like this, but half of my top 15 OF are being kept. Given that, do I grab Upton at 8, or someone like Votto/A. Gonzalez?

    Also, H2H league, 1 roster spot to play with for the Japan series and limited moves. Would you use it on a bat (Seth Smith, Smoak, Carp are best available) or Vargas?

  21. prezii says:
    (link)

    Hey buddy. Found razzball a year back havent chose anything over it since. Keep up tje good work! Just lookin for a rating on my baseball squadso, here it is

    C- mauer
    1b- fielder
    2b- j. Weeks
    SS- rollins
    3b- lawrie
    Of1- nelly cruz
    Of2-

  22. prezii says:
    (link)

    Hey buddy. Found razzball a year back havent chose anything over it since. Keep up tje good work! Just lookin for a rating on my baseball squadso, here it is… 10 team h2h league. Mixed catergories. I had last pick.

    C- mauer(8)
    1b- fielder(1)
    2b- j. Weeks(12)
    SS- rollins (6)
    3b- lawrie(5)
    Of1- nelly cruz(4)
    Of2- werth (13)
    Of3- j. Heyward(9)
    Of4- al soriano(16)
    Utl- trumbo(15)
    Utl- brennen boesh
    BN- lorenzo cain
    ,
    Sp- kershaw(2)
    Sp- sabathia(3)
    Sp- wainwright(7)
    Rp- street(18)
    Rp- betancourt
    P- pineda(11)
    P- romero(10)
    P-morrow(14)
    P-hellickson(17)
    P- kuroda
    P- jurrgens

  23. prezii says:
    (link)

    Sry doin this crap on my phone. Jurrgeuns(19) , kuroda(20), boesh(21), cain(22), betancourt(23)

  24. Sabakafest says:
    (link)

    Rasmus or LaHair?

    Tebow or Sanchez?

    • hawk says:
      (link)

      @Sabakafest,

      Rasmus, Tebow. Go!

  25. chata says:
    (link)

    @ Grey :

    the starbucks commercial is near the bottom of the page ,
    on the right side , underneath ‘create your own zwinky’ ,
    and the domino’s pizza ad .

    thank you .

  26. Blue says:
    (link)

    My contention is that you’re going to find valuation issues in all systems that are based on distributions from some mean (e.g., Z-scores). A far better way to think about Roto scoring is as a zero-sum game where there will be X number of whatever category scored in the league–your goal is not to get above that amount on a distributional basis but rather to collect shares of the events. In other words, I don’t care if a catcher is 2 sd above average catcher performance if that distribution only amounts to, say 4 home runs, 10 RBI, etc. What matters, fundamentally, is what share of those stats generated across the entire league that the player is able to contribute.

    • @Blue, I agree that SDs per position don’t mean much. I suppose my methodology is closer to what you are proposing although I wouldn’t phrase it quite the way you did.

      • BWC says:
        (link)

        @Blue. I don’t think anyone doing Z-scores is using position specific SDs. They’d likely use SDs for the entire likely player pool (should be historical SDs, might just use Projected SDs). They’d also use the entire likely player pools average when calculating SD’s but that gets cancelled out when you calculate the replacement player (after taking account MI, CI, and Util) so really you end up with SDs above the replacement level player in the position.

        • @BWC, Yes, makes sense. That’s my assumption for ESPN’s player rater methodology.

  27. VinWins

    VinWins says:
    (link)

    Interesting stuff, Rudy. Does this mean we’ll all be chasing you in the RCL?

    • @VinWins, Doubtful. When you see the next post in this series, you’ll understand…

  28. Joe Blow says:
    (link)

    @Rudy, love the analysis! Nice to see some number crunching on here as well, mixes well with the humor!

    On a side note worth trading Michael Young for Kipnis and Brandon McCarthy in a league where I have Chris Carpenter, Scott Baker and Tim Hudson down with injury at SP already?

    • @Joe Blow, Thanks. Not a great deal. In 12-Team Point Shares, Young worth $12.40 while Kipnis + McCarthy = $10.60.

  29. does anyone know where the post is that
    grey made about what the final numbers you need to win each category, searched the site and could not find

  30. Awesomus Maximus

    Awesomus Maximus says:
    (link)

    Holy Christ… that Matthew Berry article you linked to is like a turkey dinner washed down with a box of wine followed by a blowjob while watching golf and popping an Ambien. I barely made it through two paragraphs before I had to scroll down about five pages to see why he love Michael Bourn… and the reason? Because his company’s Player Rater tells him to. Wow, brilliant stuff.

    • @Awesomus Maximus, Was expecting people to just do a CTRL-F (or Command-F for Mac peeps) for Bourn :)

      • Awesomus Maximus

        Awesomus Maximus says:
        (link)

        @Rudy Gamble, I’d never read him before, so I decided to give him a shot. The non-fantasy stuff was boring and self-indulgent (“my Facebook page,” “my Lakers tickets,” “dinner with my wife”). Can you imagine writing a page worth of content that has nothing to do with FBB and is just all about your personal life?

        The FBB stuff was uninspired and naive. “ESPN’s Player Rater tells me Michael Bourn is better than Pujols and Miggy and etc.” Well, how come you have those guys as 1st round picks and Bourn as a 2nd round pick? How can you be so blissfully unaware that a steals-only guy is easier to replace than a 5-cat guy?

        “Carl Crawford was excellent for a long time. So he got old and collapsed and injured his wrist. He’s still Carl Crawford.” Well, A-Rod was the best for a long time. So he got injured and got old and started dating Ellen DeGeneres. He’s still A-Rod.

        Actually, I didn’t read enough… he may very well have A-Rod amongst the players he loves. Wouldn’t surprise me. I’m tempted to join an ESPN public league and draft based only on his advice just to see how awful it does. I will name the team “Dingle Berry’s Loves” and hope that it at least provides enough fodder for an end-of-season article.

        • @Awesomus Maximus, I don’t understand – so you’re saying you’re not a berry fan?

          • Awesomus Maximus

            Awesomus Maximus says:
            (link)

            @Rudy Gamble, I’m a fan… I’m willing to base an entire draft on his brilliant analysis!

            The Bourn infatuation is so nuts it’s hilarious. As someone on your podcast (I think) said, he’s pretty much Dee Gordon without the SS eligibility. Suppose he’s a second rounder, too.

            • If Gordon was a C, he’d be first pick overall

            • JoeC says:
              (link)

              @Awesomus Maximus, Which is why I named my RCL league (#8) “Matthew Berry is a Tool”. :)

              My special appreciation for Berry’s toolness came from that interview he did a few years ago with Razzball. Made himself look like a complete and utter idiot. That shoe definitely fits.

              • Awesomus Maximus

                Awesomus Maximus says:
                (link)

                @JoeC, Well, I guess he’s known ’round these parts, but this was my first run in with him. Can’t believe he actually gets paid for that tripe. I’ve read Facebook status updates that were less self-absorbed and way more interesting.

                  • Awesomus Maximus

                    Awesomus Maximus says:
                    (link)

                    @JoeC, At least the comments were an interesting read. If he was this lame on his previous site, it’s sad that ESPN felt the need to buy him. And if he used to actually be funny, it’s sad what an enormous corporate sellout he’s become.

                    One of the later comments about it being a “Frank from Blue Velvet” style interview was hilarious.

  31. polo says:
    (link)

    Hey Rudy – thanks so much for point shares. Really helped through this 1st auction. Can you comment on draft and answer a question?

    10 team 6×6

    r, rbi, sb, tb, obp, slg
    c mesoraco
    1 hosmer
    2 phillips
    3 longoria
    ss d gordon
    of m kemp
    of bj upton
    of c. young
    util m reynolds
    util bourjos
    util gardner
    bn c. rasmus
    bn j montero

    sv, hld, era, whip, k/9, qs
    sp cliff lee
    sp matt latos
    sp b beachy
    rp storen
    rp h street
    rp motte
    p a chapman
    p s romo
    p j broxton
    p jim johnson
    bn m sherzer
    bn ani sanchez
    bn matt moore

    okay, or 2 lite in the OF/Util?

    okay to drop k/9 killer jim johnson and his saves
    for jose veras and his holds?

  32. byrone says:
    (link)

    rudy! that’s a lot of math, holmes.

    so i heard on the podcast the other day that ur from austin- me too, hollerrrrrrrrrrrrrrrrrrrrrr. just wanted to let you know my upcoming shows in april, in case you wanna meet up for a beer and rock out or whatever:

    4/11: real estate @ mohawk (might stay for keep shelly in athens)
    4/12: terry malts @ mohawk
    4/14: frankie rose, DIVE @ red 7
    4/17: chairlift @ mohawk
    4/18: hunx and his punx, natural child @ mohawk
    4/27-29: psych fest @ emo’s east/beauty ballroom

    get back to me if ur gonna be at any of those, i’ll give u my cell#. later dude!

    • @byrone, it might be a while before i go to any shows after the SXSW deluge haven’t heard of any of those bands though i’ve been to all those venues before. thanks for the offer though!

      • byrone says:
        (link)

        @Rudy Gamble, no prob dude.

        lulz, u haven’t heard of any of those bands, huh? based on what you mentioned seeing at sx on the podcast, yeah, ur taste is kinda mainstream… i bet you’d like real estate, they’re pretty accessible. check em out sometime!

  33. The Talented Mr. Dope Man says:
    (link)

    @ Rudy,

    For your next test can you input Steamer projections as well?

    • @The Talented Mr. Dope Man, When I test projections, I’ll be testing Steamer. The next test involves rankings/$ values and Steamer didn’t do that in 2011.

  34. Sorry all who have posted their rosters – swamped with work and can’t reply. Please repost in Grey’s latest post.

    • ivebeenupthreedaysstraight says:
      (link)

      @Rudy Gamble — the dependable part about this site is the useful and hilarious baseball writing. however, the best part about this site is the occasional, full-frontal flash of genius like the one you’ve put together here… it’s like the guy at the neighbourhood pool whose speedo barely covers his enormous member randomly decides to expose himself every once in a while. you always knew what was down there, but it sure is glorious in-the-flesh. [that's a pretty foul metaphor, but it's what came to mind... and sorry for not moving on, i'm a few days overdue and decided it would be weird to post this comment after the closer report... thanks, though, either way.]

      • @ivebeenupthreedaysstraight, thanks, i think.

  35. The Talented Mr. Dope Man says:
    (link)

    Rudy – got drafts this weekend. interested in whose rankings you would suggest for hitters? The fangraphs articles seem to suggest Oliver. I was hoping for some free projections. lmk

    • @The Talented Mr. Dope Man, I haven’t tested all the systems. All I can say is I’m using 50/50 ZiPs and Steamer for 2012 Point Shares.

      • The Talented Mr. Dope Man says:
        (link)

        @Rudy Gamble, thanks , i decided to go with steamers. i dont like the fact that zips doesn’t take playing time into consideration.

Comments are closed.