Part 3 of How Valid is the ESPN Player Rater?
In two previous articles (part 1 and part 2), we’ve laid out alternative views for judging the most valuable player in 2007 5×5 MLB fantasy baseball (we say Peavy) and for pitchers – using and abusing the ESPN Player Rater in the process.
In this article, we’re going to focus on valuing hitters. The questions we will tackle are:
1) What is the value of each hitting stat?
2) How does position depth/scarcity affect a player’s overall value?
3) How does our approach to hitter value compare against the ESPN Player Rater?
To download our player rankings for 2007, please click our 2007 Fantasy Baseball Player Rater. To view the ESPN Player Rater.
What is the value of each hitting stat?
Our approach towards valuing player stats is to look at two factors: 1) the difference between a player’s statistics and those provided by the best available option (BAO) on the free agent wire (which would take position depth/scarcity into account) and 2) the impact that stat difference might have on a league’s standings (ambivalent to position).
We’ll set position scarcity aside for a second to look at the composite stats for the BAO hitter in 2007: 67 Runs / 14 HR / 65 RBI / 6 SBs / .277. The closest player equivalent to these stats is Luis Gonzalez.
We used the final standings of our fantasy league to understand the impact of each statistic by looking at the standard deviations between teams’ totals. While it would be better if we had more league standings on which to base these standard deviations, we still feel this is superior to building ratios off team averages because it takes into account that some statistics have larger percentage gaps between teams vs. others. This is most evident when looking at HR vs. SB – while the average team in our league average 1.69 HRs to 1 SB and the BAO has a HR:SB ratio of 2.33:1, the observed impact on a team was actually 1:0.77 or that a HR has more value (not even counting the R/HR/RBI/AVG implications) to a team’s rank in the standings than an SB.
The ratio for these stats based on our analysis was: 3.3 Runs / 1 HR / 2.8 RBI / 1.3 SBs / .003 AVG. Points are credited based on these ratios (a point actually equals the above ratio * 4.6) after subtracting the BAO’s stats.
Okay, let’s do two comparisons to show this works in action.
Eric Byrnes (103 / 21 / 83 / 50 / .286) vs. Miguel Cabrera ( 91 / 34 / 119 / 2 / .320)
This is an interesting one as it asks that inevitable question – how much are SBs really worth? Is it worth the addition 48 SBs to sacrifice those HRs, RBIs, and AVG that Miguel Cabrera provides? Let’s look at the points comparison of R/HR/RBI/AVG:
Runs: Byrnes 2.4 to 1.5
HRs: Cabrera 3.95 to 1.55
RBIs: Cabrera 3.95 to 1.43
AVG: Cabrera 3.48 to 1.01
(Note: While it might not look right that Cabrera’s 34 HRs could be worth 2.5x that of Eric Byrnes 21 HRs, remember that the BAO provides 14 HRs. So this is really a comparison of 20 HRs (34-14) vs. 7 HRs.)
Counting just these stats, Cabrera is about twice as valuable as Eric Byrnes (12.89 to 6.39).
But Eric Byrnes’ 50 SBs is huge given the average team only had 162 SBs in our league. A total like this could let you dominate SBs or focus on non-speed guys when filling out other positions (say, taking Khalil Greene’s 27 HRs instead of J. Lugo’s 33 SBs).
Eric Byrnes’ 50 SBs equates to 7.25 points in our scale while Cabrera’s 2 SBs equate to negative 0.45 points because it’s less than the BAO would’ve provided (which is 6). So factoring in SBs, Eric Byrnes is the more valuable fantasy hitter (13.6 to 12.4). But if your team was set for SBs, trading Eric Byrnes for Miguel Cabrera would be a no-brainer.
Placido Polanco (105 / 9 / 67 / 7 / .341) vs. Dan Uggla (113 / 31 / 88 / 2 / .245)
This comparison focuses on Polanco’s AVG contribution vs. Uggla’s power contribution.
Runs: Uggla 2.65 to 2.12
HRs: Uggla 4.0 to -0.78
RBIs: Uggla 1.94 to 0.30
SBs: Polanco -0.05 to -0.86
(Note: These comparisons do factor in position scarcity – hence, Uggla’s 2 SBs receive more negative credit that M. Cabrera’s above since the 2B BAO steals more than the average player.)
Counting these stats, Uggla is well ahead at 7.73 to 1.59 points, with the biggest driver being his 31 HRs which are worth 4.78 points more than Polanco’s 9 HRs.
But those HRs come at a price. Uggla’s .245 average is well below the BAO average of .277 (actually 2B’s have higher AVG than other positions so it’s even worse – examples of high batting average marginal 2Bs include Orlando Hudson’s .294, Luis Castillo’s .301, and Ronnie Belliard’s .290). Combining that bad average with his above average AB total (632), Uggla’s average would drop the average team’s AVG by .004 vs. the BAO 2B. This earns him a negative 3.28.
On the other hand, Polanco’s .341 in 587 ABs is worth a positive 4.77 points – more, in fact, than Uggla’s 31 HRs. He’s worth about an extra .004 on your team average meaning that swapping these two creates a .008 swing, a more dramatic swing than the 22 HR difference.
So while Placido Polanco is a negative on a team’s HR and SBs (and just about BAO level on RBIs), his high AVG catapults him into being a more valuable fantasy baseball contributor (6.4 to 4.4). If Uggla could just get to something like a .275 average or steal 20-30 SBs, his HR/RBIs could help catapult him up the 2B rankings (even with the anchor-like AVG, he ended up 7th most valuable 2B, well ahead of the .317 hitting ROY Dustin Pedroia).
How does position depth/scarcity affect a player’s overall value?
Position depth/scarcity plays a role from draft day through the end of the season.
During draft day, position depth/scarcity can increase/decrease a player’s value. A common practice is to ‘tier’ players at each position and try to group together similarly valued players. If there is only one player left in, say, the 2B tier and 5 similar valued players at SS, you may increase that 2B’s draft value because you can wait a round and likely get one of those shortstops.
After the draft, position depth/scarcity is used to compare the marginal benefit/loss of trading or adding/dropping one player over the next – e.g., I could trade Placido Polanco and replace him with little to no dropoff in any stat except AVG.
To factor this into our analysis, we extended our Best Available Option (BAO) concept to each position. We started with 10 rostered players for catchers and infield positions and 50 outfielders. We split the 1B/3B and 2B/SS positions equally and then divided up the utility position based on instinct and position depth (30% 1B, 2.5% 2B, 2.5% SS, 5% 3B, 0% C, 40% OF, 20% DH). We created composite stats for BAOs at each position – so for catcher, we took the 11th best AVG, 11th best HRs, etc. We then credited point totals based on the BAO at the position (“Position Points”) and averaged them with our average hitter BAO (“Player Points”). (Note: Since team rankings are position-agnostic – you don’t get more credit if it’s a middle infielder who hits a HR – there is a need to balance position depth/scarcity with overall stats. To keep it simple, we weighted it 50/50).
Below are the BAO stats per position (R / HR / RBI / SB / AVG) and some close statistical fits:
C – 47 / 13 / 57 / 2 / 0.273 (Paul Lo Duca, Johnny Estrada, AJ Pierzynski)
1B – 63 / 18 / 68 / 1 / 0.279 (Matt Stairs, Conor Jackson, Aubrey Huff)
2B – 79 / 11 / 61 / 9 / 0.288 (Orlando Hudson, Brendan Harris, Mark DeRosa)
SS – 72 / 11/ 60 / 11 / 0.279 (Brendan Harris, Jack Wilson)
3B – 70 / 18 / 72 / 4 / 0.279 (Kevin Kouzmanoff, Mark Reynolds, Aubrey Huff)
OF – 67 / 14 / 65 / 6 / 0.273 (Luis Gonzalez, Austin Kearns, JD Drew)
The most interesting about these BAO totals is how relatively close they are. The corner positions have a slight advantage in power and the middle infield spots have a slight advantage for runs, SBs and average. Catchers are weakest – particularly in Runs as catchers play less games and are disproportionately hitting 6th to 9th (less run opportunities).
Perhaps most surprisingly, the OF position looks no better than the middle infield positions. Wouldn’t you expect OF was a deeper position than middle infield? Isn’t BJ Upton more valuable as a 2B than an OF? Short answer: not really.
Here’s why: You’ve got roughly 15 2B, 15 SS, and somewhere between 52-57 outfielders on league rosters (OF are often used for UTIL positions). Looking at MLB rosters, you have roughly 30 starting 2B, 30 starting SS, and 90 starting OFs. FLB rosters, thus, are cutting deeper into the percentage of starting OFs vs. 2B/SS.
In addition, 2B/SS have added some pop over the years. 29 middle infielders hit at least 12 HRs. Granted, some had bad averages (Bill Hall, Juan Uribe, Stephen Drew), but the perception of those positions being power-challenged is outdated. (What IS true, though, is that it’s rare to find a middle infielder with 30+ HR power).
Outfielders, on the other hand, aren’t that deep. Only about 55 hit 15 or more home runs and that includes some players that might be at other positions (Berkman, Upton, Stairs) and the weakest ones look an awful lot like Luis Gonzalez and Austin Kearns (the BAO matches).
So while we did factor position depth/scarcity into our analysis, it really didn’t play a major role for hitters. The greatest impact was at catcher where the troika of great catchers in 2007 (Jorge Posada, V-Mart, Russell Martin) got about a 2 point boost because the Catcher position was the weakest in terms of BAO.
So Hanley Ramirez and Jimmy Rollins had extremely valuable fantasy years but the fact they played SS really didn’t add any significant value (maybe +2-3%).
How does our approach to hitter value compare against the ESPN Player Rater?
ESPN has a much simpler approach for estimating hitter value than the approach we have described above. It creates a cap at 5 points and a floor at 0 points. 5 points are awarded to the MLB leader in the stat and then each other player’s total is divided into the leader’s total and then multiplied by 5 to get their total – e.g., A-Rod led with 54 homers. David Wright had 30. He received 30/54 (.556) * 5 = 2.78 points in HR. Average is done in a slightly more complex way but the lowest possible total is zero (even if the player’s average has negative value).
From a macro-perspective, this simplistic approach works fine. The top hitters are going to appear near the top, the okay hitters in the middle, the bad hitters on the bottom. At a micro-perspective, we think ESPN’s simplistic approach has greater flaws vs. our approach. These flaws are less for hitters than pitchers, though, as the greater issues arise around ratio/average based stats and pitchers have two (ERA, WHIP) vs. one for hitters (AVG).
In a previous article, we identified four issues with ESPN Player Rater for valuing pitchers
1) Capping High Points at 5
2) Positive Ratio/Average Contributions Are Undercredited
3) Negative Contributions Aren’t Penalized
4) Overcrediting of Slightly Above Average Performance
These four issues all play a role for valuing hitters but #2 and #3 are not as major an issue because ERA/WHIP are more polarizing than AVG. For example, even low value hitters may hit .290 but only a great starting pitcher can manage an ERA near 3.00 ERA.
An additional issue we’ve found is:
5) The league leader used as the points base distorts the distribution of points – While the leader in Runs and RBIs is relatively close to the other leaders (no one had, say, 200 Runs or RBIs), A-Rod’s 54 HRs and Reye’s otherworldly 78 SBs set a very high bar for 5 points. This creates odd situations where Eric Byrnes 50 SBs (tied for 4th in majors) is worth less in ESPN Player Points than his 103 Runs (outside the top 20) and Jimmy Rollins’ 30 HRs (tied for 20th) are worth less than his 94 RBIs (tied for 42nd).
Here is the assessment on a stat by stat basis:
Runs – Overcredits for all players. For above average performance, Issue #4 plays a role (the Best Available Option’s 67 Runs warrants 2.3 points). For below average performance, Issue #3 starts taking effect (less than 67 runs should warrant negative points). An additional issue throughout is that runs are so plentiful across players that the value of a run is less than other stats (A-Rod’s 143 runs warrant 4.95 points in our estimation vs. 8.29 for his 54 HRs)
Home Runs – Undercredits great performance like A-Rod and Fielder (Issue #1). Issue #5 plays a role in underestimating the value of everyone at 25+ Homers. Players between about 15-24 HRs are slightly inflated based on Issue #4. Anyone below the BAO average of 14 are overestimated based on Issue #3.
RBIs – Undercredits the great performances like A-Rod and Matt Holliday (Issue #1). Overcredits above average performance (Issue #4). Undercredits below average performance (Issue #3). Issue #4 affects more hitters than Issue #3 (which is limited to speedsters and some 2B/SS – examples are Reyes’s 57 RBIs and Pierre’s 41RBIs)
SBs – This is the category where Issues #1 and #5 play a huge role in underestimating SB value. We have Jose Reyes’ 78 SBs at a whopping 11.5 points – the most points awarded for any offensive category. Teammate David Wright’s 34 SBs earned him a respectable 4.8 points (equivalent to Holliday’s 36 HRs and Vlad’s 125 RBIs). This underestimation affects hitter values all the way down to about 10 SBs. Issue #3 plays a very minor role – greatest for 2B/SS as speed is most common in that category (Freddy Sanchez’s 0 SBs earned him a negative 1.19).
AVG – Issue #1 only affects the top 3 hitters as Magglio, Ichiro, and Matt Holliday’s averages warranted 6+ points in our ratings. Issue #2 plays a role for the rest of those with averages above .330. Issue #4 plays a role in overestimating the value of hitters lower than .330 but greater than BAO (e.g., Luis Gonzalez’s pedestrian .278 warrants 1.99 ESPN points where it should be worth zero). For low average hitters, Issue #3 plays a role in greatly overestimating their value as they should have negative value. Uggla’s aforementioned average of .245 gets .86 ESPN points compared to our -3.28 points.
Amazingly, though, the cumulative effect of these issues seems to have little bearing on the ranking of hitters. We agree with the top 10 OFs from ESPN Player Rater with slightly different ordering. The top 10 2B match down to the order. The differences play more of a role in total player rankings – below are some examples of players differently valued (Our Ranking, ESPN Ranking).
Eric Byrnes (25, 43)
Jorge Posada (58, 92)
Juan Pierre (78, 117)
Derek Jeter (86, 106)
It’s worth noting that almost every hitter is higher valued in our rankings vs. ESPN because ESPN overvalues pitchers out of the top 20 and this pushes down all the hitters.
So while we find faults in ESPN’s methodology, we can’t fault using ESPN Player Rater to understand hitter position rankings. It works surprisingly well for hitters given its simplistic approach – it’s possible that its flaws are a bigger issue as you move down the player rankings. That said, we would caution against using the combined hitter and pitcher rankings given the flaws we’ve seen with their valuing and ranking of pitchers.
