Should you believe in Wil Myers's projections?

Edit: Big-ol' hat tip to ChiBurbRaysFan here. He clicked to download my data to look for himself, and realized that I had reversed two columns, effectively invalidating the whole thing, and in fact reversing the conclusion. This is why peer review is great, but mistakes like this cannot happen. My apologies. Article is now fixed with the correct conclusion. -Ian

As you surely already know, super-prospect Wil Myers was just promoted to the major leagues. He should slot into the lineup immediately, and provide an instant offensive boost. The ZiPS projection system predicts for him a .253/.317/.444/.761 slash line, good for a somewhat above average .329 wOBA. That's similar to (slightly below) what Ben Zobrist has produced so far this year, and plenty useful, if not eye-popping. But should you believe those projections? Will Myers do worse? Will he do better?

There's a school of thought that holds that Wil Myers strikes out to much. This school sees his 24% strikeout rate in triple-A this season, and projects it to rise to an unworkable level when he faces major league pitching. After all, as a hitter moves up each level, the quality of pitcher he faces increases, and his ability to hit those pitchers decreases. These thinkers worry.

There's another school of thought which says that truly elite prospects do not suffer performance decreases as they move up levels. These hitters have the tools (bat speed, power, coordination) to hit against any quality of pitcher. Their numbers are merely a reflection of their own development, not their talent-relationship to their opposition. Adherents to this school expect a top prospect's stats to improve as he moves up through the minors, despite the increases of difficulty with each level.

Wil Myers, this year's fourth rated prospect by Baseball America, is surely that type of elite talent. And since he was just promoted to the major leagues, I thought it was time to try to put this developmental theory of top prospects to the test. Here's my hypothesis: Top prospects, when promoted to the major leagues, generally outperform their purely statistical projections.

To test my hypothesis, I've made a list of every hitter ranked in the Baseball America Top 10 who broke into the league during the seasons for which I could find ZiPS projections (2010-present). That list contains 11 players (I've excluded Domonic Brown since I couldn't decide which year to call his "break-in" year.*

*Some of these "break-in" years are slightly arbitrary. Some prospects come up at the end of the year, but don't exhaust their rookie eligibility. I chose the last year they were rookie-eligible.

That's an illustrious group, with a relatively high (.742 OPS) projection. Seven of the eleven hitters outperformed their ZiPS projection. One, Carlos Santana, met it almost exactly, but he was also the victim of the highest projection. Only three fell short of their projection (one of those being our own Desmond Jennings). On average, the group beat their projection by .075 points of OPS. That's the difference, offensively this year, between Evan Longoria and James Loney. That's a good sign. Maybe we should expect Myers to beat his projection as well?

There's a problem with this analysis so far, though. By hand-picking the season that top prospects break into the major leagues, I've created an obvious bias. It's very likely that prospects are promoted and stick if they succeed (outperform their projections), but quickly sent back down if they struggle. Or perhaps the ZiPS projection system underrates prospects, generally?

To guard against these biases, I've created a control group. I've gathered a list of every rookie hitter, not ranked as a Baseball America top 10 prospect, who received at least 300 PAs in 2012 or 2010.** This group consists of 36 players

With their slightly lower (.735 OPS) expectations, 26 outperformed their ZiPS projection. The other 10 fell short. The group, on average, exceeded expectations by .044 points of OPS. That's a 27% failure rate, which is virtually the exact same failure rate as the top prospects. This suggests that there's relatively little special about top-end prospects that's not captured in the projection system. Any over-performance seen in prospects by studies such as this are likely simply a reflection of the selection bias introduced by only considering those promotions who stick in the major leagues.

You can review the entire group, along with the rest of my data set, here.

**Why do I omit 2011? I wasn't able to find complete ZiPS projections for that year, and I didn't feel like googling each player individually. If someone can help me find them, I'll be happy to expand the control group.

Conclusion

The hypothesis is not supported. Top prospects who break into the major leagues perform little or no better relative to their projections than other prospects who break into the major leagues. Both groups perform better than their ZiPS projections, likely because of a selection bias.

That's not great news for Wil Myers, but it's not the end of the world, either. It means that we can't necessarily discount his high strikeout rate and good but not great on-base percentage. He is likely subject to the same struggles with an increase in the talent of his opposition that other prospects are. Still, his projection of a .761 OPS is plenty to look forward to, and you excitement should not be damped upon reading this. It just shouldn't be stoked, either.