Evaluating Derek Shelton

Yesterday, the Rays announced that they plan to retain their entire coaching staff, despite the departure of Andrew Friedman and Joe Maddon. That ostensibly includes much-maligned hitting coach Derek Shelton. It's understandable why people are down on him.

There's been a perception for a long time that the Rays don't hit well, as you know, and last season that perception was finally correct; however, it's difficult to evaluate a hitting coach, and when you call for someone to lose his job, you'd better be right.

Most of us (including myself) don't know what Shelton does during his workdays. His roll could be as small as communicating nuances noticed on tape to as large as coordinating the hitting program for the entire organization, but we can still try to evaluate whether or not he succeeds at the major league level. There are a number of traps that evaluation can fall into, though. Let's step through them.

Evaluation one: The Rays don't score a lot of runs. Since Shelton took over before the 2010 season, they've only scored the tenth most runs in baseball.

Why it fails: Well, for one, tenth most sounds pretty good. But setting that aside for now, runs scored isn't a very good measure of offensive ability. The parks teams play in can either increase or suppress it, and while there's valid debate as to the extent, Tropicana Field is a pitcher's park, meaning that fewer runs are scored in it than the quality of the players to come through Tampa Bay would indicate.

Evaluation two: The Rays don't hit the ball well, even when adjusted for park. Since Shelton took over before the 2010 season, they've only hit for a wRC+ (linear weights, adjusted for park) of 103.

Why it fails: Well, let's put aside the fact that a wRC+ of 103 is not only above average, but the fifth best in baseball over that time. If we pretend that the facts supported this argument (or if you'd like, if we only focus on the 99 wRC+ from 2014), we can still discard this premise. Evaluating a hitting coach on the raw output of his hitters isn't a very good idea, since not all hitters are created equal. If I were to be appointed hitting coach of the Detroit Tigers tomorrow, Miguel Cabrera would continue to hit, despite my worst efforts. And I doubt that the very best hitting coach in the world could do much with Jose Molina. Some credit or blame goes to the coach, but we have to do a better job figuring out how much.

Evaluation three: Rays hitters consistently under-perform their projections.

Why it fails: Now we're getting somewhere. I don't know if the above statement is true. Several people around this site have performed that analysis in the past, and it's something that would be worth updating (I'm working on a larger evaluation of hitting coaches league-wide that may include this, but with the recent announcement I wanted to show a portion of it before the whole analysis is done), but this sort of method doesn't do enough to separate the coach from the player. Consider what the projection really is. It's a combination of past results with some regression and an aging curve. And the hitting coach shares some of the credit or blame for the past results, meaning that he's partially responsible for the projection itself, not just whether or not it's met.

For instance, when Sean Rodriguez underperformed his projection in 2012 with a paltry .269 wOBA, that projection was in some part a result of his work with Shelton in 2010 and 2011. Maybe, had Shelton not arrived in 2010, Rodriguez would have collapsed earlier, and only some great work by the hitting coach prevented him from doing so.

The following season though, S-Rod set a career best with a .313 wOBA in 222 plate appearances, outstripping what the projection systems thought he was capable of. Was that a great triumph of the hitting coach, or did his disastrous 2012 (for which Shelton shares some of the blame) merely lower expectations?

Another Way - The Aging Curve

If we can't compare a player to their projection and use that as a way of evaluating a long-serving hitting coach, what can we do? Well, there are a few options, but most of them are difficult. The easiest to accomplish from a data-manipulation standpoint that I could come up with is to compare hitters who played consecutive seasons under Shelton to the league-wide aging curve.

A little while ago, Jeff Zimmerman wrote an excellent article on FanGraphs noting a startling new development. There is no longer any upward motion in the aging curve. On average, players arrive in the majors at their peaks, and only decline from there. Zimmerman goes into detail about this, and there are some interesting reasons why it's happening. You should read his article, but I've grabbed a graph from it below, detailing the observed performance curves by age from 1995-2005 (red), and 2006-2013 (blue).

Now an aging curve is something we can work with. It gets past the question of whether or not a player is good or bad, because it deals only in changes of production for the same player. It also bypasses the question of how to separate a hitting coach's prior performance from a player's projections by not using projections at all. For this exercise, we're only looking at changes in production from year to ear.

In a single team, there isn't enough of a sample size to use the same method Zimmerman used, so I tried to approximate. I selected each player who received at least 200 (or nearly so) plate appearances for the Rays in consecutive years during the Derek Shelton Era. From those players, I noted the change in their wOBA between seasons, and organized them based on age. Below are the players who qualified. The average considers each player as an individual success or failure, rather than weighting by plate appearances.

Next, I took all of these data points, and ran them through a LOESS regression to smooth out the curve into something readable. Think of this as a composite of all Rays players during the Derek Shelton Era's personal aging curve.

There are a few things to note. First, the order of magnitude on the Y axis of the two different graphs are not the same, so the entire graph of the Rays would fit on the top line of the league-wide chart. Secondly, the LOESS smoothing hides the fact that there's a lot being filled in without much data. The young section of the graphs is mostly Wil Myers and Reid Brignac, the 30+ section is strongly influenced by Ben Zobrist, and the 35+ section is heavily influenced by Jose Molina. There's more data in the 24-28 range than anywhere else, and while hypothetically we should be seeing a fair amount of decline in that section, we don't. Instead, we see that under Derek Shelton, the Rays have peaked somewhere around 27 and held steady through age 33.

There are a ton of questions this analysis doesn't answer. By looking at one team only, I may be introducing biases I haven't thought of. Restricting the sample to players who remain with the same team may warp things. I do plan on replicating this for every long-tenured hitting coach, which will help to answer that question. This may be too little data to draw any conclusions, and I certainly wouldn't say that it tells us much about Derek Shelton's true talent as a hitting coach. What it does tell us is that criticizing Shelton for how hitters develop at the major league level under him is unfair and incorrect.

If you still think Derek Shelton should be fired, prove it. There's a fanpost section on the right side of this page, and any points you make and can back up with facts will be front paged immediately. I plan to keep working on this question as well. Right now though, I have to admit that he may be a coach worth retaining.