Evaluating a Pitch, DIPS Style: What's a Good Pitch?

For nearly two years now Fangraphs has listed pitch type values for each pitcher and each of their pitches. Pitch type values are novel and useful for determining how effective a pitch has been. For reference, the best pitches in the game are usually slightly better than +2 runs saved per 100 pitches. Mariano Rivera's cutter is at +2.14 runs for his career and Tim Lincecum's changeup is at +2.17 for his career. There are, however, some issues with using pitch type values to evaluate a pitch's quality:

1. They're context-dependent. Just as we don't use WPA or RE24 to evaluate a batter's talent, it's misleading to look at pitch type values to determine how good a pitch is. When trying to evaluate a pitcher's talent, a pitch that hits a player in a two out, bases empty situation is no less of a bad pitch than one that hits a player with the bases loaded.
2. They aren't intrinsically meaningful. Does +.5 runs per 100 pitches mean much to you? How good is a pitch that's +.25 per 100? While this isn't the biggest issue with pitch type values, and while the system gains familiarity after a while, it certainly isn't intuitive.
3. They are subject to defensive mistakes. If a pitch induces a pop-up that a fielder misreads and lets fall for a double, it still gets graded as a bad pitch, even though the pitch itself did a decent job in inducing the pop-up.
4. They ignore DIPS theory. This point is similar to the last but is worth repeating. The change in run expectancy done by a ground ball is immensely dependent on the infielders. Additionally, random variation is magnified when dealing with run expectancies for a couple hundred pitches.

This article looks to eliminate these four "flaws" of pitch evaluation by coming up with an objective scale for measuring a pitch's quality.

For this analysis we turn to Baseball Prospectus's SIERA. SIERA is a defense-independent pitching statistic that projects ERA about as well as xFIP - one of the best context-neutral pitching statistics available - but is more advantageous for our analysis because it factors in multiple batted ball types and is per PA, not IP. The inputs for SIERA are plate appearances (PA), strikeouts (K), walks (BB), groundballs (GB), flyballs (FB), and infield flyballs (IFFB).

Note: The following two paragraphs use statistics quite extensively. For those who never bothered with stats, here's a summary of what it says: We can do a good job of describing a pitch's expected strikeout rate by looking at the amount of swinging strikes it generates, but it's harder to identify a pitch's expected walk rate. Using a combination of various plate discipline statistics (ball%, zone%, swing% and contact%), an acceptable relationship is established between a pitch's "stuff" and how many walks it creates. Feel free to skip the next two paragraphs if you don't want the nitty-gritty details.

Coming up with values for a pitch's expected K% and BB% is no easy task. Given the extremely strong correlation between K% and SwStr% (R=.89), we can more or less get a good idea of a pitch's "effective" K%. The relationship between strikeout rate and swinging strike rate is a strong and clean one. A multivariable regression yields K%=2.08*SwingingStrike rate.

Evaluating BB% is far more difficult. Several multi-variable regressions gave outrageously specific formulae in which many variables had painfully high P-values. The best multi-variable regressions all involved F-Strike% (first pitch strike percentage), but for obvious reasons we can't use that here. Zone%, Contact%, Swing%, and Ball% are the only things that ended up having p-values below .5 when a 9-variable regression is done with all of the various stats found at Fangraphs (i.e., O-swing, Z-swing, etc.). Doing another regression with these four provides an acceptable end result: BB%=-.525+.569*zone+.15*swing-.267*contact+1.27*ball. The overall significance level is 6.15*10^-11, each of these four have ultra-low p-values (excepting swing%, which comes in at .24), and overall R=.80.

The other inputs for SIERA are easy enough to pull from Joe Leftkowitz's data. Groundball rate (GB%), flyball rate (FB%), and infield flyball rate (IFFB%) are all directly measurable quantities with little-to-no guesswork involved. Now let's put this system to work to evaluate the Rays' ace, David Price.

First, consider Price's swing and contact data for each of his pitches. Using these values, we can create an expected strikeout and walk rate for each pitch.

Here we see more or less what we'd expect, save for a few surprises. Price's curveball and slider go for balls much more often and yet the slider manages to keep its BB rating down because of its relatively low contact and relatively high swing ratings. Now, plugging the xK% and xBB% values into the SIERA formula, we reach our goal.

Price's four-seamer is absolutely lethal. The biggest revelation here is that the slider is still excellent because of an outrageously good pop-up rate (perhaps unsustainably high) and an excellent whiff rate. Additionally, his two-seamer is wildly effective because of its solid whiff rate and excellent ground ball rate. His curveball is slightly less effective because it gets fewer whiffs and doesn't stay in the zone as often. If this is to be believed, Price probably should use his slider and his two-seamer more.

You may have noticed that Price's pitches have abnormally low expected walk rates (and SIERAs, as a result of the inputs). This is because his peripheral scouting data suggests that his walk rate will decrease, which may or may not happen. As shown earlier, batter peripheral statistics are a better indicator of future walk rate than past walk rate; we'll take a look in a later post on whether this holds true for pitchers.