Lately there has been a bit of talk about the performance of our pitching staff and our bullpen. Naturally this leads to Jim Hickey. Is he a good pitching coach? Bad pitching coach? Are pitching coaches irrelevant? I'm going to attempt to shine some light to this dilemma. There just has to be some way to quantify the performance of pitching coaches....right?
1. Comparing the performance of a pitcher under Hickey to the same pitcher's performance under a different pitching coach. In order to do this I found a sample of pitchers (30 IP min) that had 1 year with Hickey and then the next MLB year w/o Hickey (or vice versa).
2. Comparing how a pitcher performs while under the eye of Hickey. Does pitchers improve or get worse over a year under the eye of Hickey?
If pitching coaches are irrelevant the metrics should remain static. Things such as declines due to age or improvements due to maturation should be filtered away due to sample size.
For Part 1 I am going to look at the first method comparing a pitchers performance with and without Hickey. My initial assumption (which is why I wanted to do this to begin with), is that performance suffers under Hickey.
Lets get right to the results
I've found a statistically significant decline in performance for the following metrics:
tRA, FIP, HR/9
I've found no statistically significant decline in performance under Hickey for the following metrics:
ERA, K/9, BB/9, and K/BB
The sample size is 37. In other words we have pairs of data (one portion of the pair is the performance under Hickey and the other is the performance under another pitching coach). Some of the pairs include Andy Pettittes 2006 with Hickey (Astros) and 2007 with Yankees. In fact nearly all of the 2006 Astros as they all had a new pitching coach the following year. Edwin Jackson from 2006 (no Hickey) to 2007 (w Hickey). Also Edwin Jackson from 2008 (w Hickey) to 2009 (no Hickey). James Shields 2006 (no Hickey) to 2007 (w Hickey).
*I did not break down the performance of the 2004 Astro pitchers compared to the 2005 Astros. It would have been too labor intensive for me to get the breakdowns for each metric for each pitcher under the previous PC in 2004 and under Hickey. Also it is questionable how much influence Hickey would have had in 2004 anyways. RJ made a post with some of these details Looking at Jim Hickey
I do wish there existed a bit larger of a sample, but this will have to do (the skewness and kurtosis have some issues). However I did want to post what I have now
Now onto the numbers
|FIP with Hickey||FIP w/o Hickey|
|Standard Error||0.15||Standard Error||0.12|
|Standard Deviation||0.93||Standard Deviation||0.75|
|Confidence Level(95.0%)||0.31||Confidence Level(95.0%)||0.25|
|tRA with Hickey||tRA w/o Hickey|
|Standard Error||0.21||Standard Error||0.18|
|Standard Deviation||1.25||Standard Deviation||1.03|
|Confidence Level(95.0%)||0.44||Confidence Level(95.0%)||0.36|
|ERA with Hickey||ERA w/o Hickey|
|Standard Error||0.27||Standard Error||0.23|
|Standard Deviation||1.67||Standard Deviation||1.40|
|Confidence Level(95.0%)||0.56||Confidence Level(95.0%)||0.47|
|K/9 with Hickey||K/9 w/o Hickey|
|Standard Error||0.29||Standard Error||0.23|
|Standard Deviation||1.78||Standard Deviation||1.38|
|Confidence Level(95.0%)||0.59||Confidence Level(95.0%)||0.46|
|BB/9 with Hickey||BB/9 w/o Hickey|
|Standard Error||0.17||Standard Error||0.19|
|Standard Deviation||1.02||Standard Deviation||1.18|
|Confidence Level(95.0%)||0.34||Confidence Level(95.0%)||0.39|
|K/BB w/ Hickey||K/BB w/o Hickey|
|Standard Error||0.16||Standard Error||0.13|
|Standard Deviation||1.00||Standard Deviation||0.80|
|Sample Variance||0.99||Sample Variance||0.64|
|Confidence Level(95.0%)||0.33||Confidence Level(95.0%)||0.27|
|HR/9 w/ Hickey||HR/9 w/o Hickey|
|Standard Error||0.07||Standard Error||0.06|
|Standard Deviation||0.45||Standard Deviation||0.36|
|Confidence Level(95.0%)||0.15||Confidence Level(95.0%)||0.12|
The data here FURTHER emphasizes how much of a crappy metric ERA is.
There are some outliers within this sample. I wish it was larger to smooth out some of the possible variables. Trever Miller has two entries that really help Hickey. Camp has two that really hurt Hickey. Percival really hurts Hickey. Jackson really hurts Hickey (had lower FIP w/o Hickey in 2006 also). Clemens did great with Hickey in 2006 and flamed in NY in 2007 etc.
Here is the file Google Doc
This is my first fanpost so pardon my crappy tables. I'm too busy watching the Rays dominate to figure that out
I'll take a look at whether players improve or decline under at least 2 years of Hickey next.