Welcome to Part 2 of my Jim Hickey Analysis.
In Part 1 we looked at how pitchers performed under Hickey compared to their performance a year before or after with a different pitching coach. I'm going to go back and make a "Part 3" adjusting some of that data while using the methodology in Part 2. I'm also going to try to use many more metrics. In Part 2 I look at batted ball types as well as pitch types. I'll eventually add that into Part 1 as part of Part 3. We found that pitchers statistically significantly performed worse under Hickey than under a different PC for the following metrics: tRA, FIP, and HR/9.
You can find Part 1 here:
We are going to look at the changes of metrics for a pitcher while they are under the Hickey reign. To do this I gathered the data two different ways and analyzed them.
The first method is "year over year". I took the 2005 Astros and compared to the 2006 Astros. The 2007 DRays to the 2008 Rays. The 2008 Rays to the 2009 Rays. In this a Rays pitcher can show up twice.
The second method is "first and last". I took the 2005 Astros to the 2006 Astros. And for some pitchers the 2007 Rays to the 2009 Rays, as well as some for '07 to '08 and '08 to '09. In this a Rays pitch can show up once (07-09 or 07-08, 08-09)
The sample size for the first "year over year" method is 26. The sample size for "first and last" is 20. I wish we had bigger samples, but we can still have some fun with this!
Remember this is all per pitcher. This is important to note and remember. For example a 3% change means on average each pitcher exhibits a 3% change.
The Metrics that will be analyzed
Advanced: FIP, BABIP
Performance Ratios: K/9, BB/9, HR/9, HR/FB
Batted Ball Types: LD%, FB%, GB%
Pitch Types: FB%, SL%, CT%, CB%, CH%, SF%
The goal here is to see if any of the changes in any of these metrics over the course of time is statistically significantly different from zero. In general if coaching is irrelevant than we would expect on average that the change in these metrics over the course of time to be 0.00. Of course there are some variables that impact these metrics other than coaching so do keep that in mind. However over time we would expect pitchers to strike guys out at the same rate, throw the same % of curveballs, give up about the same % of fly balls, and so forth.
Therefore the expected change for every one of these metrics is 0.00
To find the actual change I simply took Year 2-Year1
*Please note for some "actual changes" I took Year 3-Year 1 when applicable for "first and last"
Also note that a "negative" change indicates that the metric has decreased. For some metrics that is bad (K/9) and for some that can be good (BB/9)
|"Year over Year" average change per pitcher||"Year over Year" standard deviation||"Year over Year" P Value||"Year over Year" Is change Signifigant?|
|"First and Last" average change per pitcher||"First and Last" standard deviation||"First and Last" P Value||"First and Last" Is change Signifigant?|
*Please remember this is based upon the change per pitcher. As in for "First and Last" each pitcher on average is going to throw 2% less changeups
"Year Over Year"
With this we see that the changes in K/9, CB%, and CH% are all statistically significant than zero.
The average change per pitcher is -.683 for K/9, 1.5% for CB%, and -1.7% for CH%
The P Values (two tailed test anything under 5% is sig) are .0207 for K/9, .0326 for CB%, and .0114 for CH%
"First and Last"
With this we see that the changes in K/9 and CH% are statistically significant than zero. CB% is no longer significant, but is very close to being so.
Besides for the metrics that have significant differences from zero there are some interesting things among the nonsignificant metrics. For example it appears the FIP under pitchers actually increase (get worse) when a guy is with Hickey. It isn't significant, but it was in Part 1 of my analysis. Pitchers on average also seem to walk more guys under Hickey. Also since HR/9 was a major topic of discussion in Part 1 it is important to note that HR/9 was nowhere near a significant. In other words pitchers gave up home runs at about the same rate at the start of their Hickey tenure as they did at the end. Batted ball types were very stagnant. Each pitcher on average gave up pretty much the same % of each batted ball the first year of Hickey as they did the 2nd year of Hickey.
I also removed Scott Kazmir from the data just to see the impact. The K/9 decline was still significant
The documents can be found here:
Before I refine my data in Part 1 I'll hold off on my opinion on that data. However I find this info in Part 2 to be pretty bad for Hickey. The higher FIP is not significant so I'm not going to pin that on him, but pitchers are without a doubt (statistically significantly) averaging less K/9 as their time with Hickey increases. That is very bad sign for performance. Secondly he seems to be against the changeup and generally for the curveball. On average as his time with his pitchers increase pitchers seem to use the curveball more at the expense of the change up. I have no problem with him doing this in isolated cases if a specific pitchers skill set warranted such an action. However since the results were significant it appears he (well his pitchers) is systematically favoring the curveball at the expense of the change up.