Jim Hickey Statistical Analysis Part 2

Welcome to Part 2 of my Jim Hickey Analysis.

Quick Recap

In Part 1 we looked at how pitchers performed under Hickey compared to their performance a year before or after with a different pitching coach. I'm going to go back and make a "Part 3" adjusting some of that data while using the methodology in Part 2. I'm also going to try to use many more metrics. In Part 2 I look at batted ball types as well as pitch types. I'll eventually add that into Part 1 as part of Part 3.  We found that pitchers statistically significantly performed worse under Hickey than under a different PC for the following metrics: tRA, FIP, and HR/9.

You can find Part 1 here:

Jim Hickey Analysis Part 1

Part 2

We are going to look at the changes of metrics for a pitcher while they are under the Hickey reign. To do this I gathered the data two different ways and analyzed them.

The first method is "year over year". I took the 2005 Astros and compared to the 2006 Astros. The 2007 DRays to the 2008 Rays. The 2008 Rays to the 2009 Rays. In this a Rays pitcher can show up twice.

The second method is "first and last". I took the 2005 Astros to the 2006 Astros. And for some pitchers the 2007 Rays to the 2009 Rays, as well as some for '07 to '08 and '08 to '09.  In this a Rays pitch can show up once (07-09 or 07-08, 08-09)

The sample size for the first "year over year" method is 26. The sample size for "first and last" is 20. I wish we had bigger samples, but we can still have some fun with this!

Remember this is all per pitcher. This is important to note and remember. For example a 3% change means on average each pitcher exhibits a 3% change.


The Metrics that will be analyzed

Basic:   ERA

Advanced: FIP, BABIP

Performance Ratios:  K/9, BB/9, HR/9, HR/FB

Batted Ball Types:  LD%, FB%, GB%

Pitch Types: FB%, SL%, CT%, CB%, CH%, SF%



The Methodology

The goal here is to see if any of the changes in any of these metrics over the course of time is statistically significantly different from zero. In general if coaching is irrelevant than we would expect on average that the change in these metrics over the course of time to be 0.00. Of course there are some variables that impact these metrics other than coaching so do keep that in mind. However over time we would expect pitchers to strike guys out at the same rate, throw the same % of curveballs, give up about the same % of fly balls, and so forth.

Therefore the expected change for every one of these metrics is 0.00

To find the actual change I simply took Year 2-Year1

*Please note for some "actual changes" I took Year 3-Year 1 when applicable for "first and last"

Also note that a "negative" change indicates that the metric has decreased. For some metrics that is bad (K/9) and for some that can be good (BB/9)


"Year over Year" average change per pitcher "Year over Year" standard deviation "Year over Year" P Value "Year over Year" Is change Signifigant?
ERA -0.048 2.198 91.24% No
FIP 0.209 0.846 21.39% No
BABIP -0.009 0.068 49.94% No
K/9 -0.683 1.459 2.07% YES
BB/9 0.208 1.012 30.02% No
HR/9 -0.004 0.428 96.00% No
HR/FB -0.003 0.034 63.11% No
Batted Ball
LD% -0.001 0.038 89.48% No
FB% 0.006 0.082 72.65% No
GB% -0.005 0.068 72.64% No
Pitch Types
FB% -0.010 0.072 47.61% No
SL% -0.007 0.053 50.37% No
CT% 0.018 0.058 11.14% No
CB% 0.015 0.035 3.26% YES
CH% -0.017 0.033 1.14% YES
SF% 0.001 0.008 60.05% No


"First and Last" average change per pitcher "First and Last" standard deviation "First and Last" P Value "First and Last" Is change Signifigant?
ERA -0.062 1.950 88.77% No
FIP 0.272 0.942 20.50% No
BABIP -0.012 0.047 27.03% No
K/9 -0.889 1.576 1.60% YES
BB/9 0.270 0.878 17.72% No
HR/9 -0.006 0.510 96.18% No
HR/FB -0.004 0.040 64.17% No
Batted Ball 
LD% -0.001 0.038 87.98% No
FB% 0.007 0.079 68.11% No
GB% -0.006 0.065 67.90% No
Pitch Types
FB% -0.013 0.079 46.17% No
SL% -0.009 0.039 31.16% No
CT% 0.024 0.091 24.59% No
CB% 0.020 0.047 7.06% No
CH% -0.022 0.033 0.49% YES
SF% 0.001 0.009 59.29% No

*Please remember this is based upon the change per pitcher. As in for "First and Last" each pitcher on average is going to throw 2% less changeups


Significant Results

"Year Over Year"

With this we see that the changes in K/9, CB%, and CH% are all statistically significant than zero.

The average change per pitcher is -.683 for K/9, 1.5% for CB%, and -1.7% for CH%

The P Values (two tailed test anything under 5% is sig) are .0207 for K/9, .0326 for CB%, and .0114 for CH%

"First and Last"

With this we see that the changes in K/9 and CH% are statistically significant than zero. CB% is no longer significant, but is very close to being so.


Interesting Tidbits

 Besides for the metrics that have significant differences from zero there are some interesting things among the nonsignificant metrics. For example it appears the FIP under pitchers actually increase (get worse) when a guy is with Hickey. It isn't significant, but it was in Part 1 of my analysis. Pitchers on average also seem to walk more guys under Hickey. Also since HR/9 was a major topic of discussion in Part 1 it is important to note that HR/9 was nowhere near a significant. In other words pitchers gave up home runs at about the same rate at the start of their Hickey tenure as they did at the end. Batted ball types were very stagnant. Each pitcher on average gave up pretty much the same % of each batted ball the first year of Hickey as they did the 2nd year of Hickey.

I also removed Scott Kazmir from the data just to see the impact. The K/9 decline was still significant


The documents can be found here:

 Year over Year

First and Last

My worries/concerns

Before I refine my data in Part 1 I'll hold off on my opinion on that data. However I find this info in Part 2 to be pretty bad for Hickey. The higher FIP is not significant so I'm not going to pin that on him, but pitchers are without a doubt (statistically significantly) averaging less K/9 as their time with Hickey increases. That is very bad sign for performance. Secondly he seems to be against the changeup and generally for the curveball. On average as his time with his pitchers increase pitchers seem to use the curveball more at the expense of the change up.  I have no problem with him doing this in isolated cases if a specific pitchers skill set warranted such an action. However since the results were significant it appears he (well his pitchers) is systematically favoring the curveball at the expense of the change up.

This post was written by a member of the DRaysBay community and does not necessarily express the views or opinions of DRaysBay staff.

Log In Sign Up

Log In Sign Up

Please choose a new SB Nation username and password

As part of the new SB Nation launch, prior users will need to choose a permanent username, along with a new password.

Your username will be used to login to SB Nation going forward.

I already have a Vox Media account!

Verify Vox Media account

Please login to your Vox Media account. This account will be linked to your previously existing Eater account.

Please choose a new SB Nation username and password

As part of the new SB Nation launch, prior MT authors will need to choose a new username and password.

Your username will be used to login to SB Nation going forward.

Forgot password?

We'll email you a reset link.

If you signed up using a 3rd party account like Facebook or Twitter, please login with it instead.

Forgot password?

Try another email?

Almost done,

By becoming a registered user, you are also agreeing to our Terms and confirming that you have read our Privacy Policy.

Join DRaysBay

You must be a member of DRaysBay to participate.

We have our own Community Guidelines at DRaysBay. You should read them.

Join DRaysBay

You must be a member of DRaysBay to participate.

We have our own Community Guidelines at DRaysBay. You should read them.




Choose an available username to complete sign up.

In order to provide our users with a better overall experience, we ask for more information from Facebook when using it to login so that we can learn more about our audience and provide you with the best possible experience. We do not store specific user data and the sharing of it is not required to login with Facebook.