Jim Hickey Statistical Analysis Part 2

by matthan on Jun 17, 2009 3:40 PM EDT Comment 87 comments

Welcome to Part 2 of my Jim Hickey Analysis.

Quick Recap

In Part 1 we looked at how pitchers performed under Hickey compared to their performance a year before or after with a different pitching coach. I'm going to go back and make a "Part 3" adjusting some of that data while using the methodology in Part 2. I'm also going to try to use many more metrics. In Part 2 I look at batted ball types as well as pitch types. I'll eventually add that into Part 1 as part of Part 3. We found that pitchers statistically significantly performed worse under Hickey than under a different PC for the following metrics: tRA, FIP, and HR/9.

You can find Part 1 here:

Jim Hickey Analysis Part 1

Part 2

We are going to look at the changes of metrics for a pitcher while they are under the Hickey reign. To do this I gathered the data two different ways and analyzed them.

The first method is "year over year". I took the 2005 Astros and compared to the 2006 Astros. The 2007 DRays to the 2008 Rays. The 2008 Rays to the 2009 Rays. In this a Rays pitcher can show up twice.

The second method is "first and last". I took the 2005 Astros to the 2006 Astros. And for some pitchers the 2007 Rays to the 2009 Rays, as well as some for '07 to '08 and '08 to '09. In this a Rays pitch can show up once (07-09 or 07-08, 08-09)

The sample size for the first "year over year" method is 26. The sample size for "first and last" is 20. I wish we had bigger samples, but we can still have some fun with this!

Remember this is all per pitcher. This is important to note and remember. For example a 3% change means on average each pitcher exhibits a 3% change.

The Metrics that will be analyzed

Basic: ERA

Advanced: FIP, BABIP

Performance Ratios: K/9, BB/9, HR/9, HR/FB

Batted Ball Types: LD%, FB%, GB%

Pitch Types: FB%, SL%, CT%, CB%, CH%, SF%

The Methodology

The goal here is to see if any of the changes in any of these metrics over the course of time is statistically significantly different from zero. In general if coaching is irrelevant than we would expect on average that the change in these metrics over the course of time to be 0.00. Of course there are some variables that impact these metrics other than coaching so do keep that in mind. However over time we would expect pitchers to strike guys out at the same rate, throw the same % of curveballs, give up about the same % of fly balls, and so forth.

Therefore the expected change for every one of these metrics is 0.00

To find the actual change I simply took Year 2-Year1

*Please note for some "actual changes" I took Year 3-Year 1 when applicable for "first and last"

Also note that a "negative" change indicates that the metric has decreased. For some metrics that is bad (K/9) and for some that can be good (BB/9)

	"Year over Year" average change per pitcher	"Year over Year" standard deviation	"Year over Year" P Value	"Year over Year" Is change Signifigant?
Basic
ERA	-0.048	2.198	91.24%	No
Advanced
FIP	0.209	0.846	21.39%	No
BABIP	-0.009	0.068	49.94%	No
Performance
K/9	-0.683	1.459	2.07%	YES
BB/9	0.208	1.012	30.02%	No
HR/9	-0.004	0.428	96.00%	No
HR/FB	-0.003	0.034	63.11%	No
Batted Ball
LD%	-0.001	0.038	89.48%	No
FB%	0.006	0.082	72.65%	No
GB%	-0.005	0.068	72.64%	No
Pitch Types
FB%	-0.010	0.072	47.61%	No
SL%	-0.007	0.053	50.37%	No
CT%	0.018	0.058	11.14%	No
CB%	0.015	0.035	3.26%	YES
CH%	-0.017	0.033	1.14%	YES
SF%	0.001	0.008	60.05%	No

	"First and Last" average change per pitcher	"First and Last" standard deviation	"First and Last" P Value	"First and Last" Is change Signifigant?
Basic
ERA	-0.062	1.950	88.77%	No
Advanced
FIP	0.272	0.942	20.50%	No
BABIP	-0.012	0.047	27.03%	No
Performance
K/9	-0.889	1.576	1.60%	YES
BB/9	0.270	0.878	17.72%	No
HR/9	-0.006	0.510	96.18%	No
HR/FB	-0.004	0.040	64.17%	No
Batted Ball
LD%	-0.001	0.038	87.98%	No
FB%	0.007	0.079	68.11%	No
GB%	-0.006	0.065	67.90%	No
Pitch Types
FB%	-0.013	0.079	46.17%	No
SL%	-0.009	0.039	31.16%	No
CT%	0.024	0.091	24.59%	No
CB%	0.020	0.047	7.06%	No
CH%	-0.022	0.033	0.49%	YES
SF%	0.001	0.009	59.29%	No

*Please remember this is based upon the change per pitcher. As in for "First and Last" each pitcher on average is going to throw 2% less changeups

Significant Results

"Year Over Year"

With this we see that the changes in K/9, CB%, and CH% are all statistically significant than zero.

The average change per pitcher is -.683 for K/9, 1.5% for CB%, and -1.7% for CH%

The P Values (two tailed test anything under 5% is sig) are .0207 for K/9, .0326 for CB%, and .0114 for CH%

"First and Last"

With this we see that the changes in K/9 and CH% are statistically significant than zero. CB% is no longer significant, but is very close to being so.

Interesting Tidbits

Besides for the metrics that have significant differences from zero there are some interesting things among the nonsignificant metrics. For example it appears the FIP under pitchers actually increase (get worse) when a guy is with Hickey. It isn't significant, but it was in Part 1 of my analysis. Pitchers on average also seem to walk more guys under Hickey. Also since HR/9 was a major topic of discussion in Part 1 it is important to note that HR/9 was nowhere near a significant. In other words pitchers gave up home runs at about the same rate at the start of their Hickey tenure as they did at the end. Batted ball types were very stagnant. Each pitcher on average gave up pretty much the same % of each batted ball the first year of Hickey as they did the 2nd year of Hickey.

I also removed Scott Kazmir from the data just to see the impact. The K/9 decline was still significant

The documents can be found here:

Year over Year

First and Last

My worries/concerns

Before I refine my data in Part 1 I'll hold off on my opinion on that data. However I find this info in Part 2 to be pretty bad for Hickey. The higher FIP is not significant so I'm not going to pin that on him, but pitchers are without a doubt (statistically significantly) averaging less K/9 as their time with Hickey increases. That is very bad sign for performance. Secondly he seems to be against the changeup and generally for the curveball. On average as his time with his pitchers increase pitchers seem to use the curveball more at the expense of the change up. I have no problem with him doing this in isolated cases if a specific pitchers skill set warranted such an action. However since the results were significant it appears he (well his pitchers) is systematically favoring the curveball at the expense of the change up.

6 recs | Comment 87 comments

Story-email Email Printer Print

Comments

The K/9 decrease really concerns me

I cannot think of a variable that would cause that to happen in such a short time frame. Same pitchers, same park, virtually the same age….the only issue is sample size (which is a big issue). But we almost have n=30…

by matthan on Jun 17, 2009 3:44 PM EDT reply actions 0 recs

Did you happen to look at sum of K's/sum of IP*9?

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Jun 17, 2009 3:52 PM EDT up reply actions 0 recs

Haven't gotten that far

Kind of burnt out. Although I’d argue that would have more to do with Maddon than Hickey.

Hickeys role is to get each individual player to improve.
Maddons role is to get players performing the most innings and in the best spots.

For example if two pitchers got worse and 1 pitcher got better than that in itself is a bad mark on Hickey. However if Maddon is a “good” manager he would be able to transfer those innings of the two pitchers that lost skills and give them to the pitcher that gained them. So while the overall K/9 of the team may stay the same that doesn’t reflect the change in skills with each pitcher.

by matthan on Jun 17, 2009 3:57 PM EDT up reply actions 0 recs

Don't blame you for being burnt out, thats a lot of good work

Also could be worth looking at K%. I’m not sure I agree on the Hickey/Maddon breakdown. I think at least combining the small inning (non-starter) guys into a summation group could be useful for confirmation purposes as I did with the 1st study.

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Jun 17, 2009 4:05 PM EDT up reply actions 0 recs

Honestly I don't think the issue is looking at the individual metrics

I think the issue has more to do with at what IP are those metrics reliable for each pitcher.

At a certain level it wont matter the number of innings a pitcher has because his metric is around what it should be given his current skill set.

I do think it is absolutely critical to look at it individually. For example lets say James Shields throws 3% more changeups but everyone else throws 3% less. Since Shields throws so many given his % and his IP it would totally mask the fact that every pitcher but 1 is throwing less change ups.

by matthan on Jun 17, 2009 4:10 PM EDT up reply actions 0 recs

It's jsut about questioning the relaibility of the innings.

The starters all posted K/9s within about .15 of their previous #. The relievers deviated much farther from their previous #s, most likely due to the limited # of innings pitched. It makes you question how many innings is enough before jumping to a conclusion. A way to offset it is to combine the lesser inning guys. While it doesn’t measure the # of improved/worsened pitchers, it does measure the overall change among that group.

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Jun 17, 2009 4:41 PM EDT up reply actions 0 recs

I took out all the relievers just for kicks

K/9 now has a P-Value of 6% so it is barely insignificant. 94% confidence isn’t bad though….The change for CH% and CB% still remained significant at 95%…

I also don’t see where you get the .15. It looks like the variation…even for starters is much higher than that.

Also I agree that the larger IP is nice to get a more true K/9 number. However we should expect the variation at any IP to be normal. As in for every pitcher even at low IP it should be normally distributed around its mean. So it is highly unlikely for almost all relievers with lower IP to have a decline in K/9 due to nothing but chance. We would expect that the variation would result in some K/9 higher than their previous years.

by matthan on Jun 17, 2009 5:14 PM EDT up reply actions 0 recs

.15 was HR/9

haven’t fiddled with K, sorry if that wasn’t clear

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Jun 17, 2009 5:40 PM EDT up reply actions 0 recs

haha

“not clear”= carelessly writing Hr/9 as k/9. can’t imagine why you are confused

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Jun 17, 2009 5:41 PM EDT up reply actions 0 recs

I understand your hesitancy to skew it towards starters

However by lumping the relievers in and averaging the metrics, aren’t you skewing it towards the relievers who due to their smaller inning sample will have mroe variation?

I’m going to look into it today. If nothing else the group sum will either help verify your findings or leave the discussion open for further examination.

The other issue with the pitch calling is we still don’t know if thats Hickey’s influence or Navi’s game calling.

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Jun 18, 2009 8:34 AM EDT up reply actions 0 recs

Regarding calling games the sample includes a good handful of guys on the Astros as well as the Rays. Navi could have an influence for sure if he suddenly shifted towards another pitch, but the Astros time has an impact as well.

by matthan on Jun 18, 2009 8:48 AM EDT up reply actions 0 recs

True

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Jun 18, 2009 8:50 AM EDT up reply actions 0 recs

I think the difference is on what we are trying to accomplish

You seem to be more concerned with the impact Hickey would have on a team and a season as a whole. That is definitely a valid approach, but I think something like that would be very difficult to measure.

I’m more concerned about the expected changes on each pitcher on the staff. As in if I was the manager of the Mets right now and I hired Hickey I’d want to know the expected changes on each one of my pitchers. Of course a 3% change in a pitch type for a 200 IP guy results in a hell of a lot more different pitches than a 3% change in a 50 IP guy.

So with this analysis I said we can generally expect each pitcher to throw ~2% more curveballs after a year with Hickey, but that doesn’t mean the TEAM will throw 2% more curveballs. The composition of the team and the distribution of innings could shift in such a way where more innings goes towards CB pitches, resulting in a far larger increase than 2%. Or perhaps the distribution of innings goes towards pitchers that do not throw CBs, resulting in perhaps even a decline.

I think that is the main problem when analyzing team stats from one year to another. The distribution of the innings is always different and sometimes vastly so. If the team has more guys throwing with a higher true talent k/9 then naturally the team k/9 will go up even if on an individual basis each pitcher has a reduction of k/9. That doesn’t mean Hickey had any influence on the k/9, but rather how many innings Joe decided to pitch everyone.

by matthan on Jun 18, 2009 8:56 AM EDT up reply actions 0 recs

Far too often on baseball sites we see people say things such as

“The teams ____ went up over last year”

That just really doesn’t tell us much of anything. It could have been caused by tons of different reasons.

1. Better pitchers taking roster spots from worse ones
2. Better defense for some metrics
3. Better pitchers taking away the innings of worse ones
4. Pitchers actually getting better
5. Easier schedule (although over a full season that should balance out)

I absolutely cringe when ESPN says something like that and immediately attributes it to the pitching coach, manager, hitting coach, etc.

by matthan on Jun 18, 2009 9:03 AM EDT up reply actions 0 recs

You are right

There are flaws in any way of looking at data, which is why it needs to be examined in several different ways for confirmation. Starters and relievers can be difficult to compare for reasons beyond sample size of IP. I’m not saying your method is flawed, it should jsut be examined from other angles.

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Jun 18, 2009 9:08 AM EDT up reply actions 0 recs

Of course

Like you said the variation of virtually every metric (although probably not batted ball or pitch type) would be higher for relievers. So that definitely impacts what we would expect to see.

This data definitely does not tell us what change we would expect to see for each starter and also for each reliever. It just tells us what change we would expect for each pitcher as a whole. There is definitely a difference. No doubt the expected change for starters for something like K/9 is probably a bit lower than it is for relievers…but then again thats not what I was trying to measure.

If we were running a team we would definitely try to find a way find the expected change for lefties, righties, starters, relievers, old guys, young guys etc. Each one of those subgroups. Alas as a common member of Draysbay I just decided to do pitchers as a whole

by matthan on Jun 18, 2009 9:14 AM EDT up reply actions 0 recs

You should have saved some of your effort doing analysis this deep so that you could do more surface analysis.

"Where we all wait in earnest with pudding in hand for the Upton comet to sail through the roofed skies, so that we may meet Him."

by kericr on Jun 17, 2009 6:09 PM EDT up reply actions 0 recs

Oh and again I aplogize for the crappy tables

I must be missing a button somewhere. When I copy/paste it all my formatting goes. But this is my 2nd fanpost so I’ll figure it out.

by matthan on Jun 17, 2009 3:48 PM EDT reply actions 0 recs

I better not have to post a bitch about this post not getting to 5 recs.

"Where we all wait in earnest with pudding in hand for the Upton comet to sail through the roofed skies, so that we may meet Him."

by kericr on Jun 17, 2009 4:11 PM EDT reply actions 0 recs

Thanks

As we all know the longer and more detailed a post is the less comments it gets.

I should have studied whether Navi is statistically getting fatter. FAT!!

by matthan on Jun 17, 2009 4:25 PM EDT up reply actions 0 recs

I'm not sure I buy into the analysis as a way to analyze Hickey's performance just yet

But the work needs to be recognized, especially when it’s this level of depth.

"Where we all wait in earnest with pudding in hand for the Upton comet to sail through the roofed skies, so that we may meet Him."

by kericr on Jun 17, 2009 6:08 PM EDT up reply actions 0 recs

Google Docs are not curently being shared

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Jun 17, 2009 4:47 PM EDT reply actions 0 recs

Should work now

by matthan on Jun 17, 2009 4:56 PM EDT up reply actions 0 recs

I'm almost surely going to adjust Part 1 to fit this methodology and add in the other metrics (esp the pitch types)

So be on the look out for that

by matthan on Jun 17, 2009 6:01 PM EDT reply actions 0 recs

I'm a little late to reading your posts

But want to say thanks for the in depth work. Having more smart people contribute is part of what makes this site great.

That said, I’m debating whether this is the right statistical test to use, and more so, if using a null hypothesis of zero is appropriate. Really what you are testing is based on the assumption that a pitchers metric will stay the same year over year. Do we ever actually expect that? Did we think Balfour’s K/9 was going to stay the same this year? Or Kazmir’s SL%? Because you are testing the assumption that they do.

I’ll give this some thought tonight and respond back tomorrow if I can think of a better way to mathematically look at this. I’m a bit removed from doing this type of work, so it isn’t at my fingertips.

by Mulva on Jun 17, 2009 10:39 PM EDT reply actions 0 recs

Thanks for the response

I"m also about to crash, but yes that is exactly what I was testing for. My assumption was exactly that those metrics should not change year over year. The pitcher is with the same team, in the same park, facing the same division, facing essentially the same line ups. On average I’d expect the change over the sample of pitchers to be zero given a large enough sample.

Why would we expect a pitchers FB% to change year over year given that he is facing essentially the same schedule? There shouldn’t be a significant change.

The problem with using a more advanced forecast method, as basically I’m just saying the forecast is the previous years number, is that we would be using performance under a different pitching coach in order to get that more advanced forecast number. Perhaps for Rays pitchers that pitched 2007-2009 I could use some other forecasted number using ‘07/’08.

Of course I could be totally missing something since this is really my first dive into using this with baseball stats. I threw this one together on the fly. I really wanted to see if there was some way to take a look at Hickey rather than just talking trash about him. Either way help as much as you can. I definitely know you know your stuff. I’m just rambling on at this point.

by matthan on Jun 17, 2009 11:47 PM EDT up reply actions 0 recs

I don't really have much to add, but I rec'd it earlier, and definitely appreciate the effort put forth.

by R.J. Anderson on Jun 17, 2009 11:52 PM EDT reply actions 0 recs

Summed K/9 of the group from Yr 1 to Yr 2 Under Hickey

This includes 05-06 Astros and 07-08 Rays (sample size 11):

Group K/9 Year 1: 8.32
Group K/9 Year 2: 7.82

8 of the 11 pitchers had declines in K/9
Average Decline of the Decliners: -1.36
Average Increase of the Gainers: .78

This does seem to serve as confirmation of a decline in K/9 under Hickey

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Jun 18, 2009 9:05 AM EDT reply actions 0 recs

To find the group total you just took the ((total number of K’s/total number of innings)*9)?

Are you looking at just starters, relievers, or just any pitcher that pitched 05/06 and then 07/08?

by matthan on Jun 18, 2009 9:16 AM EDT up reply actions 0 recs

I took the pitchers from your sheet including 09 so mine did not include Kaz or Ejax

I’ll redo it with the names from the other sheet

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Jun 18, 2009 9:24 AM EDT up reply actions 0 recs

Yeah I have tons of sheets

I get confused myself looking at my work

by matthan on Jun 18, 2009 9:29 AM EDT up reply actions 0 recs

One sheet has the players first and last year with Hickey. So guys like Kaz and Ejax that would be 2007 to 2009 whereas someone like Glover would be 2007 to 2008 and Cormier would be 2008 to 2009. The other sheet has year over year. So it would be 2007 to 2008 plus 2008 to 2009 for the pitchers that were there all three years or just 07-08 or 08-09 if they were just the two years.

by matthan on Jun 18, 2009 10:17 AM EDT up reply actions 0 recs

that was all pitchers combined, I'd be happy to break it out

Be Back in a few

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Jun 18, 2009 9:20 AM EDT reply actions 0 recs

Sure

My question is that I see 9 pitchers that pitched on both the ‘05 Astros and ’06 Astros and 10 pitchers on the ’07 Rays as well as the ’08 Rays. I believe I’m using a 20 IP min this time though.

by matthan on Jun 18, 2009 9:24 AM EDT up reply actions 0 recs

I just took a quick look at the pitchers on both the 07 and 08 Rays

Sample Size 10

The “Group” K/9 (all K’s/all innings)*9
07-8.14
08-7.16
Decline of 11.9%

The “Individual” K/9 (average of all personal K/9s)
07-8.41
08-7.51
Decline of 10.70%

8 of the 10 pitchers declined. Only Balfour and Howell increased. They also have more innings pitched in 08 over 07 putting more influence on the group K/9 in 08 with their increase

Some of the bigger declines were for Edwin, Andy, and Wheeler. Those three guys had a big jump in innings. Their declines had more influece on the group k/9 because of that

Shields had the same amount of innings but his K/9 decreased by 13%

by matthan on Jun 18, 2009 9:38 AM EDT reply actions 0 recs

I think this is a good example of what I was saying before

Maddon diverting innings from worse pitchers with declining metrics (Reyes, Glover) and giving more innings to pitchers with higher and rising metrics (Howell, Balfour).

by matthan on Jun 18, 2009 9:42 AM EDT up reply actions 0 recs

Overall Group of 19

K/9 declined from 7.68 to 7.27 from yr 1 to yr 2
13 of 19 declined

I’ll break down starters and relievers a littler later

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Jun 18, 2009 9:43 AM EDT reply actions 0 recs

Starters (Astros 05-06 & Rays 07-08)

Looking at just starters (IP > 100 for both years) here is the breakdown by pitcher and +/- in K/9 from yr 1 to yr 2 under Hickey.

0.55 Andy Pettitte
0.22 Roger Clemens
-0.08 Roy Oswalt
0.91 Wandy Rodriguez
-0.91 Andy Sonnanstine
-1.85 Edwin Jackson
-1.00 James Shields
-0.60 Scott Kazmir

The average change was -.35

For the group (Total K/Total IP*9):
Yr 1: 7.52
Yr 2: 6.95

Change was -.57

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Jun 18, 2009 10:21 AM EDT reply actions 0 recs

Nice

Thats the same numbers I got on my spreadsheet. I’m not sure if google docs uploaded everything right.

Although I think you want to include Hammel also as most of his innings came as a starter in 07-08. Same with Backe. He was a starter in 05 and I believe he got hurt in 06. Just a FYI.

by matthan on Jun 18, 2009 10:24 AM EDT up reply actions 0 recs

I included Hammel as a reliever b/c of the IP size

I’m not including Backe in either group. The idea being with a smaller sample size of innings he could skew the #s as he could fall more away fromt he mean than the rest. Of course, with a cosntant pitcher typically K rates are higher as a reliever.

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Jun 18, 2009 10:29 AM EDT up reply actions 0 recs

Right

If Backe was a starter in 2005 and 2006 he should be striking guys out at the same rate even if one year he had a lower IP. Of course the lower IP would cause his 2nd number to have a higher possible error

by matthan on Jun 18, 2009 10:32 AM EDT up reply actions 0 recs

Either way it looks like our numbers show the same thing

The variation is just based on how we want to classify certain pitchers. In the context of starter/reliever I understand leaving out Backe and possibly Hammel. However in the overall context of how he impacts pitchers I couldn’t leave them out. Combined those pitchers have 350 IP with Hickey so its is definitely worthwhile.

by matthan on Jun 18, 2009 10:40 AM EDT up reply actions 0 recs

Hammel is a true tweener

I don’t recall Backe’s injury, but it could have potentially been impacting performance too. Either way I think you’ve stumbled across an interesting find.

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Jun 18, 2009 10:42 AM EDT up reply actions 0 recs

I can definitely understand leaving them out if you are just looking at starters and relievers independently

by matthan on Jun 18, 2009 11:08 AM EDT up reply actions 0 recs

Player "First Year IP “Second Year IP” K/9
Andy Pettitte 222.1 214.1 0.55
Brad Lidge 70.2 75.0 -0.64
Brandon Backe 149.1 43.0 -1.87
Chad Qualls 79.2 88.2 -1.10
Dan Wheeler 73.1 71.1 0.11
Roger Clemens 211.1 113.1 0.22
Roy Oswalt 241.2 220.2 -0.08
Russ Springer 59.0 59.2 -1.30
Wandy Rodriguez 128.2 135.2 0.90
Al Reyes 60.2 22.2 -2.84
Andy Sonnanstine 130.2 193.1 -0.91
Dan Wheeler 25 66.1 -2.17
Edwin Jackson 161 183.1 -1.86
Gary Glover 77.1 34 -0.12
Grant Balfour 22 58.1 1.60
J.P. Howell 51 89.1 0.62
James Shields 215 215 -1.00
Jason Hammel 85 78.1 -1.72
Scott Kazmir 206.2 152.1 -0.60

First column is first year IP
Second column is 2nd year IP
Third is the change in K/9 from first to second year

The “starters” avg change (inc Backe and Hammel) is:

-.637

The relivers avg change is:

-.649

The average change per starter and reliever is almost the EXACT same looking with the sample of Astro 05/06 combined with Rays 07/08

by matthan on Jun 18, 2009 10:30 AM EDT up reply actions 0 recs

Relievers

-0.63 Brad Lidge
-1.09 Chad Qualls
0.11 Dan Wheeler
-1.30 Russ Springer
-2.85 Al Reyes
-2.17 Dan Wheeler
-0.11 Gary Glover
1.61 Grant Balfour
0.63 J.P. Howell
-1.72 Jason Hammel

Avg Change -.75
1st yr K/9 8.54
2nd Yr K/9 8.19
Group Change -0.35

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Jun 18, 2009 10:31 AM EDT reply actions 0 recs

I think the group rate is lower than the average change precisely because

Of a manager going away from pitchers that aren’t working towards pitchers that are working. 2008 Balfour and Howell got a boatload more innings than in 2007 whereas Reyes ended up getting shipped out of town when his numbers declined in 2008.

So really the group number is a function of the manager allocating the innings correctly.

by matthan on Jun 18, 2009 10:34 AM EDT up reply actions 0 recs

Which I'm sure Hickey has something to do with this

No doubt he is telling Maddon which guys are doing better and worse.

But its not surprising that the group number would be lower than the individual change. That is exactly the managers job. He needs to maximize his resources. No need for a manager to stick with a sinking ship.

by matthan on Jun 18, 2009 10:37 AM EDT up reply actions 0 recs

Also starters were the inverse

It does seem more probable for that to occurthan the pen due to more steady usage. There are still so many other variables in play that I believe its a worthwhile exercise to look at the information both ways. Every way has its weaknesses.

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Jun 18, 2009 10:45 AM EDT up reply actions 0 recs

Well logically..

It is tougher to allocate innings in a different manner among starters. If Scott Kazmir has a declining K rate it is pretty tough to just replace him with a guy with a higher K rate. You can see more shuffling with your #4 and #5 starters typically, but I think if your #1 and #2 have declining K Rates it is almost definite that your starting 5 will see a drop as a group.

Relievers are much easier to swap around.

But yeah that was a good catch. I didn’t even think about that until you mentioned it.

by matthan on Jun 18, 2009 10:53 AM EDT up reply actions 0 recs

It also depends on how you define a "good" job

If Shields and Sonny increase their K/9 but the other 7 or so holdovers lose K/9 then would you consider it a good job? The overall team K/9 would be about the same year over year. But 7 of the 9 would have lost K/9. I guess it just depends how you look at it.

by matthan on Jun 18, 2009 10:59 AM EDT up reply actions 0 recs

So we now there is a correlation with Hickey and declining K rates

Now the question is, is this intentional? The only rationale I can come up with for why is to reduce walks or in the case of starters pitch more frequently to contact to minimize pitch counts. This could be plausible given the top of the line defense of the Rays.It would be interesting to see the breakdown of K% between the top and the bottom of the order of the opposition. Also worth taking a look at the average pitches/PA of opposing hitters.

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Jun 18, 2009 10:59 AM EDT reply actions 0 recs

The problem with pitching to contact is the increase in walks per pitchers

Pitchers are on average walking more guys under Hickey. It isn’t statistically significant, but I think it can help us make the conclusion that pitchers aren’t “pitching to contact”.

by matthan on Jun 18, 2009 11:01 AM EDT up reply actions 0 recs

Basically a decline in K/9 should be met by a decline in BB/9

That could then be argued it is a good strategic decision to rely more on your defense. As we see we are definitely striking out less per pitcher and quite possibly walking more…

by matthan on Jun 18, 2009 11:03 AM EDT up reply actions 0 recs

I'm going to break down unintentional walks the same way we broke down K's.

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Jun 18, 2009 11:08 AM EDT up reply actions 0 recs

I did find that BB/9 rose from between somewhere between .2 to .3 depending on the method I used

It just wasn’t statistically significant. I think the same pattern will arise with starters and relievers with BB/9 as it did with K/9. My guess would be that on average that starters have a lower variation in walk rates as do relievers.

Although I did find as a whole walks (BB/9) rose on a per pitcher average by .21 on a year over year basis with Hickey. But its not statistically significant so….

the p-value was .3 so thats just not very strong evidence that the change is significantly different than zero.

by matthan on Jun 18, 2009 11:12 AM EDT up reply actions 0 recs

I think a better explanation is actually directly in the data

Pitchers on average are throwing curveballs more than change ups. My own guess is that if a pitcher was initially throwing more change ups then for some reason he thought that was the better pitch for whatever reason. So going away from it and going towards a different pitch could possibly mean less control (more walks, less k’s) as well as more contact within the zone due to it being a worse pitch (less k’s).

by matthan on Jun 18, 2009 11:06 AM EDT up reply actions 0 recs

Actually it isn't pitchers are throwing more curves than changes ups

Pitchers on avearge are throwing more curveballs than what they used to as well as throwing less change ups than what they used to

Sorry for my mistype, but that is a big distinction

by matthan on Jun 18, 2009 11:07 AM EDT up reply actions 0 recs

My conclusion for Part 2

In conclusion there is enough evidence to say that K/9 declines for a pitcher under two consecutive years with Hickey. On average we should expect a pitcher to lose about .68 off of his K/9 each year. Broken down between starters and relievers would be a bit different. My educated guess is that the starter change would be slightly lower than the average change whereas the reliever change would be a bit higher than the average change. I could also guess that the variation among the reliever changes would be higher than the starter changes. To determine the changes among starters and relievers that we would have to look at each set, starters and relievers, independently instead of all pitchers at once. Unfortunately we already have sample size issues so it would be even more extreme if we broke it down in such a way. How the team K/9 changes year over year is certainly linked to the changes of each pitcher, but also to how those innings themselves are allocated. Higher K/9 pitchers throwing more innings may result in a higher team K/9 even if there is a .68 drop across the board for each pitcher.

Curveballs are thrown more and change ups are also thrown less under Hickey on average for each pitcher. I don’t think this really needs to be broken down by starter and reliever since I think the minimum IP needed to have equal variance is far smaller for pitch types.

What does this mean? Should we fire Hickey? Of course not, this is one piece of a very large puzzle. I won’t even say this drop is his fault. All I can say is that these changes did in fact statistically occur. That’s all.

by matthan on Jun 18, 2009 11:00 AM EDT reply actions 0 recs

Well done Matt

This is a really nice read. Would you be averse to doing something similar for the rest of the AL East?

Rays Win!

by Sandy Kazmir on Jun 18, 2009 2:44 PM EDT up reply actions 0 recs

Is anyone else still paying attention?

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Jun 18, 2009 11:09 AM EDT reply actions 0 recs

Unfortunately most people are pretty deadset on the view of Hickey and the impact of pitching coaches in general

They either like him or hate him based on some bias or some simple qualitative measure. This is differently digging into new waters trying to statistically analyze a pitching coach. Although of course we have some real bright minds on this board that would obviously be interested in this type of stuff.

by matthan on Jun 18, 2009 11:15 AM EDT up reply actions 0 recs

I find it interesting, but I just don't know if you can really draw any hard conclusions out of it.

by Suttree on Jun 18, 2009 12:25 PM EDT up reply actions 0 recs

I also found the batted ball data interesting

The batted ball year to year percent changes were not signficantly different than zero. They basically stayed stagnant for each pitcher.

So throwing more curves and less change ups (on a per pitcher basis) did not statistically change the batted ball types….

by matthan on Jun 18, 2009 11:22 AM EDT reply actions 0 recs

Outside of JP Howell does the throwing more curves really hold up?

He seemed to be the massive driver of that change.

by Mulva on Jun 18, 2009 2:20 PM EDT up reply actions 0 recs

Also interesting

That Wheeler and Balfour both increased their K rates the first year with Hickey but then had a larger drop in magnitude the second year.

by Mulva on Jun 18, 2009 2:24 PM EDT up reply actions 0 recs

Keep an eye on Balfour

His velocity has been on the rise back to last years levels. He followed a similar path last year.

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Jun 18, 2009 2:31 PM EDT up reply actions 0 recs

Balfour is an interesting case

We all can see his troubles this year. As FreeZorilla said it could be a velocity issue. I do wish we had a stronger sample size.

Although with Wheeler I wouldn’t read too much into it. The first example was with the Astros when he was younger and the Rays a couple years later.

I do think we all let ourselves be a bit biased about the information we know about each player. It would almost be better if we just totally ignored the name of each player and looked at the data.

If the sample is large enough (of course this is unfortunately just 26) then those weird outliers should balance out. For example if we take the average K/9 (-.68) as the central point and looked at the outliers you’d see that there are 3 changes of at least 2 K/9 above the mean and 3 changes of at least 2 K/9 below the mean. I can’t really do too much with it but these 26 changes does appear to somewhat exhibit normal characteristics. Thats just my first impression though

by matthan on Jun 18, 2009 3:26 PM EDT up reply actions 0 recs

JP Howell is definitely a big part of it. He threw 8% more CBs in 2008 than 2007 and then 11% more in 2009 than in 2008. That is a major increase from 2007 to 2009. He wasn’t the only one though. Shields, Garza, Balfour, Sonnanstine, Oswalt, Springer, and Jackson all had increases larger than the average increase.

All in all of the 26 comparisons 4 didnt throw curves. So that lets 22. Out of that 14 pitchers had increase in curveball usage year over year.

But you are right regarding Howell having a large influence. Also his trend towards the curveball seems to be a bit more pronounced while with the Rays

by matthan on Jun 18, 2009 3:16 PM EDT up reply actions 0 recs

Changeups is probably the most conclusive case out of them all

20 out of the 26 pitchers threw less change ups the second year

Out of the 6 that increased change ups, two of those figures were below 1%, with 1 pitcher barely above 1%.

In fact the biggest gainer may be a bit flukey since it was Al Reyes in his 2008 season with just 22 IP.

16 of the 20 pitchers that threw less changeups in fact threw over 1% less…

FYI

If we raise the min IP limit to 30 we lose three pitchers.
The new average decrease in Changeup usage is 1.9%……the P Value drops to .0049 (.49%).
The current P-Value with the 20IP limit is .0114

by matthan on Jun 18, 2009 3:38 PM EDT up reply actions 0 recs

FreeZorilla I saw you theorized that improved defense and pitching to contact may be the culprit so..

I just quickly took a look and tried to control for defense…

The assumption is the Rays defense got better from 2007 to 2008. Therefore the "pitch to contact" strategy would begin at that point. As in the strategy would be implemented in 2008 so they wouldn’t be pitching to more contact in 2009 than in 2008.

So what I did was remove the 2007 to 2008 Rays from the data and just looked at the 2005 to 2006 Astros and 2008 to 2009 Rays.

The drop isn’t as large…

This new data has a average decrease of -.55 K/9 per pitcher whereas the full set including 07-08 has an average drop of -.89

So it still exists even after the Rays pitches supposedly "adjusted" to better defense

by matthan on Jun 18, 2009 3:54 PM EDT reply actions 0 recs

I took out all the relievers

The average drop in this data set then becomes -.320 K/9 for the 05/06 Astros and 08/09 Rays

by matthan on Jun 18, 2009 3:56 PM EDT up reply actions 0 recs

Did the Stros make a conscious effort to improve defense?

Rays Win!

by Sandy Kazmir on Jun 18, 2009 6:48 PM EDT up reply actions 0 recs

No it doesn't look like it

The 05/06 Astros fielded essentially the same position players except for LF. On 2006 Preston Wilson started the most games there and in 2005 Chris Burke. Combined they only had 1 error.

Of course the Rays defense from 2008 to 2009 is static. Well at least we are playing the same guys so the pitchers expectations of defense is the same. Since they expect close to the same their strategy shouldn’t differ.

by matthan on Jun 19, 2009 9:09 AM EDT up reply actions 0 recs

While you are right, it was not predicatable and therefore not a gameplan adjustment

the Stros UZR jumped from 19.5 to 43.9 and UZR/150 from 2.9 to 6.1. However, BABIP went backwards from .289 to .294. It seems logical that there is a relationship between BABIP and K/9 given that a lower BABIP means more batted balls are converted into outs.

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Jun 19, 2009 9:25 AM EDT up reply actions 0 recs

Your assumption is those things are coincidental. They are not. Defense would get better and then the pitchers would want to throw for more contact. They wouldn’t happen at the same time.

Although I disagree with your last statement. A K has an out value of 1 every time. A batted ball certainly doesn’t. No pitcher is going to trade a K for a batted ball in a vacuum. The big variable in play are walks. Walks would have to decline at a rate that would make up for more men reaching base via batted ball (the BABIP goes down but the # of batted balls will go up).

by matthan on Jun 19, 2009 9:47 AM EDT up reply actions 0 recs

If strategy does not change meaning you are not "pitching to contact"

but more batted balls are converted to outs, you will still have a lower K/9 .

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Jun 19, 2009 9:56 AM EDT up reply actions 0 recs

There are just a lot of factors to consider. You are right in the sense if more batted balls are turned into outs then the pitcher will face less batters so his K/9 should decrease based on that.

I’m saying a pitcher isn’t going to trade K’s for batted balls even with an improved defense without having other variables adjust because a K is an out 100% of the time whereas a batted ball is an out 70% of the time. A higher K rate is always preferred no matter how good your defense is given that a strike out is more valuable than a batted ball.

But a lower BABIP will mean less total batters faced then that definitely should have an impact on K/9. However the change in total number of batters faced shouldn’t change K%. So I think that would probably be a better metric to use in this case.

by matthan on Jun 19, 2009 10:41 AM EDT up reply actions 0 recs

Right, barring a strategic change, the K% is the more telling stat

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Jun 19, 2009 10:47 AM EDT up reply actions 0 recs

I'd to see you work with IBB though

I can’t see how that alone would make up the increase in walks.

Although honestly you can’t really use IBB/9 for relievers. There just wouldn’t be enough events to justify it.

You’d probably have to manually go in and subtract the IBB for the total BB for each pitcher. Then recalculate the BB/9 and then examine the year over year change with the slimmed down BB/9

Of course I’m interested in any possible way you can imagine looking at the numbers

by matthan on Jun 18, 2009 4:14 PM EDT reply actions 0 recs

I think I'm going to run some regressions

The topic seems to have shifted on figuring out what is causing K/9 to change. If we regress the change of K/9 against BABIP and/or other variables I wonder what we will see. I’ll take a look at this some other team.

by matthan on Jun 18, 2009 5:03 PM EDT reply actions 0 recs

time

by matthan on Jun 18, 2009 5:03 PM EDT up reply actions 0 recs

By the way my sheet is coming

I’m not sure what conclusions to come to anymore and I won’t have much time today to work on. Rather than sit on it, I am going to publish it and hope others can play with it. its loaded with data. Everything broken down by startes and relievers, K/9, K%, uBB/9, K/uBB, Pitches/Inning, Strike %. I’d like to do more work with Swinging Strikes, First Strike % etc but it may not be for a few days. I’d like to keep the topic alive.

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Jun 19, 2009 9:27 AM EDT reply actions 0 recs

I'm in the same boat. I can't really dig into the numbers today or this weekend

There are a couple things I’m interested in for sure.

One would be running regressions to see what really drives K/9 (or changes in K/9). Higher walks? Lower BABIP? Pitch types (or changes of pitch types)? etc.

The second would be far more data intensive. I’d like to see how the Hickey data compares to other pitching coaches. Sure here we see that K/9 has decreased over the years as a pitcher is under Hickey. However I’d like to get the same data on a bunch of pitching coaches and compare them to see if Hickeys decline is actually statistically different than the other pitching coaches. If the other pitching coaches are losing K/9 similarly then we could say Hickey didn’t perform worse than other pitching coaches. Likewise we could do this with some other pitching coaches to see which guys, if any of them, actually performed better than the league….and those guys would be guys to target if we wanted to upgrade.

by matthan on Jun 19, 2009 9:53 AM EDT up reply actions 0 recs

Count me in.

I’ve been interested in seeing this applied to other PC’s just as a comparison tool. Let me know which coaches you want me to do and which workbook to plug data into. If a couple other people jump on board it shouldn’t be too bad, but that’s a lot for 1-2 guys to slog through.

Rays Win!

by Sandy Kazmir on Jun 19, 2009 10:07 AM EDT up reply actions 0 recs

For this analysis I have a few questions

Would there be any merit to viewing this in a little different direction.

If we want to test if Hickey does indeed have an impact on the players he coaches, could we look at these metrics for AL as a whole then compare to it the group under Hickey.

In other words, do pitchers under Jim Hickey experience changes that can be attributed to him when compared to the deviation of the rest of the league (larger population).

So take league wide changes between Year X and Year Y, then compare that to the sample of pitchers under that were not under Hickey in Year X but were under Hickey in Year Y.

by wtbudlight on Jun 26, 2009 2:48 PM EDT reply actions 0 recs