Introducing volatility to the 2009 Rays starting pitching
Given the return of Scotty Kazmir, the demotion of Sonny, and questions surrounding the starting pitching in general I figured I'd take a deeper look into our Super Six Starters.
By now we all know where each pitcher stands with their season long FIP and tRA as well as where they stand among nearly useless but popular metrics such as wins/losses and ERA. FIP and tRA are excellent, and they do paint most of the picture. However I do think they miss a key component: volatility.
Volatility is important because teams are not evaluation based upon cumulative 162 game statistics. Teams are judged based upon 9 inning incriments. There is a difference between two pitchers if one has a FIP/tRA in three games ranging from 0, 6, and 6 and another pitcher with a FIP/tRA of 4, 4, and 4. The team of the first pitcher likely goes 1-2 whereas the other team actually has a shot of winning three games. As we certainly all know W's/Ls in the public context in relation to pitchers is largely irrelevant, however, the number of times a pitcher pitches well enough to give his team a shot at winning is largely ignored and is ultimately very important.
For the time being all of this is another way at looking what has happened. I have no idea whether it has any predictive power so lets rather use at as another way to explain 2009 so far.
There are quite a few ways to look at volatility and I hope to do so in the future. In this piece we are just going to look at it in general with a simple introduction.
First and foremost I took the FIP of each game started for each pitcher**. Obviously each pitcher has a different number of starts, but here are the FIPs for eachs of our starters for each start.
Firstly I apologize for the graph being tough to read. Basically some general highlights. Price has the best start. Sonny has the worst start. Sonny and Kaz both have two starts that are far worse than their others. Shields worst start was his first and has been solid ever since.
Here are some basic statistics:
All three of these metrics are fairly important. Firstly I calculated the "Yearly FIP" by using the cumulative season counting stats for each pitcher. I also average the FIP for each start for each pitcher. They certainly are different. Both are important in their own ways. However what is even more interesting is the standard deviations. Sonny and Kaz have by far the most volatility. Neimann has the least, which is quite a bit surprising. However is a high standard deviation good or bad? I'll glimpse into that in a bit but here is what I see. Neimann is consistently below average. That is going to result in poor losses (with our elite offense a bit more wins) since he has no volatility resulting in excellent games. Kazmir and Sonny both have higher FIPs, but they have far higher volatility. Therefore they'll have more games falling into the "winning" category due to their volatility. Generally if you have a great FIP you want low volatility, if you have a high FIP then more volatility is good. All of this with the overall goal in mind, maximing wins and losses for the team.
This next chart we are going to look at the individual volatility for each pitcher. 0% is set as the average FIP per start for each pitcher. Anything over 0% is something higher than their individual average. Anything lower is less than their average. Of course 0% for Kazmir is far higher base FIP than it is for Shields.
The few things I take out of this chart. Sonny clearly is the most volatile around his average. Neimann is very consistent. Garza is remarkably inconsistently consistent. He bounces above and below his mean slightly but very often. Kazmir and Sonny have two starts that really crush them. Shields has been improving as compared to his average (as has Neimann).
The last thing I want to share is something I find very interesting. As you've seen have had quite a bit of W and L theme to this post, and this is ultimately why. I cannot emphasize enough how wins and losses in the context of MLB pitchers are irrelevant. However my modifed W and L are actually a fairly good indicator of what each pitchers record SHOULD BE given the team that they are on. Typical wins and losses are based a lot on luck, run support, bullpen, and distribution of runs. I take all of those variables out here.
I assume that the Rays RPG has no standard deviation. We score our runs per game average every game. After 74 games this is 5.66
I assume that all 9 innings of the game is performed at the FIP of the starting pitcher. Therefore taking the bullpen out of the question
Therefore if the players FIP for that game is less than the Rays average RPG then they put the team in position to win. On average they should have won the game. If it is over the Rays RPG then they deserve to have lost. Clearly the variability of runs scored is important. A 6 FIP clearly gives the team a better chance of winning than a 15 FIP, and this modified W/L ignores that. I'm merely counting the times above and below the Rays RPG
According to this Sonny should have the second most Ws on the team. Why? Because of volatility. He has had 10 games where he has put up a lower FIP than what the Rays score. Sure some of those 5 games he "lost" were pretty bad, but they all count the same. Kazmir fits the same bill. Now lets take a look at Neimann and Shields. They've been our two most consistent pitchers. However Shields has been consistent on the good side and Neimann on the below average side. You see this in the hypothetical W/Ls. Even though the Rays have the highest RPG in the league Neimann has only truly pitched to a .500 record. This is due to his low volatility at a FIP right around our RPG. Shields**** has low volatility, but has a low FIP resulting in tons of good results.
Clearly a lot more work can be done with this type of analysis. I'm particularly interested in downside risk. However hopefully this gave some folks a different angle onto how to look at our pitchers.
**My FIP will be a bit different than what you see on FanGraphs. I'm using a different constant and only starting pitching innings. It won't truly matter as this is more of a comparison between starters and I used the same formula for each starter. However be very cautious if you try to compare one of these guys in this analysis to numbers you pull off of FanGraphs or another reference
****Shields start on Friday June 26th is not included
5 recs |
38 comments
Comments
Rec'd
Although, you could have used tra if you really wanted to be thorough.
Also, Scotty Kaz is now Kitty Kaz, as now dubbed as me.
Brad Ziegler had a scoreless inning streak. Brad Ziegler had not met BJ Upton.
by P Brady on Jun 27, 2009 2:14 AM EDT reply actions 0 recs
Although
I assume that all 9 innings of the game is performed at the FIP of the starting pitcher. Therefore taking the bullpen out of the question
Whats the point if you’re going to take out NDs because of bullpen, you should do innings of SP than the average bullpen FIP for the remaining innings or whatever. Isn’t that what really killed Shields?
Brad Ziegler had a scoreless inning streak. Brad Ziegler had not met BJ Upton.
by P Brady on Jun 27, 2009 2:17 AM EDT up reply actions 0 recs
Also, I'd like to request the mean volatility to a pitcher's start. It makes sense in my sleep deprived brain, ok?
Brad Ziegler had a scoreless inning streak. Brad Ziegler had not met BJ Upton.
by P Brady on Jun 27, 2009 2:18 AM EDT up reply actions 0 recs
Yeah I decided on FIP since it was easier to get the game by game data
I’d imagine tRA would show close to the same thing.
Besides I don’t really want the W/L thing to be taken literally. This isn’t meant that Sonny should have won 10 games. Clearly RPG has its own variation and variables. And like you basically said IP (and the BP) has a large say in the matter. Two pitchers could have the exact same FIP with the exact same run support (and distribution of runs) but far different outcomes due to IP and the bullpen. No doubt about that.
I just wanted to show another way how volatility was truly impacting the outcomes. This was essentially showing how many games each pitcher gave the Rays a pretty good shot at winning.
by matthan on Jun 27, 2009 2:37 AM EDT up reply actions 0 recs
IIRC, tRA was created specifically because the creators thought FIP was flawed
So it should be at least a little different.
I can't wait until we trade him for a reliever.
by kericr on Jun 27, 2009 9:07 AM EDT up reply actions 0 recs
Yeah it would be different
Although I don’t think the comparison of pitchers would be that different. As in I don’t think it would change the fact that Sonny has been volatile whereas Neimann has been steady. The level at which each has performed would be a fair amount different though.
by matthan on Jun 27, 2009 10:38 AM EDT up reply actions 0 recs
He has pitched really really well
Whats even better is that he is getting stronger. The run support will come.
by matthan on Jun 27, 2009 2:39 AM EDT up reply actions 0 recs
47-27 record
I could live with that
Follow Me on Twitter @FreeZorilla
by FreeZorilla on Jun 27, 2009 7:28 AM EDT reply actions 0 recs
I love the demonstration of volatility
Its something I think is important out of the pen that does not get measured ie Wheeler fails infrequently but harder.
Its tough to average the bullpen FIP in given that the bullpen avg will always work in the favor of a W. There would need to some type of IP penalty factored in for a start that does not go deep. Just not sure how to do it effectively.
FIP is also set so the lg avg FIP = lg AVG ERA which only accounts for earned runs. The Rays offensive stats are based on total runs. Seems like you would need to use the same stat based on the avg opponents FIP vs the Rays for the comparison.
Follow Me on Twitter @FreeZorilla
by FreeZorilla on Jun 27, 2009 7:41 AM EDT reply actions 0 recs
Opponents FIP
4.81 assuming same 3.2 constant and = # pitched to the Rays staff, innings will be slightly off due to partial innings at the end of games but its a ballpark.
Follow Me on Twitter @FreeZorilla
by FreeZorilla on Jun 27, 2009 10:15 AM EDT up reply actions 0 recs
also didnt adjust for HBP or IBB
lazy saturday
Follow Me on Twitter @FreeZorilla
by FreeZorilla on Jun 27, 2009 10:16 AM EDT up reply actions 0 recs
I didn't think about looking at opponents FIP compared instead rpg
If I was going to use some type of modifed W/L I wanted to take out luck, distributions, bullpens etc. Using RPG I definitely kept the unearned factor. I’ll throw in the opponents FIP against us and see how that changes.
by matthan on Jun 27, 2009 10:40 AM EDT up reply actions 0 recs
Another demonstration of
how great it is to have Shields.
by Kevin Cowley on Jun 27, 2009 10:57 AM EDT reply actions 0 recs
Ok based upon FreeZorillas suggestion about using FIP to remove unearned runs I embarked on this adjustment
Basically this hypothetical W/L takes away luck in all forms, unearned runs, distribution of runs, bullpen, opposing team, etc.
This is a measure of in that particular game did our starting pitcher outpitch our total opponents FIP? I found the total FIP against us and again the assumption is that is how the opponents fare against the Rays every single game (no volatility for our offense). How often did our pitchers beat that?
The FIP I found was 4.79. That was using a constant of 3.15. My total IP for the opponents may be a tad bit off due to walk offs etc, but it shouldn’t be off by more than a few outs

by matthan on Jun 27, 2009 11:19 AM EDT reply actions 0 recs
This changes a few things, but the basic conclusion remains the same
The main difference is Sonny, Shields, Price, and Neimann lost a win.
However using the opponents FIP against us Sonny has still outpitched our opponents on a per game basis in 8 of 15 games. Price and Neimann have now been outpitched more than the other way around. Kazmir still has hand quite a few good games.
by matthan on Jun 27, 2009 11:22 AM EDT up reply actions 0 recs
RGlass and I took a similar look at this in the offseason in a different sort of way.
Here is the LINK, I’ll update to this point in the season just to get an idea of how these compare. Like this, there are a ton of assumptions involved.
Rays Win!
by Sandy Kazmir on Jun 27, 2009 11:54 AM EDT reply actions 0 recs
Here's the updated stuff, I'm also going to compare this to FIP as you did to see what it looks like.
I’ll use 3.15 as my constant.
Rays Win!
by Sandy Kazmir on Jun 27, 2009 12:49 PM EDT up reply actions 0 recs
Linking it would be smart
Rays Win!
by Sandy Kazmir on Jun 27, 2009 1:07 PM EDT up reply actions 0 recs
Very interesting
I wonder what it would look like if we were able to get the FIP against us each game instead of runs and compared it to the per game FIP for each pitcher. That would probably be quite a bit of work to get the FIP against on a per game basis.
by matthan on Jun 27, 2009 1:16 PM EDT up reply actions 0 recs
It's a good idea because I just finished up the FIP analysis and,
in the context of “wins”, quite a bit is lost as FIP/IP is generally higher than R/IP across the board. I think it could be a good ranking system, league-wide, but it is not as good of a barometer as “est. wins”. Link to the FIP stuff. I’m thinking of expanding this to the rest of the ALE and then maybe take a look at FIP Against. If I do the latter I will use the total game data (SO, BB, HR for all pitchers not just the starter) That should make it a little bit easier, in fact let me put that together real quick.
Rays Win!
by Sandy Kazmir on Jun 27, 2009 1:45 PM EDT up reply actions 0 recs
Yeah for FIP against I just use the total game
In mine all I did was look at the total cumulative Rays data and then added up the IP against the Rays (I may of missed outs due to walks offs etc).
by matthan on Jun 27, 2009 1:53 PM EDT up reply actions 0 recs
I'm just basing it on 9 innings, so yeah it will miss some outs.
Is there a way to automate something along the lines of comparing the opposition FIP against every instance of our starter FIP to get a probability of us winning that game? Then do that down the line for every opposition FIP.
Rays Win!
by Sandy Kazmir on Jun 27, 2009 1:57 PM EDT up reply actions 0 recs
Here's a link to FIP against based on 9 innings per game
LINK Editing is unlocked to all if you want to collab and play around with this.
Rays Win!
by Sandy Kazmir on Jun 27, 2009 2:00 PM EDT up reply actions 0 recs
May I ask how you found that info?
I don’t even know where to look to find the HR,IBB,HBB,K,BB for the Rays each individual game organized in a way that wouldn’t require tons of manual work.
by matthan on Jun 27, 2009 2:02 PM EDT up reply actions 0 recs
Also I think the key would be doing what you did with your probability distribution of runs with FIP against
And then comparing it to our starters FIPs. And if some way we can normalize our bullpen FIP which to add that back in given the number of IP our starter went then we would essentially reach FIW and FIL.
by matthan on Jun 27, 2009 2:04 PM EDT up reply actions 0 recs
Our Bullpen FIP is 3.92 in 221.9 IP
The IP is off just due to the way fangraphs presents the data
What do you think of using that FIP in filling in the remaining innings each game that the starter wasn’t able to fill?
Of course it isn’t perfect since we are assuming equal distribution. It probably doesn’t work at all since it actually benefits each starter other than Shields for not pitching a lot of innings.
by matthan on Jun 27, 2009 2:14 PM EDT up reply actions 0 recs
To get it accurate I think I'd have to adjust based upon early and late bullpen guys
Like have two different bullpen FIPS, on for innings 1-7 and one for 8-9. Use like Wheeler, Howell, and Percival for 8-9 and the rest of the crew the earlier innings. It would create much more of a built in penalty for leaving earlier in the game
by matthan on Jun 27, 2009 2:18 PM EDT up reply actions 0 recs
For FIP I took the batter gamelogs and isolated HR, K, BB
Applied the formula and went from there. So far I am about a 1/3rd of the way through a matrix that takes each of their FIPs divided by just our starters FIPs. Anything over 1.00 should be a win with anything under a loss, from there I’ll need to decide the next step.
Rays Win!
by Sandy Kazmir on Jun 27, 2009 2:16 PM EDT up reply actions 0 recs
Yeah thats pretty much what I wanted to do with my next step
With this I just wanted to really normalize offensive production and look at pitcher volatility. Of course run distribution is very important. This should get us really in the right direction in seeing how our pitchers performance should really be impacting the bottom line
Interesting nontheless. I’m going to have to give it up for the remainder of the day. I got some crap to do before heading to the game tonight
by matthan on Jun 27, 2009 2:20 PM EDT up reply actions 0 recs
I essentially did the same matrix with FIP but with RPG and FIP/G
I just used an if statement to isolate the wins from the losses. I’ll post my googledoc
by matthan on Jun 27, 2009 2:22 PM EDT up reply actions 0 recs
http://spreadsheets.google.com/ccc?key=rfYZL6lfSjaELgUkB_KvTXA
There is my file. The graphs don’t work but thats besides the point
by matthan on Jun 27, 2009 2:26 PM EDT up reply actions 0 recs
Oh it's good what I've put together
I’m going to make a front page story about it.
Rays Win!
by Sandy Kazmir on Jun 27, 2009 3:33 PM EDT up reply actions 0 recs

by 























