WAR, what is it good for? A lot, but not as much as....
by matthan on Jul 24, 2009 6:21 PM EDT
26 comments
...one of the most despised metrics in baseball.
Yes I'm talking about ERA
And no ERA is not better than WAR. Or FIP. Or anything for that matter.
But I did find something that ERA does a better job at (well with some help) than both WAR and FIP...
That thing is only the most important thing in baseball.....
Wins!!
Basically after our little discussion earlier today I wanted to see the correlations between certain metrics and team wins. These are the following metrics I looked at.
WAR, FIP, ERA, wOBA, wRC, wRAA, UZR
Honestly I could have chosen more or less, but these metrics were easy to sort since they were next to each other on certain pages. I'm lazier than BJ Upton.
I looked at WAR against Wins, and then the six different combinations (3 independent variables) of offense, defense, and pitching.
These were the adjusted r-squares (or r-squared for WAR). FYI I kept the constant zero for the multiple regressions as having a constant wouldn't make sense. For WAR I did not do that.
This includes all teams from 2002-2008.
| Independent Var | Adj R^2 | |
| wRC+FIP+UZR | 47.89% | |
| wRAA+FIP+UZR | 0.00% | |
| wOBA+FIP+UZR | 78.21% | |
| wRC+ERA+UZR | 55.98% | |
| wRAA+ERA+UZR | 0.00% | |
| wOBA+ERA+UZR | 83.82% | |
| WAR | 75.58% | |
Wow. So the best combination is wOBA+ERA+UZR. Surprising? WAR is pretty strong itself, and I'll get to that later.
So I dug deeper into that formula, and I found that the UZR component was insignificant*. The t-value was very small so the p-value was large. I decided to drop it. This is my result
| Independent Var | Adj R^2 | |
| wOBA+ERA | 83.89% | |
| wOBA+FIP | 73.25% | |
Quite clearly out of these variables the wOBA+ERA is the strongest in predicting team wins. Fairly surprising.
Here are the formulas for the WAR model and the best model
Wins = 0. + ( (wOBA) * 473.05 ) + ( (ERA) * -16.92 )
Wins = 46.66 + ( (Total WAR) * 0.989533 )
What is interesting is that in "real life" an increase of one WAR really only translates to 1 win. Again fairly interesting. Of course this isn't surprising as that is exactly what we'd have hoped to see. Gaining 1 WAR equals about 1 win (slightly less)
Using the formula this is what we get for 2009 and for 2008. I cannot use this WAR formula for 2009 as it is based on 162 games. The other formula adjusts to fit for 162 games. The WAR one does not.
2009
| Team | Wins | wOBA+ERA Wins |
| Dodgers | 61 | 98.5404 |
| Yankees | 58 | 96.49225 |
| Rays | 52 | 95.4496 |
| Red Sox | 55 | 93.6956 |
| Braves | 49 | 90.3258 |
| Blue Jays | 47 | 88.5576 |
| Phillies | 54 | 88.5505 |
| Rockies | 52 | 88.2502 |
| Cardinals | 52 | 87.45295 |
| Rangers | 52 | 87.204 |
| Mariners | 51 | 86.7797 |
| White Sox | 50 | 86.327 |
| Cubs | 48 | 85.49165 |
| Tigers | 49 | 84.80775 |
| Angels | 56 | 83.9821 |
| Twins | 48 | 82.67015 |
| Astros | 49 | 82.6117 |
| Giants | 51 | 82.4567 |
| Brewers | 48 | 80.53965 |
| Marlins | 49 | 78.04685 |
| Diamondbacks | 41 | 78.0433 |
| Pirates | 42 | 77.8431 |
| Mets | 44 | 77.6048 |
| Athletics | 40 | 74.36255 |
| Reds | 44 | 73.51655 |
| Royals | 37 | 70.7748 |
| Orioles | 41 | 70.08735 |
| Nationals | 28 | 69.0376 |
| Indians | 38 | 68.28815 |
| Padres | 37 | 61.91795 |
2008
| Team | Wins | wOBA+ERA wins | Error | WAR Wins | Error |
| Cubs | 97 | 99.1 | 2.1 | 95.8 | -1.2 |
| Red Sox | 95 | 98.7 | 3.7 | 104.3 | 9.3 |
| Rays | 97 | 94.3 | -2.7 | 95.0 | -2.0 |
| Phillies | 92 | 93.6 | 1.6 | 92.2 | 0.2 |
| Blue Jays | 86 | 93.3 | 7.3 | 90.0 | 4.0 |
| Brewers | 90 | 90.6 | 0.6 | 82.3 | -7.7 |
| White Sox | 89 | 90.3 | 1.3 | 90.8 | 1.8 |
| Cardinals | 86 | 90.2 | 4.2 | 88.3 | 2.3 |
| Dodgers | 84 | 90.1 | 6.1 | 82.2 | -1.8 |
| Mets | 89 | 89.1 | 0.1 | 85.4 | -3.6 |
| Yankees | 89 | 87.5 | -1.5 | 88.8 | -0.2 |
| Angels | 100 | 86.1 | -13.9 | 84.6 | -15.4 |
| Diamondbacks | 82 | 85.8 | 3.8 | 82.5 | 0.5 |
| Twins | 88 | 84.4 | -3.6 | 81.4 | -6.6 |
| Indiands | 81 | 82.5 | 1.5 | 87.0 | 6.0 |
| Marlins | 84 | 81.0 | -3.0 | 78.7 | -5.3 |
| Braves | 72 | 80.9 | 8.9 | 79.0 | 7.0 |
| Tigers | 74 | 78.7 | 4.7 | 77.8 | 3.8 |
| Astros | 86 | 77.6 | -8.4 | 76.1 | -9.9 |
| Athletics | 75 | 77.4 | 2.4 | 77.3 | 2.3 |
| Rangers | 79 | 77.1 | -1.9 | 83.3 | 4.3 |
| Rockies | 74 | 76.3 | 2.3 | 78.6 | 4.6 |
| Reds | 74 | 73.0 | -1.0 | 66.5 | -7.5 |
| Padres | 63 | 72.5 | 9.5 | 66.5 | 3.5 |
| Royals | 75 | 72.4 | -2.6 | 78.6 | 3.6 |
| Giants | 72 | 72.1 | 0.1 | 73.7 | 1.7 |
| Orioles | 68 | 69.9 | 1.9 | 72.3 | 4.3 |
| Nationals | 59 | 67.3 | 8.3 | 63.8 | 4.8 |
| Mariners | 61 | 66.6 | 5.6 | 66.5 | 5.5 |
| Pirates | 67 | 63.7 | -3.3 | 63.2 | -3.8 |
*Before anyone says anything I fully know that defense is important and that FIP is better than ERA. However this suggests that UZR may not be significant in determing expected wins and losses, and that ERA may be better than FIP in that same exercise.
1 recs |
26 comments
Email
Print
Comments
The reason for this is that just perhaps ERA explains team performance better than FIP and/or UZR
So while it caught me off guard it probably isn’t very unexpected
by matthan on Jul 24, 2009 6:34 PM EDT reply actions 0 recs
This is what I was going to say.
ERA is a poor metric because it does not fairly grasp the true talent of an individual on a team, but it does a decent job of rating an entire team…. again, decent, not great.
I think you similarly reverse the experiment and probably find that a pitcher with a lot of wins will typically have a low ERA, but not necessarily good peripherals.
by behn on Jul 24, 2009 6:50 PM EDT up reply actions 0 recs
ERA alone is bad at determing team wins
The R-squared for that alone is 47.9%
ERA and wOBA together is a whole different beast. That is the best combination that I’ve found so far. Much better than WAR alone.
by matthan on Jul 24, 2009 6:54 PM EDT up reply actions 0 recs
I really liked this. My college statstical project was something very similar to this
I tried to find correlation between team victories and various mainstream stats, including ERA, and found none of them to have any legitimate relevance (WHIP was the closest, ERA was pretty bad). it never occured to me then to do any combined stats. Good work.
I can't wait until we trade him for a reliever.
by kericr on Jul 25, 2009 11:11 AM EDT up reply actions 0 recs
Also just to clarify
I got the model based upon 2002-2008 data.
I then applied that model to what has happened so far in 2009, as well as retroactively applied it to 2008 (even though part of the model is based upon 2008) just for fun purposes.
by matthan on Jul 24, 2009 6:41 PM EDT reply actions 0 recs
A low ERA implies you have one (or more) of three things:
- Good pitching
- Good defense
- Good luck
Each is important to winning.
by R.J. Anderson on Jul 24, 2009 6:47 PM EDT reply actions 0 recs
Yeah once I posted it thats what I figured
ERA is a good measure of what your team success is, but is not a good indication on how the pitcher truly pitched.
by matthan on Jul 24, 2009 6:49 PM EDT up reply actions 0 recs
...Obviously.
Brad Ziegler had a scoreless inning streak. Brad Ziegler had not met BJ Upton.
by P Brady on Jul 24, 2009 6:50 PM EDT up reply actions 0 recs
Right, although this wasn't meant to determine a players true talent level
Rather how many wins a team should have. In that regards wOBA and ERA are better at doing that than WAR
I bet that wasn’t an “obviously”
by matthan on Jul 24, 2009 6:52 PM EDT up reply actions 0 recs
Not an obviously, but it stands to reason that a metric that combines stats that don’t measure actual performance would lose to a combination of a metric that does measure actual performance and one that uses game data.
by 17843 on Jul 24, 2009 11:49 PM EDT up reply actions 0 recs
Interesting to see that this shows that an increase of 1 war equals just about 1 real win
by matthan on Jul 24, 2009 6:59 PM EDT reply actions 0 recs
Which of course suggests a replacement team would win about 46-47 games
by matthan on Jul 24, 2009 7:03 PM EDT up reply actions 0 recs
Yeah I see Tango calculated it at 47.5
Interesting to see the formula projects that almost perfectly
by matthan on Jul 24, 2009 8:01 PM EDT up reply actions 0 recs
VINDICATION!
Actually, that’s good to hear, settles the question I guess.
by R.J. Anderson on Jul 24, 2009 7:20 PM EDT up reply actions 0 recs
Also does anyone know the RMSE for the pythag win/loss?
I’m curious to see the fit for that.
by matthan on Jul 24, 2009 6:59 PM EDT reply actions 0 recs
Glass may have done something on this last ~off-season.
Maybe not though.
by R.J. Anderson on Jul 24, 2009 7:20 PM EDT up reply actions 0 recs
I just did it for 2002-2008
Pythag is garbage compared to the two formulas listed above
The RMSE for the wOBA and ERA formula is 4.7
The RMSE for the WAR formula 5.8
The RMSE for pythag is 12.6
That isn’t even close.
If you want a better look at how many wins a team should have use those two formulas I found and ignore pythag.
by matthan on Jul 24, 2009 8:24 PM EDT up reply actions 0 recs
Actually I miscalculated it
I had 2009 in my spreadsheet which totally screwed everything up so check that
by matthan on Jul 24, 2009 8:31 PM EDT up reply actions 0 recs
What about 3rd order wins?
I can't help that I make some things look easier than they really are.
by Sandy Kazmir on Jul 28, 2009 9:15 AM EDT up reply actions 0 recs
Yea of course ERA is a good indicator of how many wins a team had.
It is not a good measure of pitcher talent. Its kind of like saying the best metric for estimitating team victories is the aggregate of wins on the pitching staff.
The purpose of FIP is to try to quicky measure a pitchers talent. THe gap between FIP and ERA goes a long way to finding under and overvalued pitchers.
Follow Me on Twitter @FreeZorilla
by FreeZorilla on Jul 24, 2009 9:14 PM EDT reply actions 0 recs
oh god this could fuel to fire for so many people...
why does this exist, it seems you understand that ERA is luck and wins in any given season can be effected by good luck.
WAR is just as good as it always was, and I’m sorry but I don’t actually buy it is any less valuable than we thought based on this information…
by Navi's_Navy on Jul 24, 2009 10:51 PM EDT reply actions 0 recs
The way I see this . . .
The team ERA is essentially RunsAllowed / Games so for a regression its the same as using Runs Allowed.
And wOBA as I understand allocates points based on the historical value of each kind of hit and a walk. So wOBA is more or less an estimate for runs scored per plate appearance. I don’t know how much plate appearances vary by team but if it isn’t much wOBA is a good proxy for runs scored, and to the extent that they do it probably lowers predictions for good offensive teams (e.g. Red Sox and Rangers in 2008)
So do wOBA and team ERA do a better job of predicting wins than runs scored and allowed?
by stevetb on Jul 27, 2009 2:49 AM EDT reply actions 0 recs
Comments For This Post Are Closed
Secondary Sidebar
User Tools
FanPosts
Community blog posts and discussion.
Certain photos copyright © 2009 by Associated Press or Getty Images. Any commercial use or distribution without the express written consent of Associated Press and Getty Images is strictly prohibited.
