WAR, what is it good for? A lot, but not as much as....

by matthan on Jul 24, 2009 6:21 PM EDT

...one of the most despised metrics in baseball.

Yes I'm talking about ERA

And no ERA is not better than WAR. Or FIP. Or anything for that matter.

But I did find something that ERA does a better job at (well with some help) than both WAR and FIP...

That thing is only the most important thing in baseball.....

Wins!!

Basically after our little discussion earlier today I wanted to see the correlations between certain metrics and team wins. These are the following metrics I looked at.

WAR, FIP, ERA, wOBA, wRC, wRAA, UZR

Honestly I could have chosen more or less, but these metrics were easy to sort since they were next to each other on certain pages. I'm lazier than BJ Upton.

I looked at WAR against Wins, and then the six different combinations (3 independent variables) of offense, defense, and pitching.

These were the adjusted r-squares (or r-squared for WAR). FYI I kept the constant zero for the multiple regressions as having a constant wouldn't make sense. For WAR I did not do that.

This includes all teams from 2002-2008.

Independent Var		Adj R^2
wRC+FIP+UZR		47.89%
wRAA+FIP+UZR		0.00%
wOBA+FIP+UZR		78.21%
wRC+ERA+UZR		55.98%
wRAA+ERA+UZR		0.00%
wOBA+ERA+UZR		83.82%
WAR		75.58%

Wow. So the best combination is wOBA+ERA+UZR. Surprising? WAR is pretty strong itself, and I'll get to that later.

So I dug deeper into that formula, and I found that the UZR component was insignificant*. The t-value was very small so the p-value was large. I decided to drop it. This is my result

Independent Var		Adj R^2
wOBA+ERA		83.89%
wOBA+FIP		73.25%

Quite clearly out of these variables the wOBA+ERA is the strongest in predicting team wins. Fairly surprising.

Here are the formulas for the WAR model and the best model

Wins = 0. + ( (wOBA) * 473.05 ) + ( (ERA) * -16.92 )

Wins = 46.66 + ( (Total WAR) * 0.989533 )

What is interesting is that in "real life" an increase of one WAR really only translates to 1 win. Again fairly interesting. Of course this isn't surprising as that is exactly what we'd have hoped to see. Gaining 1 WAR equals about 1 win (slightly less)

Using the formula this is what we get for 2009 and for 2008. I cannot use this WAR formula for 2009 as it is based on 162 games. The other formula adjusts to fit for 162 games. The WAR one does not.

2009

Team	Wins	wOBA+ERA Wins
Dodgers	61	98.5404
Yankees	58	96.49225
Rays	52	95.4496
Red Sox	55	93.6956
Braves	49	90.3258
Blue Jays	47	88.5576
Phillies	54	88.5505
Rockies	52	88.2502
Cardinals	52	87.45295
Rangers	52	87.204
Mariners	51	86.7797
White Sox	50	86.327
Cubs	48	85.49165
Tigers	49	84.80775
Angels	56	83.9821
Twins	48	82.67015
Astros	49	82.6117
Giants	51	82.4567
Brewers	48	80.53965
Marlins	49	78.04685
Diamondbacks	41	78.0433
Pirates	42	77.8431
Mets	44	77.6048
Athletics	40	74.36255
Reds	44	73.51655
Royals	37	70.7748
Orioles	41	70.08735
Nationals	28	69.0376
Indians	38	68.28815
Padres	37	61.91795

2008

Team	Wins	wOBA+ERA wins	Error	WAR Wins	Error
Cubs	97	99.1	2.1	95.8	-1.2
Red Sox	95	98.7	3.7	104.3	9.3
Rays	97	94.3	-2.7	95.0	-2.0
Phillies	92	93.6	1.6	92.2	0.2
Blue Jays	86	93.3	7.3	90.0	4.0
Brewers	90	90.6	0.6	82.3	-7.7
White Sox	89	90.3	1.3	90.8	1.8
Cardinals	86	90.2	4.2	88.3	2.3
Dodgers	84	90.1	6.1	82.2	-1.8
Mets	89	89.1	0.1	85.4	-3.6
Yankees	89	87.5	-1.5	88.8	-0.2
Angels	100	86.1	-13.9	84.6	-15.4
Diamondbacks	82	85.8	3.8	82.5	0.5
Twins	88	84.4	-3.6	81.4	-6.6
Indiands	81	82.5	1.5	87.0	6.0
Marlins	84	81.0	-3.0	78.7	-5.3
Braves	72	80.9	8.9	79.0	7.0
Tigers	74	78.7	4.7	77.8	3.8
Astros	86	77.6	-8.4	76.1	-9.9
Athletics	75	77.4	2.4	77.3	2.3
Rangers	79	77.1	-1.9	83.3	4.3
Rockies	74	76.3	2.3	78.6	4.6
Reds	74	73.0	-1.0	66.5	-7.5
Padres	63	72.5	9.5	66.5	3.5
Royals	75	72.4	-2.6	78.6	3.6
Giants	72	72.1	0.1	73.7	1.7
Orioles	68	69.9	1.9	72.3	4.3
Nationals	59	67.3	8.3	63.8	4.8
Mariners	61	66.6	5.6	66.5	5.5
Pirates	67	63.7	-3.3	63.2	-3.8

*Before anyone says anything I fully know that defense is important and that FIP is better than ERA. However this suggests that UZR may not be significant in determing expected wins and losses, and that ERA may be better than FIP in that same exercise.

This post was written by a member of the DRaysBay community and does not necessarily express the views or opinions of DRaysBay staff.

Tweet Comment 26 comments | 1 recs |

Do you like this story?

Comments

The reason for this is that just perhaps ERA explains team performance better than FIP and/or UZR

So while it caught me off guard it probably isn’t very unexpected

by matthan on Jul 24, 2009 6:34 PM EDT reply actions

This is what I was going to say.

ERA is a poor metric because it does not fairly grasp the true talent of an individual on a team, but it does a decent job of rating an entire team…. again, decent, not great.

I think you similarly reverse the experiment and probably find that a pitcher with a lot of wins will typically have a low ERA, but not necessarily good peripherals.

by behn on Jul 24, 2009 6:50 PM EDT up reply actions

ERA alone is bad at determing team wins

The R-squared for that alone is 47.9%

ERA and wOBA together is a whole different beast. That is the best combination that I’ve found so far. Much better than WAR alone.

by matthan on Jul 24, 2009 6:54 PM EDT up reply actions

I really liked this. My college statstical project was something very similar to this

I tried to find correlation between team victories and various mainstream stats, including ERA, and found none of them to have any legitimate relevance (WHIP was the closest, ERA was pretty bad). it never occured to me then to do any combined stats. Good work.

I can't wait until we trade him for a reliever.

by kericr on Jul 25, 2009 11:11 AM EDT up reply actions

Also just to clarify

I got the model based upon 2002-2008 data.

I then applied that model to what has happened so far in 2009, as well as retroactively applied it to 2008 (even though part of the model is based upon 2008) just for fun purposes.

by matthan on Jul 24, 2009 6:41 PM EDT reply actions

A low ERA implies you have one (or more) of three things:

- Good pitching
- Good defense
- Good luck

Each is important to winning.

by R.J. Anderson on Jul 24, 2009 6:47 PM EDT reply actions

Yeah once I posted it thats what I figured

ERA is a good measure of what your team success is, but is not a good indication on how the pitcher truly pitched.

by matthan on Jul 24, 2009 6:49 PM EDT up reply actions

...Obviously.

Brad Ziegler had a scoreless inning streak. Brad Ziegler had not met BJ Upton.

by P Brady on Jul 24, 2009 6:50 PM EDT up reply actions

Right, although this wasn't meant to determine a players true talent level

Rather how many wins a team should have. In that regards wOBA and ERA are better at doing that than WAR

I bet that wasn’t an “obviously”

by matthan on Jul 24, 2009 6:52 PM EDT up reply actions

Not an obviously, but it stands to reason that a metric that combines stats that don’t measure actual performance would lose to a combination of a metric that does measure actual performance and one that uses game data.

by 17843 on Jul 24, 2009 11:49 PM EDT up reply actions

Interesting to see that this shows that an increase of 1 war equals just about 1 real win

by matthan on Jul 24, 2009 6:59 PM EDT reply actions

Which of course suggests a replacement team would win about 46-47 games

by matthan on Jul 24, 2009 7:03 PM EDT up reply actions

Which is correct.

by R.J. Anderson on Jul 24, 2009 7:20 PM EDT up reply actions

Yeah I see Tango calculated it at 47.5

Interesting to see the formula projects that almost perfectly

by matthan on Jul 24, 2009 8:01 PM EDT up reply actions

It's almost like WAR is useful or something.

by R.J. Anderson on Jul 24, 2009 9:20 PM EDT up reply actions

VINDICATION!

Actually, that’s good to hear, settles the question I guess.

by R.J. Anderson on Jul 24, 2009 7:20 PM EDT up reply actions

Also does anyone know the RMSE for the pythag win/loss?

I’m curious to see the fit for that.

by matthan on Jul 24, 2009 6:59 PM EDT reply actions

Glass may have done something on this last ~off-season.

Maybe not though.

by R.J. Anderson on Jul 24, 2009 7:20 PM EDT up reply actions

I just did it for 2002-2008

Pythag is garbage compared to the two formulas listed above

The RMSE for the wOBA and ERA formula is 4.7
The RMSE for the WAR formula 5.8
The RMSE for pythag is 12.6

That isn’t even close.

If you want a better look at how many wins a team should have use those two formulas I found and ignore pythag.

by matthan on Jul 24, 2009 8:24 PM EDT up reply actions

Actually I miscalculated it

I had 2009 in my spreadsheet which totally screwed everything up so check that

by matthan on Jul 24, 2009 8:31 PM EDT up reply actions

What about 3rd order wins?

I can't help that I make some things look easier than they really are.

by Sandy Kazmir on Jul 28, 2009 9:15 AM EDT up reply actions

Yea of course ERA is a good indicator of how many wins a team had.

It is not a good measure of pitcher talent. Its kind of like saying the best metric for estimitating team victories is the aggregate of wins on the pitching staff.

The purpose of FIP is to try to quicky measure a pitchers talent. THe gap between FIP and ERA goes a long way to finding under and overvalued pitchers.

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Jul 24, 2009 9:14 PM EDT reply actions

oh god this could fuel to fire for so many people...

why does this exist, it seems you understand that ERA is luck and wins in any given season can be effected by good luck.

WAR is just as good as it always was, and I’m sorry but I don’t actually buy it is any less valuable than we thought based on this information…

by Navi's_Navy on Jul 24, 2009 10:51 PM EDT reply actions

I wasn't saying it was any less valuable

by matthan on Jul 25, 2009 5:08 PM EDT up reply actions

Could you use tRA and see how it correlates to wins?

by nolesblogger on Jul 25, 2009 11:48 AM EDT reply actions

The way I see this . . .

The team ERA is essentially RunsAllowed / Games so for a regression its the same as using Runs Allowed.

And wOBA as I understand allocates points based on the historical value of each kind of hit and a walk. So wOBA is more or less an estimate for runs scored per plate appearance. I don’t know how much plate appearances vary by team but if it isn’t much wOBA is a good proxy for runs scored, and to the extent that they do it probably lowers predictions for good offensive teams (e.g. Red Sox and Rangers in 2008)

So do wOBA and team ERA do a better job of predicting wins than runs scored and allowed?

by stevetb on Jul 27, 2009 2:49 AM EDT reply actions