Projection Systems

by R.J. Anderson on Mar 9, 2009 3:00 PM EDT

One of the more common questions asked of me is in regards to projection systems. Which should people "trust", how they should be weighed, and is PECOTA worth paying for. In reverse order, let me say that PECOTA is excellent. It's quite impressive and if you have a Baseball Prospectus subscription for the content already, then you really cannot go wrong using PECOTA. With that being said, paying simply to access the weighted means spreadsheet is hardly necessary to get an appropriate read on projections.

For instance, say you wanted to know what the systems are saying about Pat Burrell. Well, you could purchase a BP subscription and figure his wOBA the longhand way (1.75*OBP+SLG/3) and get .364 as your answer. Or, you could go to FanGraphs' player page for Burrell and have access to CHONE, ZiPS, and Marcels projections which say: .355, .369, and .377. Average those three and you get ... drum roll ... .367. Even if PECOTA is more accurate in this case, we're talking a fairy insignificant difference.

If you enjoy being a slave to information, you could always throw the PECOTA projection in as well and get a four system average, but it's not necessary if you don't already have access to PECOTA. As you'll see below, I've taken the means to gather the projections and average them out between the four systems and the three free systems.

Last	PECOTA	CHONE	Marcel	ZiPS	4S Avg	Free Avg
Pena	0.368	0.382	0.383	0.388	0.38	0.384
Burrell	0.364	0.355	0.369	0.377	0.366	0.367
Longoria	0.36	0.36	0.366	0.365	0.363	0.364
Upton	0.355	0.372	0.358	0.367	0.363	0.366
Gross	0.354	0.336	0.335	0.337	0.341	0.336
Kapler	0.34	0.323	0.337	0.334	0.334	0.331
Aybar	0.332	0.336	0.334	0.317	0.333	0.329
Crawford	0.324	0.357	0.335	0.347	0.341	0.346
Iwamura	0.323	0.325	0.33	0.324	0.326	0.326
Ruggiano	0.319	0.333	0.312	0.326	0.323	0.324
Navarro	0.318	0.321	0.319	0.312	0.318	0.317
Zobrist	0.318	0.33	0.309	0.323	0.32	0.321
Joyce	0.312	0.318	0.354	0.329	0.328	0.334
Bartlett	0.296	0.318	0.315	0.315	0.311	0.316
Perez	0.295	0.312	0.341	0.288	0.309	0.314
Riggans	0.288	0.298	0.312	0.292	0.298	0.301

I'm sure you've noticed the numbers aren't entirely different - the average difference is like 0.001 points -- and it's obvious why; we only took one of the four parts out of the equation and PECOTA is pretty much in line with what the other systems are forecasting.

Another extremely important thing to note is that league averages for each of these systems are unique. If you go to FanGraphs and see that two projection systems peg a player for a .330 wOBA and then glance to the wRAA column, odds are the two numbers will vary. Perhaps not be by much, mind you, but some (Marcels especially) depress offense and others (Bill James, which, incidentally, is why I don't use them) inflate offensive output drastically.

Okay, that pretty much answers the questions, but let's go a bit further here. Each forecasting system has some drawbacks. Try and use these to get a ballpark feel for how a player is expected to perform, but keep in mind there's a good chance that said player will exceed or fall below the projected line. Again, this is mostly common sense stuff, but sometimes it needs to be stated. If Burrell has a wOBA of .350 someone will say "But PECOTA said..." and that's simply a misinterpretation what these projections are supposed to be used for.

As for pitching stats, you can easily do the same. I know RGlass has a PECOTA sheet with FIP numbers thrown in, perhaps he can share those.

Tweet Comment 25 comments | 0 recs |

Do you like this story?

Comments

Bill Pecota and Chone Figgins should have never made that projection system

- Joe Morgan

by GomesSweetGomes on Mar 9, 2009 3:09 PM EDT reply actions

I have both FIP and real wOBA figured.

The one issue with PECOTA is all the minor leaguers make doing a BRAA or real FIP tricky. So I just use generics to get them (3.2 and .320? I believe). I don’t have access to them at work, nor do I have a computer at home, but I’ll either email them to RJ from a friends’ comp or just make a post.

by rglass44 on Mar 9, 2009 3:14 PM EDT reply actions

By real wOBA I mean I actually did the chain-weighted values as opposed to the formula RJ put above.

by rglass44 on Mar 9, 2009 3:15 PM EDT up reply actions

I have nothing to add to this discussion

But wanted an excuse to post this:

by GomesSweetGomes on Mar 9, 2009 3:39 PM EDT reply actions

Gnomes, Domes and Gomes

They’ll sell you a trip to the Cactus League over the internet,
It’ll block out the sun and keep grass off your field,
He’ll bruise your ribs and strike out 200 times in a full season.

by Thejeffg on Mar 9, 2009 4:02 PM EDT up reply actions

Well Done

by GomesSweetGomes on Mar 9, 2009 4:03 PM EDT up reply actions

FWIW

Based on reading past evaluations of different prjoection systems, I believe PECOTA generally beats all the others on pitchers but is about interchangeable as far as position players go.

by rglass44 on Mar 9, 2009 3:48 PM EDT reply actions

See, I wasn't sure about that either.

CHONE is the best at pitching I thought, but maybe not.

by R.J. Anderson on Mar 9, 2009 3:58 PM EDT up reply actions

This is what I'm going off of:

http://www.hardballtimes.com/main/article/so-how-did-tht-projections-do/

Let us now turn our attention to pitchers. The process I followed for pitchers was much the same as what I did for hitters, except instead of OPS I used ERA, and my minimum baseline was 50 innings pitched in 2008. Let’s start off with the overall results:

PECOTA, .620
THT, .629
CHONE, .665
ZiPS, .676

PECOTA has always been known for being particularly good with pitchers, and this year was no exception, as it ever-so-slightly beat out THT for first place. Something crazy, however, happens if we look only at the 55 (roughly 20 percent of the whole sample) pitchers on whom the systems disagreed the most:

PECOTA, .786
ZiPS, .849
THT, .888
CHONE, .905

PECOTA really shines here, and I’m going to examine these results in more details to see just where our system can be improved. Jonathan Papelbon, by the way, was the pitcher on which the systems disagreed most, with his projections ranging from very good (THT) to otherworldly (ZiPS).

by rglass44 on Mar 9, 2009 4:16 PM EDT up reply actions

Ah. Gotcha.

by R.J. Anderson on Mar 9, 2009 4:23 PM EDT up reply actions

Where can I find what the league average for wOBA is?

Are position/era/league/park all factored in?

by Top Gun Numba 1 on Mar 9, 2009 3:50 PM EDT reply actions

Cameron or Appelman posted them a while back, let me do some scanning.

by R.J. Anderson on Mar 9, 2009 4:03 PM EDT up reply actions

Found it.

The baseline wOBA for each system are:

CHONE (for players who have appeared in the majors) – .327
Bill James – .346
Marcel – .332

http://www.fangraphs.com/blogs/index.php/the-projections

I don’t have the ZiPS baseline yet, but I’m guessing it’s around .330 — same with PECOTA.

by R.J. Anderson on Mar 9, 2009 4:08 PM EDT up reply actions

Is wOBA normalized for park/era/league/position?

by Top Gun Numba 1 on Mar 9, 2009 4:16 PM EDT up reply actions

no

It is a straight chain-weighted average.

by rglass44 on Mar 9, 2009 4:17 PM EDT up reply actions

wOBA is not.

However the “Value” section of FanGraphs does include a park/league adjusted version of wRAA — wOBA converted to runs and compared to league average.

by R.J. Anderson on Mar 9, 2009 4:24 PM EDT up reply actions

Thanks.

Btw, ’grats on this: http://www.techcrunch.com/2008/10/29/former-aol-exec-raises-funding-for-sports-blogs-network/

I hope ya’ll are getting paid.

by Top Gun Numba 1 on Mar 9, 2009 4:26 PM EDT up reply actions

Not very much.

by R.J. Anderson on Mar 9, 2009 4:28 PM EDT up reply actions

FIX SOME MOUFS

by Top Gun Numba 1 on Mar 9, 2009 4:33 PM EDT up reply actions

Fun jobs don't pay well

Become a lawyer.

Just kidding.

Vogt early, Vogt often.

by Brickhaus on Mar 9, 2009 8:55 PM EDT up reply actions

I was always under the impression

that the place where there’s a material difference (for hitters) is when the player has less than two seasons of major league statistics to back them up.

Vogt early, Vogt often.

by Brickhaus on Mar 9, 2009 4:23 PM EDT reply actions

A note about projections.

Be it financial or baseball stats, projections are pretty much guaranteed to be wrong. They’re really just supposed to give you an idea of the ballpark a player’s numbers should end up in.

by SQUAD on Mar 9, 2009 11:10 PM EDT reply actions