Navigation: Jump to content areas:


Pro Quality. Fan Perspective.
Login-facebook
Around SBN: Trent Richardson Interviews Fellow Brown Brandon Weeden

Bullpens are Useless, Pt. 1

Troy Percival's windup drove me crazy. He had a fast, high leg kick and then seemed to tumble down the mound like Chris Farley on a coffee table. The windup seemed so quick, so erratic, so nearly unrepeatable, that I was surprised pitches ever made it to their destination (and near the end of his career, they typically didn't).

Sadly, in a phenomenal career, marked by a 3.17 ERA and a 3.87 FIP, Percival's time with the Rays featured a ~6.00 FIP and scores of fans in the hospital with cardiac and pulmonary issues. His bombastically bad tenure in Tampa -- and the team's outstanding results in spite of it -- has made me suspicious of bullpens and their usefulness.

Also, as a part-time fan and interested follow of the Chicago Cubs, I must yearly endure off-season signings most painful: Average relievers given above-market deals. Meanwhile, Andrew Friedman continues to cobble together useful bullpens like a Junkyard Wars episode. So, in effect, it seems: Bullpens don't matter.

If Friedman makes his bullpen -- year after year -- on the cheap, and Jim Hendry makes his bullpen -- year after year -- on a Hollywood budget (complete with Michael Bay explosions), then why bother investing in bullpens!? The Rays made it to the World Series with a 6.00-FIP closer!

So, let us endeavor, together, to discover the true value of bullpens.

Star-divide

In this first act, let's look at the last four years of bullpennery. The first question we need to ask is simply: Is there any correlation between a bullpen's results and a team's record?

To explore this question, I have created an interactive chart below. Go ahead, scroll down the page and take a look at it. Go on.

(The numbers next to the data points are win totals for that team.)

Observe the sliders in the middle -- these are filters we can use to change the data set. By moving the 'Pen ERA slider, we can look at the win-loss record of just team with, say, ERAs ranging from 4.09 (just above league average for this period of 'pens) to 6.16.

If we play around with the W-L% slider, then we can constrain our field to just winning teams (.501 to .636 W-L%). 

Toy around with these functions, and then continue reading for some analysis.

NOTE: For our purposes, ERA does a sufficient job. We are not necessarily looking for the most talented bullpens, but rather how a bullpen's results affect a team's W-L record. Runs allowed (RA/9) is probably an even better measurement than earned runs allowed, but for now, just shut up.

ALSO NOTE: I've included the FIP to ERA chart on the bottom to reassert we are not examining bullpen talent, per se, but bullpen results.

Observations:

  • I expected the red regression line would be as flat as Hak-Ju Lee's chest, meaning bullpens have zero affect on a team's success. I was, clearly, wrong. No matter how many regressions I ran, the result were similar: Bullpens are not useless.
  • If we constrain our field to just winning teams, the regression line becomes much flatter. Smaller ERAs have increasly less affect on the team's success. This could mean a variety of things...
  • Such as: There are diminishing returns on elite bullpens. This coincides with the results of my polynomial regressions, where the net results of improved ERAs nearly flat-lined after a 3.50 ERA.
  • Math talk, science words, blah blah blah, regressionary co-linear, but multiple causality? Yes, but only in quantum reversal kittens.
  • If we constrain ourselves to look at teams with only below-average ERAs (i.e. >4.08), then we see: (a) lots of losing team, but (b) a few very good teams (including two 97-win teams and two 94-win teams).
The next step in this analysis: Find the direction of causality. When I ran these regressions with FIP instead of ERA, the regression was more linear. This may imply good teams have good relief ERAs because they also (a) face weaker competition, (b) have better defenders, or (c) simply utilize their relievers more affectively (as in according to leverage instead of traditional roles).

What other observations did you idiots make?

Comment 33 comments  |  0 recs  | 

Do you like this story?

Around SB Nation

2009 Fantasy Baseball Sleepers

Mar 2009 from Fake Teams - 2 comments

Monday's Frosty Mug

May 2009 from Brew Crew Ball - 14 comments

Wednesday Lunch Thread

Jun 2009 from Halos Heaven - 12 comments

Comments

Display:

i think a better analysis would be looking at teams' starting pitcher's innings vs wins. High correlation.

less bullpen innings mean better pitchers going longer in games that are at least close if not winning, plus wel—rested relievers. A crappy bullpen ERA wouldn’t mean much if they only go 2-3 innings per game. I think this is teh key to our season this year, btw (outside of Longo’s health, etc) – how many innings can our starters give us…

by SamRay on Feb 18, 2011 10:53 AM EST reply actions  

Something to keep in mind about the Troy Percival Experience

is that he was not terrible his first few months here. He was actually pretty good. And it was those first months that established us not only as a contender, but built us a cushion for when he…um…less-than-stellar. He it’s not like Percy was consistently awful while he was here.

by nomoredevil on Feb 18, 2011 11:02 AM EST reply actions  

Yeah, that's what I was thinking too... Until I actually looked up his numbers.

Month——-IP—-FIP—-ERA
Mar/Apr—-9.0—-1.47—-0.00
May———12.1—-5.08—-5.11
Jun———6.2—-5.53—-5.40
Jul———4.2—-6.77—-3.86
Aug———6.1—-8.50—-4.26
Sept/Oct—-6.2—-10.48—-9.45

Like FreeZo says, it was one month only — and not just in 2008, but 1 good month for the entirety of his Rays tenure.

A DRaysBay writer from Cubs Stats and Twitter @BradleyWoodrum

by BWoodrum on Feb 18, 2011 11:12 AM EST up reply actions  

one month only

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Feb 18, 2011 11:07 AM EST up reply actions  

I loved the tone of this

-What other observations did you idiots make?
-Runs allowed (RA/9) is probably an even better measurement than earned runs allowed, but for now, just shut up.

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Feb 18, 2011 11:08 AM EST reply actions  

That's Woody for you, taking out his latent aggression on the readers.

I’m kinda scared for the quantum reversal kittens.

I love Casey Fossum. Now try and take me seriously.

by Steve Slowinski on Feb 18, 2011 11:11 AM EST up reply actions  

Oh, it's great...it definitely made me laugh.

And we coddle our readers here too much anyway.

I love Casey Fossum. Now try and take me seriously.

by Steve Slowinski on Feb 18, 2011 11:23 AM EST up reply actions  

...
I want to love you, ZoRilla.
You haven’t even begun to peak. And when you peak, you’re gonna peak so hard all over us.

by Suttree on May 4, 2009 3:20 PM EDT actions

I love Casey Fossum. Now try and take me seriously.

by Steve Slowinski on Feb 18, 2011 11:23 AM EST up reply actions  

And so basically, teams have succeeded with any sort of quality bullpen.

It’s just that if you have a sucky bullpen, you’re more likely to have a worse record than a good one. And you don’t want your bullpen’s ERA to be over 4.50, as that seems to be the edge of the wasteland.

I love Casey Fossum. Now try and take me seriously.

by Steve Slowinski on Feb 18, 2011 11:27 AM EST reply actions  

That all makes a lot of sense.

I love Casey Fossum. Now try and take me seriously.

by Steve Slowinski on Feb 18, 2011 11:27 AM EST up reply actions  

My polynomial regression -- if it was indeed trustworthy -- seemed to indicate

that there are diminishing returns both directions. In other words, a bad pen can hurt you only so much, and a good pen can help you only so much.

Still, my regressions at this point have been simplistic and therefore limited in their application.

A DRaysBay writer from Cubs Stats and Twitter @BradleyWoodrum

by BWoodrum on Feb 18, 2011 11:50 AM EST up reply actions  

I suspect there are some big selection biases in the data.

In that winning teams are more willing to spend more on bullpen pieces, while losing teams are less likely to invest in bullpens in favor of spending towards the future.

So it may not be that a bad bullpen can’t hurt you that much, but instead that more already bad teams have bad bullpens. And instead of a good bullpen can’t help you that much, it’s that winning teams tend to be the ones shoring up their bullpens as the marginal win is more valuable for them.

In my theory, teams would basically be separated into three groups. Non-contenders with bad bullpens, contenders with good bullpens, and some weird middle teams.

by ThePanda on Feb 18, 2011 12:01 PM EST reply actions  

I thought about this too, and attempted to look into the bullpen salary figures to no avail.

Any idea how I could get around this problem?

Also, we must keep in mind that — during the off-season — something like 75% of organizations see themselves as competitive (in my estimation, at least). Otherwise, we wouldn’t see the Mariners or the Mets or the Royals signing (what they believed to be) impact relievers these past few years — which they have, IIRC.

A DRaysBay writer from Cubs Stats and Twitter @BradleyWoodrum

by BWoodrum on Feb 18, 2011 12:16 PM EST up reply actions  

How many teams would willingly punt a season?

We thought the Rays would do it this year, but they’ve made (justifiable) moves to remain in the playoff hunt.

by Suttree on Feb 18, 2011 2:47 PM EST up reply actions  

Have you thrown in starters' ERA/FIP as a variable?

I would guess that teams with crap starters have crap bullpens since relievers are often failed starters. If they are bringing up good pitching talent, then it is more likely their bullpen is filled with middle of the road guys instead of garbage.

It may make the coefficient on bullpen ERA approach zero.

I sort of feel like a simplistic Wins = x * Starters R/9 + y * Bullpen R/9 + z * Offensive R/9 may yield a y that is statistically indistinguishable from zero.

by david_a on Feb 18, 2011 12:09 PM EST reply actions  

(See the first comment.)

No, I haven’t thrown in starters yet. It’s part 2. But good thinking, though!

A DRaysBay writer from Cubs Stats and Twitter @BradleyWoodrum

by BWoodrum on Feb 18, 2011 12:17 PM EST up reply actions  

Comments For This Post Are Closed


User Tools

Founded in 2005, DRaysBay is home to, "Progressive statistical analysis and reasoned argument."

Please read our Community Guidelines.

FanPosts

Community blog posts and discussion.

Recommended FanPosts

Spts-shelton_small
Which Red Sox Fan Are You?
Spts-shelton_small
The Opportunity of the Century
Ryu_small
This one goes out to all the chicken littles
Small
MLB needs to help bring All-Star game to Bay Area
Small
Who's That in My Old Seats?

Recent FanPosts

Untitled4_small
The Rays Should 'Retaliate' By Winning
Whelk_small
Rays 7, Sox 4; Immediate Reactions
Images_small
Post-Game (Over)Reactions here
Small
Proposal: Rays play a series in Durham each year
Ag_-_hector_gomez_-_63_small
Visiting Tampa Bay from Colorado
Images_small
Immediate Reactions: Rays Win, But Likely Lose Another Guy

+ New FanPost All FanPosts >

FanShots

Quick hits of video, photos, quotes, chats, links and lists that you find around the web.

Recent FanShots

Catch For the Ages
Brandon Guyer out for season
Other Side of the Coin: Rodney's Fortune on Called Strikes
Does anyone have any information on Josh Sale? He doubled in a run in his...
Orlando Hudson released
Rays sign Garko to minor league...
Baseball Card Fans...check out my Ebay Auctions
Cameron Seitzer Growing Up Baseball
Longo injury apparently is apparently not good
Hak-Ju Lee, SS, Rays (Double-A Montgomery) Lee had a breakout year in...

+ New FanShot All FanShots >

Friends of the Site

DRB Suggestion Box

Drb4_medium

Yahoo_full_count

Managers

Slowsky__1__small Steve Slowinski

Images_small Jason Collette

Brad_small BWoodrum

Price_small Erik Hahmann

Analysts

Small td32

Rutg_uakjmedjwh9ndzd4lkll_small Imperialism32

100_1952_small MrNegative1

Steak-with-crown_small CBJones

Whelk_small Whelk

Scaled_php_small mr. maniac

Tampa_theatre_small jcmitchell

Me_small John Gregg

Small SGrauer