Clarifying Regression

by R.J. Anderson on Aug 6, 2009 10:26 PM EDT Comment 87 comments

If the Jason Bartlett thread taught me anything, it's that somehow, someway, people don't understand regression. Having that concept down almost certainly changes the context in which people take the post's message and well, that's my fault for not explaining it better.

Here's what I'm saying:

From this point on (and this point isn't important, it could've been May 5^th, June 19^th, July 30^th, whatever) you would expect Bartlett to hit closer to his expectations than his current rate. This DOES NOT mean he will finish the season with a total line near his expectations*, it simply means from Point X until season's end, we would expect him to hit closer to expected.**

Some people confused this as me saying:

Since Bartlett hit .400 through his first 400 at-bats, and we have him projected at .300, he's going to go 35 for his next 250 to even things out.

That IS NOT regression. That IS the gamblers fallacy. Take a coin and flip it 10 times, if you get seven heads, that doesn't change your expectations of having a 50% chance at a tails flip, does it? It also doesn't mean you'll get seven out of the next 10 flips landing on tails to "correct" the balance.

There are counterpoints to be made, I'm not arguing them, and I'm sure someone will take this as an attack on Bartlett, it's not, but people read what they want. We can replace Bartlett's name with any player who gets off to a slow or hot start and the same statements apply.

---

*Yes, we adjust his expectations with the increasing data amount provided this year.

**And how do we get these expectations? By using historical player data dating back three years and regressing to major league mean - or if you feel daring, by all shortstops aged so and so six feet in height. Marcels doesn't go that in depth, PECOTA does, the difference is rather minimal despite the extra bells and whistles.

0 recs | Comment 87 comments | Share

Story-email Email Printer Print

Comments

inb4peoplewhodidntreadthearticlecomplaining

Brad Ziegler had a scoreless inning streak. Brad Ziegler had not met BJ Upton.

by P Brady on Aug 6, 2009 10:34 PM EDT reply actions 0 recs

this post. the entire spiel about expecting regression.

what does it accomplish? it’s not news. it’s certainly not hard-hitting journalism. it’s not even interesting. how am I better off now hearing for the 300th time that Bartlett is eventually going to regress? the only way this would even come close to being post-worthy is if we were a Boston messageboard and we needed something to blame the recent series sweep on and use Bartlett playing over his head as a scapegoat.

by digs the long ball on Aug 7, 2009 12:25 AM EDT up reply actions 0 recs

Um.

This site is about analysis first and foremost. The “hard-hitting journalism” is left to people who care about covering that aspect of baseball. If someone told you this place is for that, they lied.

If you don’t find it interesting, then you probably won’t find a lot of things here interesting. As it turns you may not have even read the post, otherwise you would understand that some people missed on what the theory meant.

by R.J. Anderson on Aug 7, 2009 12:41 AM EDT up reply actions 0 recs

Whatev.

I do find this place interesting – almost every article is. I just don’t see why anyone wants to hear (yet again) that one of our better performing players is set to regress. Just trying to help people save themselves from heartbreak when the regression occurs? I don’t know. Just seems unecessarily negative and comes off as more annoying than informative.

by digs the long ball on Aug 7, 2009 12:49 AM EDT up reply actions 0 recs

But that's not the point of the article at all

Yeah, Bartlett’s the example, but there’s a deeper principle at play here that many people don’t understand. Maybe you get it, maybe you don’t, but it’s worth posting since it needs to be said.

by antimatter on Aug 7, 2009 1:56 AM EDT up reply actions 0 recs

I find this post less interesting than the article, which is more interesting then the paper I'm failing to write at this time.

In conclusion, I think the only unnecessarily negative thing here is you, Negative Nancy.

Maybe I’m just an old fashioned BJ suck-off-er. (Good call Sandy)

by Lurch's Lobbyists on Aug 7, 2009 2:20 AM EDT up reply actions 0 recs

because people insist he should be getting the most at bats on the team when, in fact, he is not that good

leave him up there for lefties only. As R.J. said, if people understood this then this wouldn’t have had to be written, but he majority of articles this year have been written under the assumption that the reader gets this, which hasn’t necessarily been the case as exhibited by the bartlett thread.

by Navi's_Navy on Aug 7, 2009 4:38 AM EDT up reply actions 0 recs

have you seen his lefty righty splits?

.337 for lefties, .338 for righties. What am I supposed to read into that? Maybe for his career that’s been true, but he’s just having a career year this year. This makes this not the norm. Will he hit this way next year? probably not. there are not a lot of players that hit .330 on a consistent basis.

Right now, he should be getting the most at-bats with how he is performing RIGHT NOW. You ride the hot hand until it gets cold. That’s common logic.

What people get most frustrated with, with this article, is yes, he will regress. EVERYONE knows that. But if he regresses to a .300 hitter, so what? He’s still then a good hitter.

If you want to write a good article on this, write more on how much of a regression can we expect, because in that answer lies how good of a performance we can expect.

by raysfaninminnesota on Aug 7, 2009 9:27 AM EDT up reply actions 0 recs

Some quick math

So far Jaybles has been a .338 hitter by going 105/311. There are 58 games left and if you figure he gets 3.5 AB’s per game you get 189 remaining AB’s, which is perfect to give us a nice round 500 on the season. If he were to regress to his career norm of .286 (I know this isn’t his true talent and contains this year’s high average and isn’t indicative of what he can really do, but we have to use something and this seems about right) over those 189 AB’s you are looking at 54 more hits. Giving him 159/500 on the season, or .318. This is all not to the point that he is sure to hit worse than better going forward, but I thought it would be nice to see what it would look like if it turned into his career self tomorrow.

I can't help that I make some things look easier than they really are.

by Sandy Kazmir on Aug 7, 2009 10:07 AM EDT up reply actions 0 recs

thanks

that makes sense and actually has a point that’s worth making.

by raysfaninminnesota on Aug 7, 2009 10:09 AM EDT up reply actions 0 recs

Bartlett isn't even hitting .400.

It’s called an example because that was the thread most people blew up on.

My heavens.

by R.J. Anderson on Aug 7, 2009 11:21 AM EDT up reply actions 0 recs

Good post, RJ

This really does clarify the idea.

by Thaddeus? on Aug 6, 2009 10:37 PM EDT reply actions 0 recs

This is something that is typically misunderstood...

Glad you clarified.

by tallyray on Aug 6, 2009 10:51 PM EDT reply actions 0 recs

Thanks for clarifying how unbelievably wrong you have been about Bartlett all season long. He will likely finish above .320 and blow your ill-founded theory out of the water.

by alorfi on Aug 6, 2009 11:11 PM EDT reply actions 0 recs

Congratulations, you are officially the biggest idiot to comment on this post~!

Brad Ziegler had a scoreless inning streak. Brad Ziegler had not met BJ Upton.

by P Brady on Aug 6, 2009 11:18 PM EDT up reply actions 0 recs

Ill-founded theory?

I’m sorry, I usually don’t ban for being a dumbass, but if you think I invented the idea of regression in statistics then you deserve to have your membership revoked.

by R.J. Anderson on Aug 6, 2009 11:21 PM EDT up reply actions 0 recs

I have no idea who this guy is, but I think he MIGHT be arguing that regression is not a definitive event

which leads to an interesting idea…

Since regression is an outcome, is there a way to calculate the probability of regression? Does anyone do that?

by plasticman on Aug 6, 2009 11:26 PM EDT up reply actions 0 recs

...So you've never taken statistics?

EVERYONE regresses.

Brad Ziegler had a scoreless inning streak. Brad Ziegler had not met BJ Upton.

by P Brady on Aug 6, 2009 11:31 PM EDT up reply actions 0 recs

What are you talking about?

by plasticman on Aug 6, 2009 11:32 PM EDT up reply actions 0 recs

Read this.

http://www.insidethebook.com/ee/index.php/site/comments/most_people_still_dont_understand_the_concept_of_regression_toward_the_mean/

You expect everyone above/below average to regress.

by R.J. Anderson on Aug 6, 2009 11:32 PM EDT up reply actions 0 recs

I understand regression is expected and likely

I’m interested in HOW likely.

Or is it statistically trivial?

by plasticman on Aug 6, 2009 11:36 PM EDT up reply actions 0 recs

It's not likely, its pretty certain

If you’re talking a timetable, thats a lot trickier.

Brad Ziegler had a scoreless inning streak. Brad Ziegler had not met BJ Upton.

by P Brady on Aug 6, 2009 11:37 PM EDT up reply actions 0 recs

Wait, you think regression is a one-hundred percent definitive outcome?

Then, you my friend, have never taken statistics:)

by plasticman on Aug 6, 2009 11:37 PM EDT up reply actions 0 recs

At SOME point its going to happen

If you’re asking when, thats an entirely different question.

Brad Ziegler had a scoreless inning streak. Brad Ziegler had not met BJ Upton.

by P Brady on Aug 6, 2009 11:38 PM EDT up reply actions 0 recs

Just for the sake of statistics

It is entirely possible that someone does not regress. Likely? No way—but definitely possible. That’s pretty much inherent in any statistical model.

I’m not saying that is going to happen with Barty. I’m just wondering if some sort of statistics are applied to regression. For example: the degree of regression, or what you suggested, a timeframe.

by plasticman on Aug 6, 2009 11:42 PM EDT up reply actions 0 recs

Ok, there's a 99.5% chance of regression

Happy? :P

Brad Ziegler had a scoreless inning streak. Brad Ziegler had not met BJ Upton.

by P Brady on Aug 6, 2009 11:45 PM EDT up reply actions 0 recs

Yes. Very.

by plasticman on Aug 6, 2009 11:47 PM EDT up reply actions 0 recs

Sure, it's possible that someone or something could never regress...

But kind of in the same way it’s possible that a fair coin could keep on coming up heads forever. The probability isn’t zero but it’s not really useful to talk about it since it’s astronomically unlikely.

A baseball season is short enough that season-long flukes can happen, but they’re still rare. If you’ve gotta make a prediction of likelihood, regression to some sort of mean is always more likely than not (unless there’s information that’s missing).

by antimatter on Aug 7, 2009 2:05 AM EDT up reply actions 0 recs

if it doesn't happen it didn't happen because it didn't have time to occur

it will happen at some point if given an indefinite amount of time. When/where is up to speculation

by Navi's_Navy on Aug 7, 2009 4:39 AM EDT up reply actions 0 recs

Regression

Any player having a career year is going to have some amount of regression. Howvever, when that same player is a speed guy, and you move him up in front of a future HOFer, such as CC, that player is going to see a significantly higher amount of hittable pitchs. The effect of this is, since this guy has good speed and is getting on base more, the player after him is going to usually see more fastballs, allowing the catcher a better opportunity to have a better chance to throw out the potential basestealer…another effect of this is, it generally will cause the pitcher to pitch from the stretch, which reduces the velocity of the pitchs, which enables the hitter (Crawford) a better opportunity to make contatct and put the ball in play…and should have the same flow for the immediate following hitters(unless they are hitting under .230 and on a pace to strikeout 130-200 times).

by Blue or CONKZILLA on Aug 7, 2009 4:44 AM EDT up reply actions 0 recs

This is Stop Foley from MLB.com threads

Am i correct and or Alorfi from Rays Republic

Duemig is alive, NOT FOR LONG though....

by Some other guy who does not care on Aug 7, 2009 2:28 PM EDT up reply actions 0 recs

Have Bartlett's expectations changed?

I asked this in another thread, but has what we expect Bartlett to regress to changed any?

Bartlett’s batting average peaked on May 18th at .384. Since that day he’s hit .301 over his next 192 PAs. That works out pretty nicely.

However, his OPS peaked at 1.017 on June 18th. Since then he’s had an OPS of .794. Again, it works out pretty nicely and shows that he has regressed much closer to his current performance.

That said, an .800 OPS from Bartlett is about 100 points higher than was expected. At what point do we admit improvement?

by tallyray on Aug 7, 2009 9:32 AM EDT reply actions 0 recs

same logic as applied to Edwin Jackson and Jeff Niemann

the minute someone improves, it’s automatically assumed there’s regression. I’m surprised there hasn’t been much said about Zobrist and regression. He also is having a peak year without a prior one to go up against. But again, even if there is a regression, the key is how much regression. Because at this point, Bartlett has room to work with and still be a very good player.

by raysfaninminnesota on Aug 7, 2009 9:39 AM EDT up reply actions 0 recs

No one's ever said Bartlett wasn't a good player.

by Suttree on Aug 7, 2009 9:56 AM EDT up reply actions 0 recs

really?

because people insist he should be getting the most at bats on the team when, in fact, he is not that good leave him up there for lefties only. As R.J. said, if people understood this then this wouldn’t have had to be written, but he majority of articles this year have been written under the assumption that the reader gets this, which hasn’t necessarily been the case as exhibited by the bartlett thread.

by Navi’s_Navy on Aug 7, 2009 4:38 AM EDT up reply actions 0 recs

by raysfaninminnesota on Aug 7, 2009 9:59 AM EDT up reply actions 0 recs

then what's the point of this article?

a guy hits .330. someone says he will regress. How is that worth the time it took to write it?

So what if he regresses? The point would be to examine how much of a regression to expect. That would be a good article.

by raysfaninminnesota on Aug 7, 2009 10:01 AM EDT up reply actions 0 recs

jesus h. christ.

the article had next to nothing to do with bartlett. it was simply clarifying the idea of regression for people who haven’t taken the time to at least learn it, which is exactly what happened the last time a barty/lead off thread happened.

by Suttree on Aug 7, 2009 10:02 AM EDT up reply actions 0 recs

well then people are idiots

saying there’s regression is calling the sky blue. I don’t get the point. Now if you want to try and qualitate regression, then fine.

A player regresses. Regressing doesn’t mean he’ll all of a sudden hit crappy to fall in line with his normal average, but occurs from here on.

The complaint is, so what? What is the point?

by raysfaninminnesota on Aug 7, 2009 10:12 AM EDT up reply actions 0 recs

THIS MEANT FOR IDIOTS

UNTWIST YOUR PANTIES

by Suttree on Aug 7, 2009 10:16 AM EDT up reply actions 0 recs

was meant

i think i’m developing a brain tumor

by Suttree on Aug 7, 2009 10:17 AM EDT up reply actions 0 recs

good luck with that

no, i do understand who it was intended for and it’s sad it had to be written in the first place. I think part of the reason people got so frustrated was because they didn’t understand why the first article was written.

so there’s regression…what exactly was the point being made in the first place? And just in case, I’m going to go back and read the first article…maybe I’m developing a brain tumor too and just didn’t get it.

by raysfaninminnesota on Aug 7, 2009 10:19 AM EDT up reply actions 0 recs

go back and read the other article.

by Suttree on Aug 7, 2009 10:21 AM EDT up reply actions 0 recs

Well I'm not on board with that statement.

Players will regress, and players playing way above their head will regress more quickly and sharply than others. I’m not arguing that. My point is simply when do we admit an incremental improvement. Bartlett didn’t become a .330/.375/.475 player, but he’s looking a lot more like a .300/.345/.450 type player than he is looking like his career norms.

by tallyray on Aug 7, 2009 9:59 AM EDT up reply actions 0 recs

Compare Bartlett's pre-season ZiPS to his rest-of-season ZiPS over at Fangraphs.

That’ll show you how much half a season matter. I believe his projected wOBA has gone up from about .330 to .340.

Beyond the Boxscore Not a member? Sign up.

by Sky Kalkman on Aug 7, 2009 9:40 AM EDT up reply actions 0 recs

But when we compare his 2010 ZIPS to his 2009 ZIPS will we see a bigger change?

And I don’t think you’re arguing that ZiPS quantifies everything perfectly.

by tallyray on Aug 7, 2009 9:57 AM EDT up reply actions 0 recs

thanks

What do we truly know about Bartlett? He doesn’t strike out a lot. He is around a 50 walk player. He can steal 20-30 bases. He plays good defense. .300 hitter? That really is the question. He’s done it before (once) and in the following season, in his first full season, he took a step back (something a lot of players do in their 2nd season and first full). He was also battling a hamstring injury I believe, which limited his effectiveness.

In first season with Rays, he batted .286, but his walks were down. This year he’s batting out of his mind, but even better are the rest of his stats. Good walks, good steals, good defense.

When healthy, Bartlett can be a good hitter with maybe some gap power. Add that with his speed and walks, and by golly, you have a good leadoff hitter.

Regression or no regression, Bartlett can sustain most of what he’s doing.

by raysfaninminnesota on Aug 7, 2009 9:58 AM EDT up reply actions 0 recs

No, he can't.

He can sustain being a 290ish hitter.

He can’t sustain a 390 BABIP.

by Suttree on Aug 7, 2009 10:00 AM EDT up reply actions 0 recs

Even if he does end the year with a 380-390 BABIP, he'll go back to being Royce Clayton next year.

by Suttree on Aug 7, 2009 10:00 AM EDT up reply actions 0 recs

which part can't he sustain that I mentioned?

by raysfaninminnesota on Aug 7, 2009 10:02 AM EDT up reply actions 0 recs

Sorry, you're right on this one.

I misread your post.

There’s a 5% chance Barty is wearing a Rays uniform next year.

by Suttree on Aug 7, 2009 10:04 AM EDT up reply actions 0 recs

.286 on his career

I can't help that I make some things look easier than they really are.

by Sandy Kazmir on Aug 7, 2009 10:11 AM EDT up reply actions 0 recs

he can't sustain that?

by raysfaninminnesota on Aug 7, 2009 10:14 AM EDT up reply actions 0 recs

I misread, I thought Sut said he can't sustain being .290ish

I can't help that I make some things look easier than they really are.

by Sandy Kazmir on Aug 7, 2009 10:16 AM EDT up reply actions 0 recs

286 with few walks is not particularly helpful

by Suttree on Aug 7, 2009 10:16 AM EDT up reply actions 0 recs

how many is few walks?

by raysfaninminnesota on Aug 7, 2009 10:17 AM EDT up reply actions 0 recs

ZIPS is projecting him to finish with 39.

That’s solid enough for a 315 hitter, which is probably where he’ll finish the season.

by Suttree on Aug 7, 2009 10:21 AM EDT up reply actions 0 recs

That's pretty good for a number 9 hitter

UH-OH HE WENT THERE

I can't help that I make some things look easier than they really are.

by Sandy Kazmir on Aug 7, 2009 10:19 AM EDT up reply actions 0 recs

nice

If between walks and batting average he can be around the .350-.370 range, that’s not bad for a leadoff hitter. But it does make for a very nice number 9 hitter.

ESPN has him on pace for 44…but I’d rather trust ZIPS than ESPN. Either way, he does have a 50 walk season under his belt. I see no reason why he can’t come awfully close to that on a consistent basis.

by raysfaninminnesota on Aug 7, 2009 10:27 AM EDT up reply actions 0 recs

In the 50 walk season, he hit 265.

I’m inclined to believe he was taking pitches and hoping to get walked.

by Suttree on Aug 7, 2009 10:27 AM EDT up reply actions 0 recs

His SO/L that year was the lowest of his career...

by tallyray on Aug 7, 2009 10:38 AM EDT up reply actions 0 recs

Our other leadoff hitter has proven to be in the .380+ range.

And here…we….GO.

So long, Sweet Lime!

by PlayOnWords on Aug 7, 2009 10:31 AM EDT up reply actions 0 recs

Well played

I think BJ should leadoff because he takes long strides, he can’t help that he makes some things look easier than they really are.

I can't help that I make some things look easier than they really are.

by Sandy Kazmir on Aug 7, 2009 10:37 AM EDT up reply actions 0 recs

His walks aren't good and his defense is declining.

He’s a 30-year-old slap hitting shortstop.

God.

by R.J. Anderson on Aug 7, 2009 11:23 AM EDT up reply actions 0 recs

I can't wait till we trade him for a middle reliever.

by Suttree on Aug 7, 2009 11:31 AM EDT up reply actions 0 recs

Hopefully half the other GMs think he's Jeter and deal us a boatload for him.

by R.J. Anderson on Aug 7, 2009 11:34 AM EDT up reply actions 0 recs

The number of dumb GMs grows less and less everyday.

But, there are a still a handful we could put a good deal together with.

by Suttree on Aug 7, 2009 11:41 AM EDT up reply actions 0 recs

I'm sure some don't understand regression.

by R.J. Anderson on Aug 7, 2009 11:49 AM EDT up reply actions 0 recs

Do you think Bartlett would draw a far greater package than Reid?

This is a question I have been struggling with. Not who I value, but what the market would return.

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Aug 7, 2009 11:50 AM EDT up reply actions 0 recs

Bartlett.

People pay for proven goods.

by Suttree on Aug 7, 2009 11:53 AM EDT up reply actions 0 recs

I'm not sure.

Take Seattle. We know they want(ed) Brignac because he fits their park, is cheap, and potentially available. But they’re also a smart team. Same with Pittsburgh (same three factors as well) but for a team like say….Kansas City, I’m certain they’d take Bartlett for all the non-performance reasons.

by R.J. Anderson on Aug 7, 2009 11:53 AM EDT up reply actions 0 recs

Well, Kansas City could use him for performance reasons, as well.

Overrated or not, he is a solid defensive shortstop who’s bat will be above average for the position.

Also, he isn’t Yunesky Betancourt.

by Suttree on Aug 7, 2009 11:58 AM EDT up reply actions 0 recs

So the best solution is to test the market with each b/c it isn't quite clear

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Aug 7, 2009 12:04 PM EDT up reply actions 0 recs

Who's the better player over the next two years?

Brignac or Bartlett?

by tallyray on Aug 7, 2009 12:33 PM EDT up reply actions 0 recs

Not all teams will base the trade decision based on 2 years

Cost control and defensive decline also play into the decision. Given a two year choice with no salary, cost control or opportunity cost cosdierations, I’d take Bartlett.

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Aug 7, 2009 1:01 PM EDT up reply actions 0 recs

Ok... Factor those into it..

Who would you rather have at SS? Bartlett or Brignac?

Of course the Rays save a couple million if they give the job to Brignac, but the cost of that is uncertain. Brignac is still an uncertain commodity where Bartlett has a more stable value (if you consider his career numbers and discount this year).

by tallyray on Aug 7, 2009 3:00 PM EDT up reply actions 0 recs

Can't make that decision without knowing what each one could bring in for loot

I do think Briggy is a defensive upgrade and a big offensive downgrade vs LHP.

Follow Me on Twitter @FreeZorilla

by FreeZorilla on Aug 7, 2009 3:07 PM EDT up reply actions 0 recs

Haha. We'll have these same arguments until the end of time re: regression.

But I think the onus is on the pro-Barty peeps to provide comparables, here. Name another SS who suddenly hit 30% LD at age 30 and sustained that rate into the future.

So long, Sweet Lime!

by PlayOnWords on Aug 7, 2009 9:57 AM EDT reply actions 0 recs

I wouldn't say I'm pro-Bartlett

Even though I think he’s getting criticized for playing so well…

But here are some players who had a career year at 28-30:

Carlos Guillen: went from 104 OPS+ (.331 wOBA) at 27 to 141 OPS+ (.391 wOBA) at 28. Maintained a wOBA from .347-.391 over next four seasons.

Rich Aurillia: went from 102 OPS+ (.339 wOBA) at 28 to 146 OPS+ (.395 wOBA) at 29. Terrible his next three years (wOBA from .297-.320) then rebounded and was decent for a couple more years

Mark Grudzielanek: went from 78 OPS+ (.301 wOBA) at 28 to 110 OPS+ (.357 wOBA) at 29. Reverted back to career norms afterwards.

Basically only Guillen sustained any success. Guillen was 6’1 180 lbs and basically a slap hitter. His ISO was basically .100 – .140 for his first four seasons then jumped to .220. Bartlett’s ISO was between .075-.095 before jumping to .189 this year.

by tallyray on Aug 7, 2009 12:49 PM EDT up reply actions 0 recs

All of those guys were moved off short by the time they hit 31 right?

I can't help that I make some things look easier than they really are.

by Sandy Kazmir on Aug 7, 2009 12:51 PM EDT up reply actions 0 recs

Does that matter?

by tallyray on Aug 7, 2009 2:49 PM EDT up reply actions 0 recs

Considering you get about a win and a half added to your WAR just for playing there

yeah it’s kind of a big deal

I can't help that I make some things look easier than they really are.

by Sandy Kazmir on Aug 7, 2009 3:17 PM EDT up reply actions 0 recs

Does it matter in the sense of this discussion?

PlayOnWords was asking for offensive comps near Bartlett’s age. Whether the moved off SS isn’t really relevant to that discussion unless we’re assuming that increased offense means Bartlett can’t play SS which I don’t think we’re assuming.

by tallyray on Aug 7, 2009 3:37 PM EDT up reply actions 0 recs