There Actually Are Good Umpires

Let he who is without sin in cursing at an umpire cast the first stone. Yea, that is what I thought. After all, the cries for robot umps still echo around these hallowed halls throughout the off-season and well into the night after any game that Angel Hernandez or Joe West works. Joe West himself has generated 176 different user comments on this site throughout its history including one of him practicing his ejection calls naked in front of a mirror (get that image out of your head). Needless to say, umpires are often a point of contention with one or both fanbases in any contest.

One of the main criticisms with umpires is a perceived lack of accountability as they continue to have inconsistent strike zones and blow obvious check-swing calls in crucial moments mostly because umpires do not have to explain their calls to the media in the post-game. If umpires do speak to the media after the game, the crew chief usually does the talking and at times those conversations can be more frustrating than the controversial call in the game.

Even more frustrating for fans is the fact the technology is in place to stop obvious mistakes such as this beauty last season but instead, we are left with umpires adamantly defending their errors and then taking pleasure in ejecting those that call them out on it.

The fact is, baseball does have a system in place for accountability, but not one that is transparent to the average fan or the growing number of fans that want to quantify what they perceive as good or bad performance. Thanks to the fine folks at BrooksBaseball, fans now have a way to begin to do their own look into umpire performance behind the plate in how they call balls and strikes.

Yesterday, Dan Brooks and Harry Pavlidis posted a piece at The Hardball Times which explained their new umpire statistics that they have published in the same popular format as the player cards on their site. The first reaction one might have with access to this data would be to go look for the worst performers and make a mockery of them but Brooks is quick to warn against that.

But at least in the public domain, there’s been little systematic survey of umpiring. There are several reasons. First, because of the way people would naturally use the data without proper instruction, it would create unnecessary controversy. No one wants to be at the center of a media scandal involving umpiring. And so, consider this your proper instruction: If you use these data to rip umpires, consider yourself an idiot.

If Luke Skywalker could still see good in the Dark Lord of the Sith, then certainly we can find good in umpires using the umpire player cards. The player cards contain different categories that can be displayed in both their raw totals as well as percentages:

via www.hardballtimes.com

The quickest way to simplify your view to this card is to focus on the d' and c areas of the card. The d' represents discriminability on how well an umpire performs and the larger that number is, the better while c shows bias toward hitters (positive numbers) or pitchers (negative numbers). The ideal umpire card would show high numbers in the d' and many zeroes in the c column.

The umpire cards are able to show overall figures as well as splits by pitcher handedness, batter handedness, and even down to the granular level of ball and strike counts. For the purpose of this exercise where we are accentuating the positive, the data will be for the high-level data set of all pitches to both left and right-handed hitters by all pitchers.

There are umpire cards for 109 different umpires available for everyone to peek through individually but the burning question remains - who is the best of the best for each pitch type?

Yes, even Angel Hernandez does a few things well but not as well as Fieldin Culbreth who shows up in four of the six pitches listed above.

How can this data be used? The easy way would be to run to the card and look at the cards but understand, these guys are all pretty damn good at what they do. The highest d' score for sliders was Doug Eddings at 2.457 but the 107th umpire, who shall rename anonymous, still scored at 2.363. For fastballs, the top score was 2.19 while the 107th best score was still 2.07.It gets to a certain point where we are splitting hairs in looking at some numbers, but this public data at least gives us some kind of gauge at how much better or worse some umpires are at calling balls and strikes than others.

A more productive use of the data would be to hope for a convergence of certain pitchers lining up with certain umpires. Since Andy Fletcher is one of the best umpires in calling balls and strikes on curveballs and changeups, it would be ideal for the Rays if he were behind the plate when James Shields or Jeremy Hellickson were on the mound. As luck would have it, Fletcher called balls and strikes in Hellickson's second career start in which he threw seven innings of shutout baseball against the Tigers. Fletcher has been behind the plate for Shields three different times and the numbers are quite good.

Go enjoy the umpire cards and dive into the data as it is a fascinating data set that you can easily find yourself lost in as you review percentages and heat maps for everyone. Also heed the final words advice from Dr. Brooks:

Be an educated, informed fan who recognizes that ripping an umpire because he missed a call is often a selfish act with little justification. Often, looking at the bigger pattern of data can give you more answers than a single pitch.