Shutdowns, Not Saves: The Logic and the Leaders from 2014

Who led the league in saves in 2014? Hopefully, you don’t know the answer off the top of your head. Saves aren’t a good measure of anything relating to player performance or talent and with so many things you could remember about the 2014 season, you probably don’t want to waste vital brain capacity on a random piece of trivia like who had the most saves.

The reason saves aren’t very useful is because the rule itself is not designed to provide much information. You can earn a save if you strikeout Miguel Cabrera, Victor Martinez, and J.D. Martinez in a one run game or you can earn a save if you allow five base runners against the bottom of the Padres’ order. You don’t earn a save if you preserve a tie, or if you preserve a lead in the 7th inning. Nearly everything about the rule is arbitrary, which leads you to find arbitrary results.

But the idea of something like a save is compelling for many people. There is a desire for a statistic that measures the number of a times a reliever comes in and pitches very well in an important spot. We can look at rate stats like ERA, FIP, or xFIP or cumulative numbers like RE24 or WAR, but it’s perfectly fine to want some sort of counting stat that tracks how many times a reliever slammed the door (or didn’t).

Luckily, we carry such a statistic here at FanGraphs. It’s called the Shutdown (SD). The logic is very simple. If a reliever posts a Win Probability Added (WPA) mark of 0.06 or greater in an individual game, they are given a “shutdown.” If their WPA for the game is -0.06 or worse, they are given the opposite, a “meltdown.” If it’s between the two, they are given neither.

This solves one of the key flaws in the save statistic. Relievers earn SD or MD based on the Win Expectancy of their appearance rather than a weird and arbitrary definition, which means that you can earn one without pitching in the 9th inning with a lead and finishing the game (other innings matter too!). Alternatively, if you enter in a tie game and give up the lead or if you enter down by a run and give up a lot of runs, you get dinged with a meltdown when before you would not have picked up a blown save.

Essentially, SD and MD set the same basic threshold of the average save or hold (that’s where the 6% WE comes from), but applies it to all reliever appearances rather than just a subset of them. There’s no reason not to credit or debit relievers who are pitching in tie games or when trailing, or those who pitch in non-save situations with the lead. This scaling means that 40 SD is roughly equivalent to the bar set by a 40 save closer. However, relievers will generally accumulate more meltdowns than blown saves because you can earn a meltdown in more situations than you can earn a blow save.

This statistic, inspired by Jeff Zimmerman, coined by Tom Tango, and implemented at FanGraphs provides a much better approximation of reliever performance that saves. It shouldn’t be the only metric you look at for relief pitches, but it’s a nice tool.

To that end, let’s take a look at some different ways to use SD and MD to identify some top performers from 2014 (all numbers are for relievers with 40 innings or more in relief).

Let’s open with save leaders and their number of save chances and their save percentage to get a sense of the baseline.

Name SV SV% SVOpps
Fernando Rodney 48 0.94 51
Craig Kimbrel 47 0.92 51
Greg Holland 46 0.96 48
Trevor Rosenthal 45 0.88 51
Kenley Jansen 44 0.90 49
Francisco Rodriguez 44 0.90 49
Huston Street 41 0.93 44
Jonathan Papelbon 39 0.91 43
Steve Cishek 39 0.91 43
David Robertson 39 0.89 44

So Fernando Rodney is the answer to the opening question. He locked down 48 games and blew just three saves. Greg Holland only saved 46 games but he had just 48 chances. If you like rate stats, maybe he’s your guy for 2014. But you’ll notice, if you’re up on your relievers, that a ton of great pitchers aren’t on this list at all. Let’s try something else. Let’s do the same thing, but now let’s look at SD, SD% and SDOpps (SD+MD).

Name SD SD% SDOpps
Tony Watson 44 0.846 52
Jonathan Papelbon 43 0.896 48
Zach Britton 43 0.896 48
Steve Cishek 42 0.875 48
Mark Melancon 40 0.833 48
Cody Allen 40 0.816 49
Joe Smith 40 0.889 45
Huston Street 38 0.905 42
Trevor Rosenthal 37 0.740 50
David Robertson 37 0.804 46

This is a much different list. We see Cishek, Street, Rosenthal, Papelbon, and Robertson again, but the other five names are new. Tony Watson was 2 for 9 in official save chances in 2014, but he was awesome in shutdown situations. Tyler Clippard, Pat Neshek, and Wade Davis are other names you’d see near the top of this table if it extended further.

Among relievers with at least 10 SDOpps, Ken Giles shares the best SD% at 90.5 (19 for 21) with Street. Fernando Rodney, for example, has just 25 SD in 32 opportunities, making his SD% (78.1) much lower than his SV% (94). There are plenty of fun examples to pull out and examine, but let’s peak at one more. Let’s look at pitchers with at least 20 save opportunities and see which pitches have the biggest differences in SV% and SD%.

Twenty nine pitches had 20 or more save chances and only Sergio Romo had a better SD% than SV% (85.4% to 82%). Here are the five biggest gaps.

Name SD% SV% Diff
Craig Kimbrel 0.769 0.92 -0.152
LaTroy Hawkins 0.731 0.88 -0.154
Fernando Rodney 0.781 0.94 -0.160
Aroldis Chapman 0.786 0.95 -0.162
Jenrry Mejia 0.667 0.90 -0.237

So while you probably won’t see pages and pages of posts about SD and MD on FanGraphs because we typically speak in runs and wins (above replacement) rather than counting stats like these, they’re a great tool if you’re looking for something that fills the niche of saves without being totally useless.

This link will provide you with the data used for this post if you want to play around, and as always, feel free to post questions in the comments section.

We hoped you liked reading Shutdowns, Not Saves: The Logic and the Leaders from 2014 by Neil Weinberg!

Please support FanGraphs by becoming a member. We publish thousands of articles a year, host multiple podcasts, and have an ever growing database of baseball stats.

FanGraphs does not have a paywall. With your membership, we can continue to offer the content you've come to rely on and add to our unique baseball coverage.

Support FanGraphs




Neil Weinberg is the Site Educator at FanGraphs and can be found writing enthusiastically about the Detroit Tigers at New English D. Follow and interact with him on Twitter @NeilWeinberg44.

newest oldest most voted
Bull City Cock & Bull
Guest
Bull City Cock & Bull

In 2009, members of the Orioles Forum on MLB.com came up with an eyeball-test statistic called the STB, which approximates your Melt Down, but with much more, um, color.

http://www.forums.mlb.com/discussions/Baltimore_Orioles/_/_/ml-orioles/44115.1