Shutdowns, Not Saves: The Logic and the Leaders from 2014

Who led the league in saves in 2014? Hopefully, you don’t know the answer off the top of your head. Saves aren’t a good measure of anything relating to player performance or talent and with so many things you could remember about the 2014 season, you probably don’t want to waste vital brain capacity on a random piece of trivia like who had the most saves.

The reason saves aren’t very useful is because the rule itself is not designed to provide much information. You can earn a save if you strikeout Miguel Cabrera, Victor Martinez, and J.D. Martinez in a one run game or you can earn a save if you allow five base runners against the bottom of the Padres’ order. You don’t earn a save if you preserve a tie, or if you preserve a lead in the 7th inning. Nearly everything about the rule is arbitrary, which leads you to find arbitrary results.

But the idea of something like a save is compelling for many people. There is a desire for a statistic that measures the number of a times a reliever comes in and pitches very well in an important spot. We can look at rate stats like ERA, FIP, or xFIP or cumulative numbers like RE24 or WAR, but it’s perfectly fine to want some sort of counting stat that tracks how many times a reliever slammed the door (or didn’t).

Luckily, we carry such a statistic here at FanGraphs. It’s called the Shutdown (SD). The logic is very simple. If a reliever posts a Win Probability Added (WPA) mark of 0.06 or greater in an individual game, they are given a “shutdown.” If their WPA for the game is -0.06 or worse, they are given the opposite, a “meltdown.” If it’s between the two, they are given neither.

This solves one of the key flaws in the save statistic. Relievers earn SD or MD based on the Win Expectancy of their appearance rather than a weird and arbitrary definition, which means that you can earn one without pitching in the 9th inning with a lead and finishing the game (other innings matter too!). Alternatively, if you enter in a tie game and give up the lead or if you enter down by a run and give up a lot of runs, you get dinged with a meltdown when before you would not have picked up a blown save.

Essentially, SD and MD set the same basic threshold of the average save or hold (that’s where the 6% WE comes from), but applies it to all reliever appearances rather than just a subset of them. There’s no reason not to credit or debit relievers who are pitching in tie games or when trailing, or those who pitch in non-save situations with the lead. This scaling means that 40 SD is roughly equivalent to the bar set by a 40 save closer. However, relievers will generally accumulate more meltdowns than blown saves because you can earn a meltdown in more situations than you can earn a blow save.

This statistic, inspired by Jeff Zimmerman, coined by Tom Tango, and implemented at FanGraphs provides a much better approximation of reliever performance that saves. It shouldn’t be the only metric you look at for relief pitches, but it’s a nice tool.

To that end, let’s take a look at some different ways to use SD and MD to identify some top performers from 2014 (all numbers are for relievers with 40 innings or more in relief).

Let’s open with save leaders and their number of save chances and their save percentage to get a sense of the baseline.

Name	SV	SV%	SVOpps
Fernando Rodney	48	0.94	51
Craig Kimbrel	47	0.92	51
Greg Holland	46	0.96	48
Trevor Rosenthal	45	0.88	51
Kenley Jansen	44	0.90	49
Francisco Rodriguez	44	0.90	49
Huston Street	41	0.93	44
Jonathan Papelbon	39	0.91	43
Steve Cishek	39	0.91	43
David Robertson	39	0.89	44

So Fernando Rodney is the answer to the opening question. He locked down 48 games and blew just three saves. Greg Holland only saved 46 games but he had just 48 chances. If you like rate stats, maybe he’s your guy for 2014. But you’ll notice, if you’re up on your relievers, that a ton of great pitchers aren’t on this list at all. Let’s try something else. Let’s do the same thing, but now let’s look at SD, SD% and SDOpps (SD+MD).

Name	SD	SD%	SDOpps
Tony Watson	44	0.846	52
Jonathan Papelbon	43	0.896	48
Zach Britton	43	0.896	48
Steve Cishek	42	0.875	48
Mark Melancon	40	0.833	48
Cody Allen	40	0.816	49
Joe Smith	40	0.889	45
Huston Street	38	0.905	42
Trevor Rosenthal	37	0.740	50
David Robertson	37	0.804	46

This is a much different list. We see Cishek, Street, Rosenthal, Papelbon, and Robertson again, but the other five names are new. Tony Watson was 2 for 9 in official save chances in 2014, but he was awesome in shutdown situations. Tyler Clippard, Pat Neshek, and Wade Davis are other names you’d see near the top of this table if it extended further.

Among relievers with at least 10 SDOpps, Ken Giles shares the best SD% at 90.5 (19 for 21) with Street. Fernando Rodney, for example, has just 25 SD in 32 opportunities, making his SD% (78.1) much lower than his SV% (94). There are plenty of fun examples to pull out and examine, but let’s peak at one more. Let’s look at pitchers with at least 20 save opportunities and see which pitches have the biggest differences in SV% and SD%.

Twenty nine pitches had 20 or more save chances and only Sergio Romo had a better SD% than SV% (85.4% to 82%). Here are the five biggest gaps.

Name	SD%	SV%	Diff
Craig Kimbrel	0.769	0.92	-0.152
LaTroy Hawkins	0.731	0.88	-0.154
Fernando Rodney	0.781	0.94	-0.160
Aroldis Chapman	0.786	0.95	-0.162
Jenrry Mejia	0.667	0.90	-0.237

So while you probably won’t see pages and pages of posts about SD and MD on FanGraphs because we typically speak in runs and wins (above replacement) rather than counting stats like these, they’re a great tool if you’re looking for something that fills the niche of saves without being totally useless.

This link will provide you with the data used for this post if you want to play around, and as always, feel free to post questions in the comments section.