Context: Neutral or Dependent?

Every statistic is an answer to a question. “How often does a batter reach base?” is answered by On-Base Percentage. “How many extra bases does a hitter average per at bat?” leads us to Isolated Power. A statistic is only as good as it’s generating question and if you’re asking a silly question, the statistic may give you a silly answer. Stats like pitcher wins, saves, and RBI all answer questions, but they don’t really answer questions we really want to know the answer to.

RBI, for example, tells you how many times a batter has had their hit, walk, or sacrifice fly lead directly to a runner crossing the plate. On the surface, this may seem like a useful statistic as a measure of run production. But you soon realize that RBI is reliant on the number of opportunities each player has to drive in runs. Coming to the plate with a man on first and coming to the plate with a man on third are not the same type of RBI opportunity, even if the batter hits a single in both situations.

In other words, RBI is a very crude context-dependent statistic. Generally, RBI isn’t very useful because it doesn’t provide you with a lot of information about individual player’s role in the production of a run. If they have a lot of RBI, did they have a ton of opportunities? Did they cash in on a large percentage of their opportunities? You don’t really know. But the fact that RBI doesn’t provide much insight does not mean that context-dependent stats aren’t valuable when designed properly. Essentially, context-neutral and context-dependent stats are both useful, but they are simply answering different questions.

A context-neutral stat is one that values a given action equally no matter the circumstances in which the event occurred. For example, stats like wOBA, wRC+, wRAA, and many others give equal credit for singles in the first inning of a blowout and the 10th inning of a tie game. A double has the same value if the bases are loaded with two outs or if the bases are empty with no outs. The action is the action, no matter what.

A context-dependent stat varies the values a given action has depending on circumstances over which the player has no control. Stats like RE24 or WPA give different values to the same action dependent on the state of the game around the player. A double with the bases loaded is treated differently than a double with no one on base.

In sabermetric circles, there is a lot of emphasis on context-neutral numbers because research has generally found that “clutch” performance isn’t a skill, and that good hitters will generally hit well and bad hitters generally won’t. Performance in certain contexts rarely predicts performance in future, similar contexts above and beyond a player’s skill level. As a result, when we look at individual players, we often look at their context-neutral numbers. That doesn’t mean context-dependent numbers are useless, they just generally don’t answer the questions that many saber-minded people are asking.

But context-dependent stats certainly have merit in describing past value if you are specifically interested in how a player performed relative to his situations. Many people prefer context-dependent numbers for awards like MVP because they care about how much a player’s hits increased their team’s chance of scoring or winning. It might help to think of these as “impact stats.” A double is a double according to Batting Runs (basically wOBA scaled to PA), but how much did that double impact the inning in question? We need RE24 (change in run expectancy) for that.

According to Batting Runs (the batting component of WAR), a double is worth somewhere around 0.73 runs above average. In other words, on average, doubles add 0.73 runs for a team. But every single double doesn’t add exactly 0.73 runs because partial runs aren’t a thing. A bases-empty, two-out double is only worth about 0.2 runs on average and a bases-loaded, two-out double is worth about 2.6 runs on average. Do you want to give credit to the batter based on the average value of the action or the value of the action as it happened? It’s a decision you have to make as the person asking the question.

You can take it a step further and add in the inning and score, which will help you get to Win Probability Added (WPA) instead of simply run expectancy. Now that bases-loaded, two-out double is worth even more if your team was down two runs, but much less if your team was up by six. It all comes down to how much context you want to include in your evaluation.

Most people would argue that RBI includes too much of the wrong kind of context. RBI only gives you credit if you’re the batter who drives the run in and doesn’t give you credit if you are 1) the runner who got on base to be driven in, or 2) if you move a runner up and they are later driven in by another batter. RE24, on the other hand, gives you proper credit no matter what your role was in scoring that run.

For example, if an inning starts with three singles and then three strikeouts, the first batter likely gets a run scored, the second batter gets nothing, and the third batter gets an RBI. With RE24, the first batter gets 0.37 runs, the second batter gets 0.54 runs, and the third batter gets 1.00 runs (assuming no one goes first to third, and runner from second scores). This still gives more credit to the guy who gets the third hit, but it also gives proper value to the man in the middle, who was critical to the run scoring.

It all comes down to deciding what kind of evaluation you want to make when using value based statistics. If you’re looking at OBP, you either reached base or you didn’t, but when you’re looking at wOBA, those weights are based on average run values. Sometimes you might want to use those average values and sometimes you might want to know the run expectancy of the individual situations. If you care about the player’s individual performance, context-neutral probably makes sense. If you care about their impact on individual moments, context-dependent is probably the way to go.

The key is always to use the right stat for the question. If you want to know how well a player hit, you probably want to only judge the pitcher-batter confrontation, but if you want to specifically know how much a batter aided his team’s odds of winning today, WPA might make more sense.  Never pick out a stat and use it to determine your question. That’s how people have wound up thinking RBI is a good measure of context-dependent value even though it isn’t. Determine the question you want to answer and then find a stat that is designed to do so. Sometimes that leads you to a context-neutral number and sometimes it leads you to a context-dependent one.