Even the best statistics, things like wRC+, are imperfect. You can’t take wOBA as a perfect measure of truth or be certain that FIP is a perfect estimate of pitcher performance. In many cases, they may be the best we have, but we acknowledge the limitations. While it’s true that even our favorite metrics have flaws, that doesn’t mean that we should give equal considering to extremely flawed statistics.

This post will be the first in a series, scattered across the offseason months, that demonstrates the serious problems associated with some of the more popular traditional metrics. Many of you are well aware of these issues, but plenty of people are reading up on sabermetrics for the first time every day and our goal here is to create a comprehensive guide that helps everyone get the most out of everything we have to offer. Part of that puzzle is explaining why you might not want to look at things like batting average, RBI, and wins. Today, we’ll start with Runs Batted In (RBI).

Read the rest of this entry »

It’s always important to know exactly what question you’re asking. In baseball, one of the most difficult distinctions for many people is difference between how a player has performed and how the player is going to perform. It is very common to see analysis, even from well-versed fans of advanced metrics, that goes something like this.

“You want Player X up in this situation because he has a .380 wOBA against LHP this year.”

Even if the sample size is sufficient, that statement isn’t an ideal reflection of our expectations about the future. We frequently treat the recent past as an estimate of future skill even though we know that it isn’t. Certainly, the educated observer doesn’t need to be convinced that the ten most recent plate appearances aren’t useful information on their own, but even the last 600 PA aren’t what you want. What you want, when making a claim about the present or future, is a projection.

Now you might not always be asking a future-oriented question. If you want to decide who the best hitter in baseball was in 2014, you don’t need a projection. If you want to know who has thrown the most effective slider since 2012, you don’t need a projection. But if you want to know who is the best hitter in baseball right now or who is going to be a better signing next year, you want a projection.

A projection is a forecast about the future. It is certainly imperfect. It’s an estimate. Projecting a .400 wOBA doesn’t mean you make a $1,000 bet on that player running a .400 wOBA, it means that’s the best guess for how that player is going to perform. On average, some players will do better and some players will do worse. There’s error involved in the actual calculations, but the idea behind it is sound.

You want to make decisions about the future based on every single piece of relevant data and you want to weigh that data by its importance. Steamer projects Miguel Cabrera will have a .407 wOBA in 2015. What that means is that Steamer, based on everything it knows about Cabrera’s history and the way players typically age, we should expect a .407 wOBA. Steamer knows that Cabrera had a “down” year in 2014, but it also knows he had a great 2013 and that hitters of his caliber usually age in a certain way. It’s all built in. You don’t just care how a guy did last year or how he did in his career, you care about the entire body of work and the underlying factors that are driving it.

Think about it like the weather. You want to know if it’s going to rain today. How would you go about predicting whether or not it will rain? You would obviously pay some attention to the recent weather, but you would also look at historical weather patterns, and then you would look at the conditions in and around your area. It rained to your west last night: When that happens, how likely is it that the rain will come your way? There is a certain mix of pressure and air flow, what does that usually lead to? It’s all relevant information.

The same is true for baseball players. You care how Cabrera has hit for the last 600 PA. Those are super important data points, but they aren’t the only ones. You also care about the 600 PA before that. And before that. The older the data, the less important, but it never becomes useless. Additionally, you don’t just care about performance, you care about the underlying numbers.

If a player has a .400 wOBA with a.390 BABIP, you know most of their great season is predicated on getting lots more hits on balls in play than average. You wouldn’t automatically expect that .390 BABIP to continue, so you need to determine the typical BABIP regression for players of this type based on everything else you know about them.

You never want to make a decision based on a player’s simple past. You want to use that data to make a valid inference about the future and the process of doing so constitutes a projection. There are all sorts of different methods. Some are as simple as taking a couple years of data and weighing them by recency. Some like ZiPS, Steamer, Oliver, etc use much more advanced methodologies to estimate how well they think a player will perform using all sorts of information about that player and similar players of years past.

There is no ideal system, but the idea of projection is ideal. You care that the Royals won X number of games last month, but that doesn’t mean they’ll win X games this month. The last month is relevant, but it isn’t the whole story. Baseball is volatile and unpredictable any one sample of data is going to deviate from the true, underlying skill of a player. You want to do your best to make the best guess you can about their future and then use that to make decisions. That’s projection.

We like projections at FanGraphs. They’re useful for approximating current true talent levels and they help us predict which teams will be successful and which teams won’t. You could guess who was going to win the divisions based on the previous year’s player performances, but those players are going to perform differently this year and you want to account for that.

Many people are turned off by the idea of projection because projections seems like a black box. If you see a guy is hitting .380 wOBA this year but the projection says he’s a .340 wOBA hitter, you can’t easily internalize that a .340 wOBA hitter has produced a .380 wOBA to date. It’s human nature to assume the outcomes we observe are measures of truth, when in reality, they are influenced by randomness.

So when a stat-geek says they don’t want Player X to hit because they aren’t great, even though the player has a .350 wOBA during the last 400 PA, it doesn’t make sense. They have hit well, so they are good. But that isn’t exactly right. Their last 400 PA matter, but they don’t tell the whole story. A projection is trying to tell the whole story.

The systems aren’t perfect and the nature of the beast means they won’t get very many players exactly right, but they do a better job predicting the future than the last six weeks or six months of data will.

But it all comes down to the question. You might not care very much about predicting the future or approximating true talent. If you only care about past value, you can stick to the raw stats. But if you want to say something about how well a player is going to perform and what their true talent is, you want a projection. FanGraphs houses many of these each year and you can follow along, not only with the preseason numbers, but how they change based on the data of a new season.

Questions about projections? Ask them in the comments!

Fielding Independent Pitching (FIP) is one of the more prominently featured statistics on FanGraphs and one of the bedrocks of sabermetric analysis. We all know that FIP is an imperfect measure of pitcher performance because it assumes average results on all balls in play, but we also know that it does a better job isolating the individual pitcher’s performance than simply looking at their ERA or RA9 because it only looks at strikeouts, walks, home runs, and hit batters. It’s a very informative tool, but it’s a metric derived from a subset of results.

When a pitcher’s ERA is significantly different from their FIP, the standard credo is that they were lucky or unlucky, but there are genuine reasons why a pitcher might have results that are better or worse than their FIP. To illustrate this, let’s take a peak at the biggest FIP over and under-performers of 2014.

Read the rest of this entry »

One of the most common questions I get when talking about advanced metrics with people who are new to the experience is “what’s the best stat for looking at X?” My standard response depends on the particular question, but I almost always drop the caveat that you should always be looking at multiple pieces of information rather than one single stat and I don’t think I’m alone in offering that advice.

As our metrics for evaluating baseball improve there’s a desire among many for the new stats to push the old stats out of the conversation. Now that we have wOBA, why would you ever use OBP? And then once you have access to wRC+, is wOBA even necessary anymore? If we have K%, isn’t K/9 completely useless?

In some cases, that’s a fine idea, but in many you would rather have access to as much information as possible because stats that don’t do very well on their own can still be informative in the context of other statistics. Wins Above Replacement (WAR) is the best single metric we have to determine a player’s complete value, but WAR only conveys the answer to a very specific question. If you want to know about how good a player is overall, WAR is great. If you want to know if he’s a power hitter or a player with a good eye, WAR doesn’t do very much.

The same is true for wRC+. You know a 150 wRC+ means someone has had a very good season, but you don’t know if he’s doing it with a high average, good patience, excellent power or some combination of them. We’re striving for better measures of performance but you can’t only look at one or two numbers because baseball is full of questions that require a variety of tools to evaluate.

Read the rest of this entry »

One of the hallmark statistics available at FanGraphs is Wins Above Replacement (WAR) and we’ve just rolled out an updated Library entry that spells out the precise calculations in more detail than ever before. There’s always been a clear sense of the the kinds of things that go into our WAR calculation, but we’re never just dropped an equation in front of you and said, “Here!”

As of today, we’ve done that and I encourage you to go check out our basic primer on WAR and our detailed breakdown of how we calculate it for position players. If you’re a hands on learner, grab a pen and paper or spreadsheet and follow along. I’m going to walk you through a complete examples of how to calculate WAR for position players. Let’s use the 2013 version of Joey Votto as our exemplar.

Read the rest of this entry »

We’ve spilled a great deal of virtual ink and audible podcasting words on the nature of Wins Above Replacement (WAR) and defensive metrics recently. Jeff Passan of Yahoo! Sports and many who responded to his critique of the current WAR calculation dug into the relative merits of the metric itself and how well we’ve estimated it to date. That’s a great conversation to have and Dave has done the heavy lifting on behalf of FanGraphs in that regard. I’d like to pivot and discuss a very important point about the use of statistics in baseball: Everything has flaws.

Every single statistic is wrong. Your eyes are wrong. It is all wrong. Nothing we have will provide you with perfect information or even truly accurate information with respect to the underlying variables about which you care. You don’t get to choose between flawed and not flawed statistics, you get to choose between useful and not useful statistics. More importantly, statistics become useful based on your awareness of the proper way to wield them.

Read the rest of this entry »

There’s a decent chance you’ve arrived at this page without a serious desire to hear more about defensive statistics. Trust me, I understand your frustration and your fatigue. Defensive stats like Ultimate Zone Rating and Defensive Runs Saved are controversial in some circles because they are reasonably new and the underlying data is somewhat hidden from view. You hear words like “flawed,” “absurd,” and “subjective” surrounding them. You’re tired of it.

Yet I’d like to lay out why we have advanced defensive statistics and how they work in the abstract. You won’t get to the end of this post and decide that UZR has perfectly measured Alex Gordon’s defense, but hopefully you will have a better appreciation for why we measure defense the way that we do.

Read the rest of this entry »

One of the things people love about baseball is that the game is both very simple and very complicated all at once. Baseball is simple in that all you’re trying to do is score more runs than the other team during 162 finite, nine inning contests. You are trying to reach base and advance runners and you are trying to prevent the other team from doing the same. How you go about doing those things is where baseball gets complicated. Jeff Sullivan often refers to baseball as being “obnoxiously complicated,” which I find to be a fitting description.

Think of all of the different possible outcomes of every pitch and all of the different pitches and locations from which the pitcher can choose. The complicated part of baseball is what makes baseball interesting, but the simple part of baseball is where you need to start to get your head around sabermetrics and player evaluation. Baseball is about producing and preventing runs.

As a result of that simple reality, the heart of baseball analysis is determining what leads to run scoring and run prevention. Specifically, how many runs is each possible action worth? If a player hits a single, how much has that player just increased his team’s odds of scoring a run? If a fielder makes a nice running catch, how many runs has he prevented? We don’t actually care about hits and walks and double plays, we care about how those finite events contribute to the overall goal.

Read the rest of this entry »

If you spent time hanging around the comments section of Dave’s Alex Gordon piece, you lurked in the shadow’s of his conversation with Jeff Passan on Twitter, or you’re one of those people who Twitter searches the word “FanGraphs,” you probably saw a decent amount of skepticism about single-season defensive metrics this week. People tossed around words like “flawed” and “absurd.”

The interesting part of the debate, for me at least, was that there was skepticism from both sides. The sabermetric elite dove into an esoteric debate about how to best incorporate defense into WAR and less analytically minded fans used Gordon passing Mike Trout in WAR as kindling for their “WAR is silly” crusade.

Dave’s piece does a nice job covering exactly what it means to say Alex Gordon leads position players in WAR, but the fact that Dave had to write that piece in the first place speaks to a problem we often run into when using advanced metrics. It’s a communication problem. Dave addresses it, but I’d like to expand on it here because it’s vitally important.

Read the rest of this entry »

One of the things baseball fans and analysts work very hard to do is isolate individual performance. At the end of a game, there is a final score that tells you how many runs each team scored. At a very basic level, that’s all that really matters. Baseball is a battle to score more runs than your opponent over the span of nine innings repeated 162 times. Yet analyzing the game requires more information than that because we want explanations. We want to know which players are good and which players aren’t so good. We care about how individual performance contributes to winning.

For pitchers, this is especially difficult because while pitchers have a huge impact on the number of runs they allow, they don’t have complete control. You can’t just look at the number of runs a pitcher allowed and say they were definitively responsible for those runs and call it a day. You aren’t isolating their performance and if you aren’t isolating individual performance you’re looking only at outcomes, and that’s not typically very interesting.

Every statistic, or really any analysis in general, should start with a question. On a basic level, the question we have is “How good is this pitcher?” which more specifically translates into “How effective is this pitcher at preventing runs?”

Read the rest of this entry »