One of the more consistent refrains regarding the usage of statistics in basketball, from both stats gurus and the anti-stats crowd alike, is that basketball analytics lack the proper context to be relied upon.
This is fairly objectively not true, for a number of different reasons.
First, the basketball statistics are always built upon variables that are tied into concrete occurences on the basketball court; no stat is taken without an actual basketball thing happening to inform the math. No basketball stat occurs in a vacuum of pure math on an excel spreadsheet. So, every statistic comes weighted with the context of the basketball that informs the data.
Perhaps a better complaint would actually be that it’s difficult to divorce data from its context. For example, Tyson Chandler’s Rim Protection statistics (from Nylon Calculus’ Seth Partnow) weren’t as good last season as people have come to expect of Tyson, but that’s in large part due to his injuries/lethargy from being on a horrific Knicks team.
Though, this leads me into my second — and probably more important — reason why analytics don’t “lack context.” The analytics have all the necessary context: data comes from players playing on a particular team with a particular role (or lack thereof).
So, the only degree to which the statistics lack context is the degree to which the interpreter chooses not to engage with that context.
The information that you want is available, (how good was the team that this player was on? How much spacing was there? Was the whole team good defensively? What role did the player have? How did this all affect the statistic?) and any failure to add meaning to the data because of the “context” is a failure in interpretation.
How we discuss and frame basketball analytics should be a point of contention, then. How can we best encourage an interpretation so that no one is worried that we’re “avoiding context?”
Something that a lot of analytics oriented bloggers and writers forget is that the framing of the statistics in concrete basketball terms is probably as important as the analysis itself, if not more so.
Making sure that people understand how, exactly, an analysis applies to what people watch on the court is almost (if not quite) more necessary to widespread comprehension and usage than including methodologies and formulas.
The problem, as I understand it, is essentially that writers haven’t done a good enough job in general of saying “this is calculated by doing X and Y. But what that tells us is that Z is happening on the floor.” Without that last bit of information, it can be really hard to say confidently what we know and need to know about players and teams from the data available.
The other side of all of that, though, is that it’s really important to have a knowledge base when starting along with either doing or reading or consuming basketball analytics. It’s important that the writer explains what the data means about the game, but it’s also essential that the fan understand what is happening in general in the game.
Which, ultimately, brings us back to the relationship between analytics and “the eye test” that so many people are so fond of discussing (and which I probably shouldn’t condescend, since it’s the subject of this article).
I’m tempted to condescension, though, because as several people have pointed in out in a few places, the eye test is a form of analytics. Watching the games, scouting games, and watching film carefully is a way in which we learn more about the game of basketball.
But beyond that, even, the eye test is foundational to our understanding of basketball statistics. The game, what actually happens on the floor and how we interpret what we see, is what forms the knowledge base out of which comes the statistics.
The eye test informs statistics, first, because no statistician would have a question to try and answer without watching the game. No one asks, “I wonder if I can measure how well a player defends the rim,” without watching a player like Tyson Chandler first excel and then disappoint.
Just as importantly, though, the eye test is essential to analytics because it forms the baseline knowledge against which the validity of an analysis can be measured.
Points per game, for example, were the first real statistic that anyone kept track of. Somewhere down the line, though, someone looked at the list of team’s points per game, and saw that the league leader didn’t seem, per the eye test, to actually be the best offensive team in the league.
From there, it was easy to ask, “well, what may be causing this disparity?” and to eventually say, “lets measure an offense by the amount of points a team scores per possessions instead.”
Similarly, it’s a large part of why Wages of Win’s “Wins Produced” is held in such low regard: any statistic that says that DeShawn Stevenson had a bigger impact on the 2011 Mavericks championship team than Dirk Nowitzki is pretty fundamentally broken.
We know that, in large part, because we watch the games. It’s necessary for analysts to vet their own logic if it doesn’t match up with what’s blatantly obvious from watching the game.
In essence, the eye test is a form of analytics not just because we can learn something from analyzing our experience of a game, but because what we watch is our fundamental knowledge base. Basketball doesn’t exist without people watching. It is the beginning and end of all analysis.
I say this, mainly, to point out that most statistics are in no way competing with the eye test, because the eye test was fundamental in forming the statistic.
I also bring it up, however, to point out that the fact that if statistics’ place in basketball — relative to the eye test — is still in real contention, then the analysts haven’t done a good enough job of explaining what, exactly, any given statistic is relative to people’s understanding and experiences of games.
Without an understanding of methodology, there’s no way to confirm whether or not a statistic does what you claim (and/or whether or not the calculation is sufficiently rigorous), but without an understanding of application, there’s no way to know why it is or why anyone would care, and there’s no way to learn from the data.
The whole point of analytics is fundamentally to learn something about basketball that we didn’t know before, that we can apply to watching and playing the game in such a way that understand more about how the game works and why.
For that reason, it’s essential that we always make sure we understand what the analytics are, why they exist the way they do, and what it is that they tell us about the sport that we watch. Otherwise, they get us nowhere.
So, keep that in mind the next time you read or write something about statistics. We’re here to learn. Lets keep it that way.
