Skip to content

Estimating Player Impact

September 16, 2012

Ok, so I did some experimenting, but then I figured, what the hell, I might as well go for it.

Yes, this means I’ve created my own method for estimating the on-court impact of basketball players. While my method is not groundbreaking, I think it’s unique enough to write about and present. Let me start by saying that I’m not going to try and hard sell you. There are some drawbacks to this method. But using it in conjunction with what we can observe otherwise, I think it can be a valuable tool for assessing players. (I know what you’re thinking: ugh. another box score metric. just what we need. Well, deal with it.)

Remember in Jurassic Park when they sit down in that room and the little DNA guy summarizes how the scientists were able to recreate dinosaurs? They found dino DNA in mosquitoes that were preserved in amber. Except there was a problem – there were gaps in the dino DNA, missing chromosomes or something along those lines – but they solved the problem by filling the gaps with frog DNA. Well, that’s basically my approach to estimating the production of NBA players. We know how their teams performed, we have the basics of how each player performed, except we have gaps – what role do assists play? how much impact did player X have on his team’s defense? How does player A’s presence impact his team’s offense? Et cetera. Well, the goal is to use clues from recorded player stats and overall team production to try and fill those gaps. And yes, I’m aware that the frog DNA contributed to the downfall of Jurassic Park since it enabled the dinosaurs to spontaneously change genders which allowed them to reproduce.

I started with the basics. Rather than use a number found in a regression on multiple seasons or average points per possession over multiple seasons to determine the value of possession stats like missed shots, rebounds, steals, and turnovers, I used average points per possession for that particular season. Thus, at the heart of the metric, or I guess the skeleton, points are worth points and possessions are worth points per possession for that particular season. Yes, that means the player’s value shows up in point margin produced, which I can then convert to wins or estimate into point margin produced per 100 possessions. Now let’s break this into offense and defense.

Defense

Let’s start with defense, the big box score unknown. Every box score method ever created struggles in one way or another with defense. Some only take defensive rebounds, blocks, and steals into account, some do that plus assign the value of the remaining team defense to players equally by minutes played, and some bold metrics even attempt to assign value based on counterpart data, which of course is only available for certain, very recent seasons, and can’t possibly take into account switches, help defense, and awkward defensive assignments (I’m not saying using counterpart data can’t be helpful – I would even consider using it, it’s just really, really tricky. Synergy might also be helpful here). Ok, here’s what I did.

Defensive possessions can end in three ways:

  1. Defensive stops
  2. Forced turnovers
  3. Opponent made field goals or free throws

Let’s start from the top. Defensive stops show up as defensive rebounds (and team rebounds, but we’ll get to that later). Think about it, the bad guys miss a shot. If they rebound it, it’s still the same possession. If you rebound it, it’s a defensive rebound. So essentially a defensive rebound is not one man ending the opposition’s offensive possession by grabbing the rebound. Rather, it is a team member securing the ball after a defensive effort by the team. Thus, the defensive rebound must be spread among team members or else Troy Murphy, Kevin Love, and Carlos Boozer will end up looking like defensive monsters.

I give the rebounder a portion of the credit for the defensive stop – after all, he secured the possession and kept the opponent off the offensive glass. The team (including the rebounder) get the remainder equally for the stop. But that can’t possibly distinguish dominant defensive players from poor ones. So how can we do that? Well, we know that a team’s dominant defenders are almost always its interior players who block shots – the bigs anchor the defense. Think Russell, Olajuwon, Duncan, Garnett, Ben Wallace. So I allocated a chunk of the value of defensive stops to players by percentage of the team’s blocked shots. This serves to credit defensive anchors by not only rewarding them for their blocks, but also for their interior presence since players with great interior defense are almost always good shot blockers. It’s important to note that, using this method, good shot blockers on great defensive teams are better than great shot blockers on average defensive teams. Also, blocks are used as a function of defense rather than being added on top of defense, so our average point margin will add to zero at the end of the day.

While this method of allocating credit for defensive stops seems to be very accurate overall, there are two notable downsides:

  1. While steals help us to tease out good perimeter defenders, we still can’t properly attribute value to the most impactful defensive stoppers on the perimeter (like Pippen, Payton, and Artest), which I think is the largest drawback of this method.
  2. In rare cases, a good defense’s good shot blockers aren’t good defenders (I’ll come back when I think of an example), and, also rarely, some great defenders on great defensive teams don’t block many shots. I think this mostly gets back to #1, although there are probably a few great interior defenders that don’t block shots (again, can’t think of an example off the top of my head EDIT: maybe Jason Collins?)

Ok, that’s the hard part. The rest of defense is pretty straight forward. The team gets its opponent made field goals allocated equally among its players by minutes played. Again, this doesn’t enable us to punish guys who get lit up, but I think in general when a team allows baskets, more than one defender is at fault. Likewise, the team shares the credit for opponent turnovers, except for steals, in which the player who got the steal gets the bulk of the credit, with a portion going back to his teammates (most steals involve teamwork – if LeBron and Wade double Durant at the top of the key and force him to throw a bad pass that Bosh intercepts, Bosh isn’t alone responsible for that steal). Finally, players are punished for opponent made free throws by their percentage of their team’s fouls, and team turnovers and team rebounds are credited to players equally by minutes played.

Offense

Offense is much more easily captured by the box score than defense. But that doesn’t mean we all agree on the value of particular offensive activities. Like a defensive possession, an offensive possession can end in three ways:

  1. Made field goals or free throws
  2. Missed field goals or free throws that are rebounded by the defense
  3. Turnovers

So that’s simple enough right? A made shot is worth 2 or 3 points minus the value of the possession that you just used, and a  missed shot or a turnover is worth the negative value of a possession since you just gave the defense a possession. Except it’s not that easy. You see, offensive players are interacting at a much deeper level than that! People are setting screens and moving without the ball. Players are making passes that set up other players. And even deeper. Some players are impactful by their presence because of how good dangerous they are – they might not even have to shoot a high percentage to have a positive impact! What do we do?!

First, I allocated value like I explained above. Then, I took a portion of each player’s positive scoring value (aka points) and attributed it to the teammates responsible for creating easy shots by percentage of team assists – this is important because again, I’m treating assists as a function of a team’s offensive production rather than adding it on top.

Turnovers are treated very basically. If a player turns the ball over, he gave up a possession, so he gets the value of points per possession deducted from his value for every turnover he commits.

Let’s not forget about the offensive rebound. While an offensive rebound doesn’t secure a new possession in itself, it allows the offense to take another shot, so its value, on its face, is the same as a possession. But what is an offensive rebound? Is it the miraculous product of a super player defying all odds to get the team another chance, or is it merely an expected result, a product of circumstance, or worse, a negative that ends up hurting the team in the long run since it’s a sign that the players didn’t get back on defense (after all, offensive rebounds correlate negatively with team defense)? I think it’s somewhere in between. You see, offensive rebounds are important, but because of the necessity of the team to get back on defense, only certain players on a given team can actually go for the offensive rebound. These players may be a designated position – maybe center or power forward, but they don’t have to be. This is why Dirk is treated relatively poorly by Wins Produced: his 1.1 offensive rebounds per game pales in comparison to the average PF’s 3.4. But Dirk’s role on offense isn’t the same as a typical power forward. In addition, offensive rebounds are largely a product of circumstance. Still, players should get credit for offensive rebounds, and they do here – a little more than for defensive rebounds. And the rest of the value is spread to the shooter (a small portion) and the rest of the team (a larger portion).

Ok, now that we have that, there’s one more issue that needs to be resolved: usage. The traditional argument for implementing usage is the skill curve: the more offensive possisions a player uses, the more his efficiency will drop. My argument (maybe its different, maybe it explains why the skill curve exists) is that players who use a lot of possessions attract more defensive attention, which allows their teammates to score at higher rates. That is, their offensive presence is valuable enough to provide a positive impact in addition to their actual field goal percentage. The most extreme example , I think, is 2000 Shaq (and by extreme I mean easy to observe). Watch an early 2000s Lakers game. When Shaq touches the ball in the post, he’s like a magnet. The entire opposing defense pays attention to him. His offensive presence creates a positive team offensive impact.

So how did I take usage into account? Basically I multiplied a small portion of the difference between a player’s usage and the average player’s usage by the player’s scoring production. So Carmelo gets a little better on offense, Chandler gets a little worse.

Like with defense, this method of estimating a player’s offensive impact has a couple of drawbacks:

  1. We can’t distinguish between a player’s different assists. That is, if Steve Nash uses his quickness and supreme passing ability to create a free basket for Amare Stoudemire it is treated the same as if Rajon Rondo passes it across the perimeter to Ray Allen, who used other players’ screens and his ability to move without the ball to get open.
  2. A player whose only role is to shoot the ball when he’s open, and does it with remarkable efficiency (Kerr, Novak) tend to get overrated on offense.

It’s important to note, and I think this is one of the main strengths of this method, that there is no fudge factor or fitting. Every year, the average point margin adds to zero. I also want to note that I used a traditional pace adjustment to adjust for team pace rather than estimating it from team blocks or some other method.

Results

All right, how about some results, huh? Here’s the 10 year estimated impact for players who played at least 10,000 minutes in that time (OPM100 is offensive point margin produced per 100 possessions, DPM100 is defensive, and PM100 is total point margin produced per 100 possessions):

And here’s this season (Min. 1610 MP):

Conclusions and Future Adjustments

As for prediction ability? I ran a retrodiction test similar to what Alex at Sports Skeptic did, except I used wins rather than point margin as the measurement and compared only Wins Produced, Win Shares and my method, which I guess for lack of a cleverer name, I’m calling Estimated Impact. Estimated Impact outperformed both over a ten year period, and  I’d invite anyone to reproduce the results.

I think perhaps the most room for improvement lies within the possibilities presented by play-by-play data. In particular, I hope to use it to refine estimations by using results while players were playing rather than using overall team results by minutes played.

So what do you think? Did I take usage too far? Does my attempt to separate great defenders fall short? Am I underrating rebounding? Is my method just entirely too subjective? I’m open to suggestions. Regardless, I feel like I’ve learned a lot about both box score limitations and basketball in general as a result of this project. And it’s been fun, too.

-James

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: