Deep Dive on Regularized Adjusted Plus Minus II: Basic Application to 2017 NBA Data with R

In our previous post, we introduced the theory associated with Regularized Adjusted Plus Minus (RAPM) through an illustrative example. In this post, we walk through a vanilla-flavored methodology for building a RAPM model for NBA data. In this article, we focus on the data necessary, the required data manipulation process, and methodology for determining required hyper-parameters. All of this done, by request of the readers, in R. Apologies ahead of time if my code is spaghetti. I rarely program in R. I will also provide screen caps of the code, so it can fit and be readable on zoom. If you’d like a copy of the code, feel free to message me.

In Case You Missed It: Why RAPM

In our previous article, we asked how to identify player contribution with respect to scoring in a game. We provided a relatively simple example and walked through the most fundamental statistic: plus minus.

Identifying that the statistic really isn’t all that helpful for understanding player contribution, we set out to identify how players interact and use a basic regression model. This yielded Adjusted Plus Minus (APM). In this model, however, multicollinearity crushes us and we either have to get creative with inputs or deleting players in a strategic manner. The resulting multicollinearity rears its ugly head in the form of over-inflated variances. So our resulting estimates aren’t statistically different.

Instead, we control the degree of multicollinearity by introducing a little bias. This method is called ridge regression and yields the Regularized Adjusted Plus Minus (RAPM) model. We are able to control the impact of multicollinearity in attempt to get stable and comparable numbers between players. We give up a degree of interpretability in the model, but are still able to perform prediction and can compare the relative contribution of players.

Start With the Data: What’s an Observation?

First, we have to define an observation of data. By asking the question, “who contributes the most to building a positive point differential for a particular team?” we are effectively asking what players are on the court and how do we quantify positive point differential? We can either ask as a possession model and treat every possession as an observation. Or we can ask as a stint model and treat every stint as an observation. The goal is to select an interpretation and proceed accordingly.

In the classic sense of RAPM, analysts tend to use stints as observations. A stint is a period of time that the same set of ten players are on the court. Anytime a substitution is made, the stint ends and a new stint begins. For each stint, the number of possessions are counted. If this possession count is zero, then the stint has done nothing on the court and is not considered an observation.

If a possession is logged, then we obtain a viable stint and start recording the point differential. Recall that a possession is ended by a converted last free throw, made field goal, defensive redound, turnover, or end of period. For this exercise, we ignore end of period possessions.

Start With the Players

Before we even begin, we start with the players. We need to know who is available before we begin. To this end, we can simply walk through every game and build a dictionary of players.

Screen Shot 2017-09-18 at 1.20.20 PM

Once we know how many players we anticipate in all the games, we can initialize the data frame to pull in each stint.

Screen Shot 2017-09-18 at 1.20.37 PM

Now we march through every file, build stints and count possessions. Many folks like to do this using time. Instead, I partition on the type of play to obtain a smaller matrix and transition through the possession ending, and non-possession ending scoring situations.

Screen Shot 2017-09-18 at 1.24.31 PM

There are a list of columns of the data indicating the away players on the court and the home players on the court. I use these to indicate stints. Anytime this ten-some changes, I check to see if there are any possessions and then form the observation.

I initialize the stint using the first action of the game.

Screen Shot 2017-09-18 at 1.26.09 PM

Then walk through all shot/defensive rebound/turnover/free throw situation. First things first, as the next action occurs, I check the current ten-man rotation and see if it changes.

Screen Shot 2017-09-18 at 1.27.20 PM

If it hasn’t, we go right into checking possessions and counting points.

Screen Shot 2017-09-18 at 1.29.02 PM.png

We see the possession ending situations. If points changed for a team, we add them to the stint. In all cases, we increment possession counts.

Free Throws are Miserable. Not For Possessions But Stints.

We have to manage free throws. In order for a possession to end, the shooter must convert the final free throw of the bunch. In this case we check how many free throws a shooter gets and compare to which number of free throw they are on.

Screen Shot 2017-09-18 at 1.32.59 PM.png

Here we keep track of time as substitutions can be made between free throws. If this is the case, we cannot write the stint observation until all free throws are concluded. This is due to free throws being awarded to the stint prior to changing.

Unfortunately, if a team substitutes, and the last free throw is missed with an offensive rebound; we double count the possession. This is the nature of the fact that we either divide by a slightly larger number or throw away points. We could get tricky and count the possession as one-half possession for each stint. But, I didn’t do that here.

Paying close attention to the free throw block of code, there are really two sub blocks. The first block check to see if the free throw is the last free throw. If it isn’t, we just check score. The second block is if the free throw is indeed a final free throw, if it is possession ending.

Substitutions are Made

When substitutions are made, we have to check free throws first. This is the only situation where a previous stint gets credit for points when they are not on the court. To ensure the stints are indeed correctly called out, we simply asked when times of free throws are taken. Since time cannot advance on a free throw, the time matches and we know to add the points to the previous stint.

Screen Shot 2017-09-18 at 1.40.42 PM

Screen Shot 2017-09-18 at 1.41.04 PM.png

We perform the same calculation for when free throws occur immediately after a substitution. If this time matches, then free throws are added to the stint. This is due to time outs after fouls.

Save Off the Stint

Now that the free throw situation is over, we can save off the stint.


Screen Shot 2017-09-18 at 1.43.51 PM.png

Before we move on in the file, we check if this new line-up has any non-free throw activity. And start populating the new stint.

Screen Shot 2017-09-18 at 1.45.06 PM.png

Voila, It’s Done!

After iterating through all 1230 NBA games, we obtain a stint matrix where the response is point differential per 100 possessions. This results in slightly over 34,000 stints for the 2017 NBA season. Now we are able to proceed with APM and RAPM.

Adjusted Plus Minus

Recall that if we performed straight-linear regression, we will have an uninvertible matrix. To perform this task, we trawl through and located all players who played less than x minutes. In this case, we eliminate all players who played less than 125 minutes; and then again with less than 250 minutes.

Sorry, I cut off the names of all the players. It’s a long list.

Screen Shot 2017-09-18 at 1.49.35 PM

Now we can run the APM model, which is just a regression.

Screen Shot 2017-09-18 at 1.50.46 PM

The summary yields:

Screen Shot 2017-09-18 at 1.51.38 PM.png

We see that the variance is still too large for all the players; as indicated by the associated p-values. This is no different than in the example with our last post. Despite this, let’s take a look at the top 20 players:

Screen Shot 2017-09-18 at 1.54.03 PM.png

In case you can’t zoom in, that’s (in order):

  1. Damien Lillard
  2. Cody Zeller
  3. George Hill
  4. Stephen Curry
  5. Beno Udrih
  6. Bradley Beal
  7. JJ Redick
  8. Klay Thompson
  9. Lou Williams
  10. Darren Collison
  11. Harrison Barnes
  12. James Harden
  13. Paul Millsap
  14. CJ McCollum
  15. Karl-Anthony Towns
  16. Kyle Lowry
  17. LeBron James
  18. Devin Booker
  19. Nikola Jokic
  20. Dirk Nowitzki

Now, mind you, while this list is not surprising; there are some questionable additions such as Beno Udrih and Cody Zeller. Take that back, I thought Zeller was questionable. One team exec told me “that pretty reasonable.”

Despite this, the r-squared (no relation to squared 2020) is an abysmal 0.0245. Compare this to Rosenbaum’s equally abysmal 0.15 in his studies. Sure, we get answers, but the multicollinearity is so high, there must be further action taken.

Regularized Adjusted Plus Minus

So let’s move to ridge regression. In this case, we need to identify a proper lambda. To do this, we turn to scree plots! A scree plot is a cross-validation tool that identifies predictive performance of a model. Cross-validation, in turn, is a method for splitting our data up into a test set and a train set. We build the regression model on the training set and apply it to the test set. By walking over all possible combinations of test and train sets, we obtain a jackknife measure of error associated with the model. We perform this for every lambda in ridge regression and we obtain a function of lamdba and jackknifed error. Plotting this is a scree plot.

Screen Shot 2017-09-18 at 2.05.21 PM

Screen Shot 2017-09-18 at 2.04.49 PM

Ideally a scree plot will have a convex shape, so we can find a minimum. In many cases, this never happens. Therefore we look for areas that flatten out. This is entirely subjective.

Taking a look at the jackknife error, we don’t see much variation. Hence we will take at roughly lambda = 1000. This equates to column ten of the RAPM output.

Screen Shot 2017-09-18 at 2.09.57 PM.png

The ordering here is:

  1. Damian Lillard
  2. Stephen Curry
  3. JJ Redick
  4. Klay Thompson
  5. George Hill
  6. Bradley Beal
  7. Paul Millsap
  8. Lou Williams
  9. Cody Zeller
  10. Harrison Barnes
  11. LeBron James
  12. Darren Collison
  13. Kyle Lowry
  14. CJ McCollum
  15. Nikola Jokic
  16. Karl-Anthony Towns
  17. Devin Booker
  18. Kawhi Leonard
  19. Isaiah Thomas
  20. James Harden

Here we see names we expect with a couple exceptions: Darren Collison and Lou Williams. I’ll leave Cody Zeller alone this time. While these players contributed significantly to their teams successes (Sacramento’s big time run towards 8th in January before the Boogie trade and Lou’s carrying of the Lakers before shipping to Houston); they cannot be adequately viewed as top 20 talent in the league. Therefore we can perform further analysis.

Choose Your Own Adventure in Models

We only looked at point differential. What happens if we change point differential to rewards? Here, we replace points differential per 100 possessions with positive activity per possession. And instead of stints, we use possessions. Further, we apply a ridge regression. What do we obtain here?

Screen Shot 2017-09-18 at 2.17.15 PM.png

This is a similar list, with similar attributes; such as Solomon Hill climbing up the ranks. We could use these ranking lists and then perform a rank-aggregation analysis such as Kemeny-Young to obtain a listing that combines both positive activity on the court and point differential.

Next Steps…

We see that RAPM is a great improvement on plus-minus models, but many of the assumptions are failed to be met. Ignoring this, we still obtain understandable; somewhat non-iterpretable values that adequately measure player contribution.

Advanced models build off this and investigate multi-liner models, where player statistics are incorporated. Some use different definitions of responses, like we summarized above. Some use different input schemes. Whichever the methodology, the process is almost identical: build the data set, identify characteristics of the data to perform model assessment, and test the model.

That said, what do you have in mind for a model?


One thought on “Deep Dive on Regularized Adjusted Plus Minus II: Basic Application to 2017 NBA Data with R

  1. Pingback: Deep Dive on Regularized Adjusted Plus Minus II: Basic Application to 2017 NBA Data with R by Justin Jacobs | Advance Pro Basketball

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s