An annual discussion that takes place roughly around the start of every NBA season is whether teams are “good” at perimeter defenders. This discussion arises due to spurious, early returns on defensive three point percentages. This year is no different as the Twitter feed becomes log-jammed with discussion about whether there is a “leave the cruddy shooter open” effect to if we are “seeing randomness.” And ultimately, for at least the past four seasons, the Boston Celtics are brought up.

Getting to brass tacks, the argument is this: Can we use defensive three point percentage to rate defenses? If you know me, you know my answer is, sort of: Taking the percentage as a global effect that completely describes capability is foolish; but assuming that there is no “let the weaker player take the shot” effect is similarly foolish: Figure out how quickly you will be promoted to spectator if you tell a staff that Steph Curry is no better a three point shooter than Russell Westbrook in game situations. The crux of the problem is: how do we quantify the effect of defensive three point shooting and say something intelligent?

Start Simple: Percentages and a Two-Sample Test

Let’s start simple. First, let’s look at defensive three point percentages from year to year.

There is only one team in the top ten in each of those seven years: Boston. In fact, they are in the top seven every year and .001 of being within the top five every year. Does this mean that Boston is the best perimeter defensive team out there?

The Athletic’s Seth Partnow takes this one step further by looking at “wide open” three point attempts:

Here we see several new teams crack into the top 10, such as Orlando, Utah, and Phoenix. Similarly, Boston disappears from a list entirely almost falls off another, indicating much more randomness in the success “wide open” attempts. This makes sense as “wide open” indicates little-to-no defensive pressure on a shooter.

Probabilistically speaking, we need to break down three point field goal percentages into a partition:

The above tables fill in two of these probabilities. Table 1 provides a raw estimate of P(3FGM), while Table 2 provides a raw estimate of P(3FGM | Wide). Ideally, we should consider wide open three point attempts not only in the context of efficiency, but also frequency. In the case of Boston versus Golden State in the 2016-17 NBA season, Boston yielded 1104 wide open attempts while Golden State yielded 1120. The difference is less than one per game. But yet Boston was 1.5% “better” than Golden State. Is this a manifestation of randomness?

The short answer is yes, for the most part.

The long answer is this: Let’s write Boston’s defensive three point percentage as (375, 1104) and Golden State’s defensive three point percentage as (397, 1120). If we make assumptions such as each wide open attempt has the same probability of conversion dependent on player skill, that no two attempts are related, and that the effects of fatigue, game state , etc. are negligible, then we can perform the basic two-sample test:

where:

to obtain a test statistic of -0.733 (p-value of 0.46). Hit this with an effect size calculation using the angular transformation and we find that its effect size is significantly weak: 0.03. This indicates that Boston’s dominance of opponent three point percentage is indeed a random effect.

If Boston is so random, then why are they always near the top of these lists? Of course, we made some pretty wild accusations of the data collection process; so we will need to address these assumptions later. But first, let’s get some order into our analysis.

Include Ordering

To understand one reason why Boston is in the top seven for every year, let’s first hypothesize that defensive three point percentage is indeed random and that Boston is purely lucky. Then, we should be able to write this out mathematically and derive the necessary probabilities. Of course, this means we must understand the order statistics.

Illustratively, let’s assume that every team’s defensive three point percentage is identically the same distribution. Then, the probability that Boston is the best team across seven seasons is derived as a simple counting argument as (1/30)^7; which is considerably small. But it happening twice as it did within the seven season span above, well that’s 1.1% of the time.

However, in all three point attempts, Boston is in the top seven every year. In this case, the probability of that happening randomly is

In this case, we are looking at less than one percent of the time. While this is still a fairly possible event, it’s a little uncomfortable to suggest “well that’s how randomness works!” Instead, let’s investigate the endpoints as opposed the nearest neighbors.

Boston vs. Washington (2017)

Let’s start by comparing Boston’s numbers to that of the worst team in the league for the 2017 NBA season, the Washington Wizards. From stats.nba.com, we find that Washington’s opponents shot 456 of 1114 for wide-open three point attempts. In this case, we obtain a test statistic of -3.38 (p-value of 0.0006) and an associated effect size of 0.144. This indicates a fairly small effect despite the significant difference. When we see this type of discrepancy, we suggest that there is indeed a difference identified through non-random chance, but the effect is not significant. Interpreted into basketball language, Boston opponents indeed shot worse than Washington’s opponents but this may be the one of spurious results that occur in experiments. Unfortunately, the bottom ten teams from the 2017 suffer this specific effect. This suggests that despite the “unimportant” interpretation of effect size, we find that there is indeed some underlying factor.

After three steps of analysis, we are left with the following picture: Boston is doing something to be a “better” team when it comes to “holding” opponents’ wide open three point percentages low. But there is clear randomness when it gets into the top ten (or even top 15, we didn’t belabor that edge).

Partition Further…

Now that we do see a significant difference in Boston’s defense than a portion of the league, we would wish to further partition attempts into something measurable, but yet tangible. One theory that floats around on Twitter (and some league offices) is that teams are adept at leaving “the right” shooters open. This theory makes sense at an intuitive level, but we must figure out how to quantify that. This requires player x open data.

Another theory that floats around is the location of a wide open attempt. This too makes intuitive sense as it has been shown that above the break threes are susceptible to lower efficiencies than corner three point attempts. For example, the league as a whole shot three percent better from the corners (.387) than above the break (.351). But breaking this down to open looks, we need location x open data.

A tertiary theory is that the type of attempt is a factor, as catch and shoot three point attempts are more efficient than pullup attempts. For example, the league as a whole shot five percent better using catch-and-shoot (.371) than pullup (.321). In this case, we need to look at shot type x open data.

So let’s look at them.

Next Steps: Location vs. Wide Open

The easiest effect is to look at the location of the three point attempt. Here’s we perform a basic query against the defender distance on the field goal attempt and plot relative to the region of the court. For the 2016-17 NBA Season, the Boston Celtics’ opponents had this type of distribution.

Immediately, we see there are some anomalies in the data. We can eliminate those events. Other than that, there’s not much to glean from this image; other than the three point line of the court drawing doesn’t necessarily match up with reality. Instead, we tabulate the corner and above-the-break three point attempts:

Immediately, we see that there is no effect, but we can test for the effect to be complete in our analysis. The associated t-test is 0.012 (.99 p-value) and the effect size is effectively zero. This indicates that, at a high level, shot location (corner vs. above the break) really is not much of a factor when it comes to wide open three point attempts.

Complex Type vs. Wide Open

When we shift our focus to complex type, we find a much different story. For starters, we work against the catch-and-shoot filter and assume all other types are non-catch-and-shoot. In this case, we obtain:

This looks significant, but it’s really not. The test statistic is 1.356 which leads to a p-value of .176. The associated effect size is stronger than in shot locations, but not still relatively weak at 0.13.

Compare to the Wizards

When we compare to the Washington Wizards, we find different results. For starters, the complex type is significantly different:

For the Wizards, the catch-and-shoot rate is much higher than that of the Celtics. We can perform a t-test for the sake of completeness; comparing catch-and-shoot efficiencies between Washington and Boston. In this case, the test statistic is -3.15 (p-value of 0.0005) with an effect size of 0.16. This suggests that the Wizards are either excessively unlucky or there is a weak effect being created by the Wizards.

Changing this to location-dependent:

Here, we see that Washington has a potentially significant difference in wide open three point shooting. Performing the tests above, we find that test statistic is 1.553, which has a p-value of 0.1205 and effect size of 0.11. This suggests there’s a fairly weak relationship between locations within the open space.

However, when we compare corner three shooting between Washington and Boston opponents, we find that there is a significant difference with a moderate effect size. This indicates Boston and Washington are doing something different; but what? It’s not complex type or location as those don’t have significant effects.

One of the first places we compare is when the attempts are occurring:

Here, we see the ebb-and-flow of Washington’s opponents when it comes to above-the-break three’s and corner three’s. The sharp cuts at zero and one are low probability events (only one or two attempts across 82 games) and the missing pieces are where no attempts where taken over the 82 game sample. An interesting effect is that Washington’s opponents improve their above-the-break shooting, albeit with larger variance, over the course of the game. We also see their opponents exploit wide open corner three’s during second-string lineups.

Boston sees a similar second-string effect, but that’s where the similarities end. In fact, the above-the-break percentages drop over the course of the game (negative trend) and corner three’s creep up towards the end of the game. The impressive segments occur in the first few minutes of each half. This indicates that starters have a distinct effect, the only question being: What kind of effect?

Further Steps

Unfortunately, when originally writing this piece, there was a couple items that came about that identified specified, nuanced trends in players and actions on court. Notice that we did not go into player versus openness. Some work has been performed publicly and for the most part it is consistent with research done behind closed doors. However, there is a distinct difference between top end shooters such as Steph Curry, Bradley Beal, and Damian Lillard versus Russ Westbrook, Nic Batum, and Giannis Antetokounmpo. One end of the spectrum, teams must keep gravity high; regardless the position of the ball. The latter set of players, yielding the open shot is not a problem, provided the spot is not where a player is most comfortable shooting, such as in Antetokounmpo’s case:

That said, it’s extremely difficult to predict where a player will be throughout the course of a possession as offenses have turned more into ensembles of actions that react against defensive schemes rather than orchestrated set pieces that looks to seize a match-up advantage. Therefore, trying to scheme defensively to leave a particular player open is a fool’s errand as it will become exploited by mismatches and moderately capable players will burn the team; such as the Marcus Smart phenomenon has shown time and time again.

A further point is that game-planning to “let a particular player shoot” effectively changes a 38-percentage shooter to a 32-percent shooter if done effectively. For this to become a successful endeavor and see a single point change on the scoreboard, a team would require 17 successful possessions. If one of those possessions go awry, the entire scheme is ineffective.

More importantly, a 32-percent open attempt from three still has a higher expected value than a 47-percent midrange attempt. Also, for fun, here’s a listing of the Top-40 NBA Players that attempted the most wide open threes in the 2016-17 NBA Season, per tracking data:

Notice that’s a lot of really good percentages for the most part.

So What Are the Causes?

From the above, we have found that Boston is a relatively robust team that could be explained as being a random chance team for hitting the top seven every year for seven years; but has a significant difference than teams such as the 2016-17 Washington Wizards. The difference, we found, is really not quite random and that the main difference is not really location of the court nor the complexity of the shot type.

When we looked at the player type, we do find there is a slight effect, but it’s irrelevant at the player level. The response is effectively: DON’T GIVE STEPH OR KLAY SPACE. All other players tend to flitter about 35-40% making any “let them shoot” defensive scheme fruitless.

So to give a little insight without selling the farm: the difference happens a few seconds prior to a shot attempt. Wide open pullup attempts are rare to begin with: teams average about two per game. So the primary effect is with respect to the physics of a field goal attempt. Players in catch-and-shoot position are already in shooter-rhythm. So what are the items that can disrupt a shooters’ rhythm?

One great present day resource to start investigating is here: https://twitter.com/inpredict/status/1254102840382898177

Tying It All Together

Ultimately, our goal was to better understand whether defensive three point percentage can be used to rank perimeter defenses. The effect of randomness suggests no, but the nuanced effects indicate other defensive schemes. For instance, the above section looking at physics is tightly related to paint presence. That is, disrupting drives and battering passing lanes influence shots much more than leaving players open or funneling them to other open spots along the perimeter.

Given the above, note that we only looked at one small piece of the puzzle. Perimeter defense also looks to eliminate wide open perimeter shots without giving up rim attempts. But that’s for another day.