On Selection Sunday, the NCAA Tournament committee will need to decide on which 36 teams should receive an “At-Large” bid. Many of these teams are simple to select; for instance Kansas will be at At-Large team thanks to their loss to TCU on Thursday in the Big 12 conference tournament. However, there are many “Bubble” teams that still exist; such as Syracuse.

Since Illinois State’s loss to Wichita State in the Missouri Valley Conference final a few days ago, people have been arguing about the inclusion of a second MVC team as an At-Large. This team is Illinois State. While Illinois State has lost every power conference game they played in this year and lost two out of three against a teetering “Top 25” team, it’s very difficult to rank them higher than a team who has wins against Top 10″ opponents and several power-conference wins. However, as the argument goes, Illinois State boasts a 31st best Rating Percentage Index (RPI). So if Illinois State is consistently between 50th and 65th in rankings across most accepted rankings methods; why are they so heavily favored using RPI?

Note: Due to the impending conference champions, a ranking of 46th is effectively the cut-off point for At-Large bids. That is, 10 conference champions reside in the “Top 46” for almost every rankings analytics.

Rankings Analytics Are Good and Bad:

Rankings analytics have existed for centuries; since Borda and Condorcet back in the 1700’s. Over the centuries, statistical rigor has been attempted in finding mathematical merit in describing “which item is better than the other.” Sometimes methods hit the mark (Bradley-Terry, Mallows, Plackett-Luce), some methods are honest attempts at rigor (Colley Matrix, Massey), and some methods are just wild guesses. The RPI is the wild guess. So let’s break down what RPI is.

 

RPI: Three Step Process

The RPI considers three factors in a team’s score: Winning Percentage (WP), Opponent’s Winning Percentage not including your games (OWP), and Opponent’s Opponent’s Winning Percentage (OOWP). The number of times you play an opponent, you count their percentage as many times.  These are weighted to sum to one:

 

RPI = 0.25*WP + 0.50*OWP + 0.25*OOWP

 

So let’s break down Illinois State’s RPI as an example to illustrate the methodology.

Step One: Winning Percentage

Winning percentage, since 2004, is weighted based on location. If you win at home, you get 0.6 wins. If you win on the road, you get 1.4 wins. If you win on a neutral court, you get 1 win. Similarly, losses are valued at 1.4 at home, 0.6 on the road, and 1 at home. This keeps the “average one win, one loss for every game” mentality with the classical win-loss record. There is no justification other than “teams tend to win two-thirds of their games at home” during the regular season. This is why you will see “Record, Road, Neutral, Home, Non-DI” with RPI scores. This breaks down that weighting.

For Illinois State, they were 27-6 overall with 9-4 road record, 3-2 neutral court record, 14-0 home record, and 1-0 outside of Division I. Let’s tally these wins and losses up:

• 9 road wins = 9*1.4 = 12.6 wins.
• 14 home wins = 14*0.6 = 8.4 wins.
• 4 road losses = 4*0.6 = 2.4 losses.

This means that Illinois State’s weighted record is 24 – 4.4; or a 0.845070 winning percentage. Since this is only a quarter of their RPI. Illinois State’s RPI is currently at 0.211268.

Step Two: Opponent’s Winning Percentage

In the previous step, we have no indication if Illinois State played Baylor 33 times or the Poor Sisters of the Blind 33 times. To attempt to discern this information, RPI includes opponent’s winning percentage. In this case, we have to do a little work. Illinois State played 19 of the possible 350 D-I opponents.

In this step, there is no weighting of wins and losses. Let’s go through their opponents; (L) indicates a loss to that team:

• Murray State (L) : 13-17 = 0.433333
• IP Fort Wayne : 16-11 = 0.592593
• Texas Christian (L) : 17-14 = 0.548387
• IUPUI : 12-17 = 0.413793
• New Mexico : 17-13 = 0.566667
• Tulsa x2 (L) : 14-15 = 0.482759
• UT Martin : 19-11 = 0.633333
• St. Joe’s : 11-19 = 0.366667
• Hawaii : 13-15 = 0.464286
• San Francisco (L) : 18-12 = 0.600000
• Evansville x3 : 14-14 = 0.500000
• Loyola Chicago x2 : 16-12 = 0.571429
• Missouri State x2 : 16-14 = 0.533333
• Indiana State x2 : 10-18 = 0.357143
• Southern Illinois x3 : 16-13 = 0.551724
• Wichita State x3 (L, L) : 27-3 = 0.900000
• Bradley x2 : 12-18 = 0.400000
• Drake x2 : 6-21 = 0.222222
• Northern Iowa x2 : 13-14 = 0.481481

Taken into account, we take the 32 records (double and triple counting the multiple games) and divide them by 32. This will result in an opponents winning percentage. For Illinois State we calculate:

0.433333 + 0.592593 + 0.548387 + 0.413793 + 0.566667 + 2*0.482759 + 0.633333 +

0.366667 + 0.464286 + 0.6 + 3*0.5 + 2*0.571429 + 2*0.533333 + 2*0.357143 +

3*0.551724 + 3*0.9 + 2*0.4 + 2*0.222222 + 2*0.481481

This is a total of 16.570965. Divide by the 32 teams/games and we have an opponent winning percentage of 0.517843. With OWP getting half the weight in RPI, this break down to an RPI of 0.258921 for Illinois State.

Combine OWP with WP, and Illinois State has an RPI of 0.470189.

Step Three: Opponent’s Opponent’s Winning Percentage

This step is a slight misnomer. Instead of actually tallying the opponent’s winning percentage, instead the OOWP is calculated by averaging the OWP’s of your team’s opponents. This is a rough approximation to a “second hop” analysis of your opponent’s opponents.

To give an example, let’s consider Illinois State’s road-block for the season: Wichita State. Wichita State played 21 of the possible 350 NCAA D-I teams. This is their OWP breakdown:

• South Carolina State : 9-19 = 0.321429
• Long Beach State : 13-17 = 0.433333
• Tulsa : 15-15 = 0.500000
• Maryland Eastern Shore : 13-18 = 0.419355
• LSU : 10-20 = 0.333333
• Louisville (L) : 23-8 = 0.741935
• Michigan State (L) : 18-13 = 0.580645
• Colorado State : 20-9 = 0.689655
• Saint Louis : 12-20 = 0.375000
• Oklahoma : 11-19 = 0.366667
• Oklahoma State (L) : 18-12 = 0.600000
• San Diego State : 18-12 = 0.600000
• Evansville x2 : 14-15 = 0.482759
• Loyola Chicago x2 : 16-12 = 0.571429
• Missouri State x3 : 16-13 = 0.551724
• Indiana State x2 : 10-18 = 0.357143
• Southern Illinois x2 : 16-14 = 0.533333
• Illinois State x3 (L) : 25-5 = 0.833333
• Bradley x3 : 12-17 = 0.413793
• Drake x2 : 6-21 = 0.222222
• Northern Iowa x2 : 13-14 = 0.481481

This amounts to a 16.654636 / 33 = 0.504686 OWP for Wichita State.

Note: At this moment Wichita State has a WP of 27 – 4 = 0.870968. Without OOWP included, Wichita State has .470084 for RPI while Illinois State is at 0.470189; despite Wichita State having a better record, playing harder teams, and winning multiple games over Illinois State.

Now, repeating this for the other 18 teams on Illinois State’s schedule, and we have the final piece with Illinois State’s OOWP being 0.509643. This accounts for an extra 0.127411 for their RPI; yielding an RPI of 0.597599. This is indeed 32nd best in the nation.

Got it? Now you’re a pro at calculating RPI! If you would like code on calculating RPI scores for teams feel free to shoot me a message.

OK… so how did Illinois State game the RPI?

Illinois State did not game the RPI as much as they fell into a gaming of the RPI. Many teams understand that RPI is critical for Selection Sunday as committee members do not understand pairwise comparisons of teams based on wins and the associated mathematics that has existed for centuries. That’s a damning statement and it sounds harsh; but to be fair, the committee gets most of the brackets correct when compared to more statistically sound models. A clear example came a couple years back when they selected UCLA on the “Eye Test” which sent many into a rage.  However the Bruins were in the last four teams according to many models; even broadcasted here.

Illinois State fell into the gaming strategy of win all home games, lose on the road and play all better teams on the road. For all other games, find middle-of-the-road to tougher non-power-conference opponents. With the removal of games against Illinois State, middle-of-the-road opponents turn into winning opponents. Similarly, tougher non-power-conference opponents will go .250-.400 against power conference opponents, but run the tables for a third or higher finish in their conference. This is exactly Illinois State’s schedule; minus the power conference opponents on the road. In fact, Illinois State only played on power conference opponent: TCU. And it was on the road.

How to Game the RPI

Using the strategy outlined above, we can break down how a team an inflate their RPI. It’s effectively a four-step procedure.

Step One: Be A Winner, Especially at Home…

This is the hardest step of all. You effectively can’t win every game. Kentucky tried. St. Joe’s tried. It hasn’t been done since the glory days of UCLA before advanced planning and training regimens dominated the current association. However, getting as many wins as possible can score you only up to 0.25 for your RPI score.

Step One B: …and Lose on the Road

Losing on the road is critical. Win at home at all costs. Take for instance three games:

Kansas, North Carolina, Poor Sister of the Blind

Suppose we play these three teams and have a choice of where to play them if we play the tough opponents on the road and lose; that’s a score of 0 out of 1.2. Lose to them at home, and we’re at 0 for 2.8. That’s a huge difference. Suppose we lose to the good teams and obliterate the bad teams. Poor scheduling would see us with a winning percentage of 0.176 (play all games at home) versus a good schedule yielding 0.538462 (play all games on the road). That’s a fairly violent swing for going 1-2 in both situations.

Fortunately, teams have to play a sizable chunk of their games at home. So let’s strategically game plan.

Step Two: Book Some Hard Opponents

Since RPI also includes opponents records, we don’t want to book teams that may end up being terrible. OWP is half the score. If we beat them, great! That reduces their loss total and makes them looks slightly better than their record. But we also need to make sure they do not tank everywhere else in the season. If we manage to book one team that will go roughly 27-3, then we eat their 0.900000 WP. Given this, honest scheduling would expect an OWP of about 0.500000 across all schedules. Given one game against an 0.900000 WP team, we could play a really really bad team with 0.100000 WP; or play three more teams with 0.366667 WP’s. Think about that for a second. Seriously…

Scenario One: Illinois State plays Wichita State once (0.9 OWP contribution) and they can play the bottom of the Southland Conference. This is a 0.500000 OWP total contribution. They are effectively guaranteed a 1-1 record. Book those games both on the road and we have an RPI of 0.425000 without considering OOWP. They just need their opponents to play hard enough teams that win roughly two-thirds of their games. That’s not too far-fetched; but difficult to dictate with a cruddy 0.100000 team in the mix.

Scenario Two: The better strategy is instead play Wichita State and pick up FIVE relatively decent teams; each with 0.42 OWP contributions. These teams are 12-18 teams. There’s still a relatively high chance of beating these teams if we are a bubble team. Let’s make them all from mid-major conferences, so their OOWP is relatively low. If all are on the road, we can expect our 5-1 record and maximize the the RPI at 0.921053 for WP and 0.500000 for the OWP to give an RPI of 0.480263. To get to Illinois State’s current RPI of 0.597? These teams only need to play sub-.500 teams. That’s it. How many times did Illinois State play Wichita State? Three times? We’ve just dictated 18 of Illinois State’s 32 games.

This second scenario leads us into…

Step Three: For Every Tough Team on the Road, Book a Beatable Road Game and a Near-Guarantee Home Game.

This step is where the art of scheduling comes into play. Suppose we book Kansas. Don’t book them at home unless we are overly confident in beating them. That’s rare. So play it safe and book them on the road. Why book them? They play strong opponents that boost your OOWP and OWP (step two). You get a 0.6 loss contribution (step one). This is a best case scenario for a non-power conference team. Even if that team goes .500 or so like Michigan State; they play a lot of OOWP boosting teams.

For every Michigan State you book on the road, you get 2-3 games of play. Want a great example of Illinois State? Look at TCU and their Big 12 boosting OWP and OOWP.

So not only Illinois State gets 5 games to play with for every Wichita State game; they also get an extra 3 games to play with from TCU. And oh yes… TCU was played on the road.

Step Four: Hope No One Notices the Scheduling Tactics

This final step is the obfuscation part of gaming the system, if a team is malicious in gaming the RPI; or the portion where RPI apologists say “take care of business with what they are given.” Either way, this step is critical in the entire RPI process.

Notice that we never really mentioned who they beat; but rather than the win-loss records of who they play and the win-loss records of teams those teams play. That’s it. Since each team plays approximately 10-14 games out of conference, we do not get to see how all the teams are connected using RPI. Furthermore, with how little the OOWP contributes (see all steps above) we just need to find a couple GREAT teams to lose to on the road and fill the rest up with sub-.500 teams. Where do those sub-.500 teams need to come from? Doesn’t have to be power-conferences! They can pile up a bunch of 14-18 teams from middling non-power conferences such as Conference USA, Sun Belt, Western Athletic, Summit. These are teams with relatively terrible RPI’s. Just “take care of business” with these teams and let steps one through three take care of the rest.

This is Illinois State’s entire resume in a nutshell.

 

Weaknesses of RPI

So you have seen how the RPI can be gamed and how we just dissected the entire Illinois State schedule to show how their weak schedule can be tailored into a high RPI without ever winning against a power-conference team of have a winning record against tournament-bound teams.

The RPI fails to understand the intricacies of scheduling and therefore throws away information relative to actual 5000+ game results. Apologists state that “UNLV versus UTEP should not influence what Illinois State does.” The problem is it does. That’s like saying that Illinois State defeated Hawaii and Hawaii beat Cal State Northridge. However, Cal State Northridge beat Long Beach State; who in turn lost to Wichita State; who beat Illinois State. How does Illinois State compare to Long Beach State? RPI says throw away Cal State Northridge versus Long Beach State. See the problem? Information lost.

This is the chronic problem with RPI. Add in the gaming aspect, and RPI becomes quite meaningless when comparing teams that are within plus-minus five spots of each other.

 

RPI on Steroids: Colley Matrix Method

Roughly 15 years ago, Wes Colley introduced the “Bias Free” Matrix method for ranking teams. Don’t worry. It’s definitely biased; we’ll get into that shortly. 

Colley uses a simple pairwise Bernoulli (Bradley-Terry with only main effect) rankings model and attempts to generalize it using a clever trick on the Multinomial distribution (Bradley-Terry as well). What’s the clever trick?

Number of Wins = 0.5*(Number of Wins – Number of Losses) + 0.5*(Number of Wins + Number of Losses)

Note: Number of Wins plus the Number of Losses is the total number of games, N, played. So the second half of this equation is 0.5 times a ‘ranking’ of 1/2 added N times.

Colley proposed changing this 1/2 to the teams actual ranking. He then performs the mathematics to show that we get a very nice matrix equation that can be solved. Using this method, Illinois State is actually 40th overall with a score of 0.8177. This puts them into the NCAA tournament, by the way, with the 6th to last spot.

Colley proposed this to be biased because in the Binomial case (only two teams) we have a posterior model that has an unbiased expectation. This is not the case in the engineered solution. In fact, while third, fourth, and all hops across every team is considered; the biased is put in to that sneaky equation above.

 

Part One: Number of Wins – Number of Losses is Effectively Win Percentage

In the matrix equations, we will eventually divide by something of the size of N. Hence the first part of this equation: Number of Wins – Number of Losses is equivalent to WP. Let’s break this down. 0.500 equates to a value of 0. A WP of 0.25 equates to -2/4, -4/8, etc.

By multiplying this part by 0.5; we are now weighting the WP with 0.5 instead of 0.25 as in RPI.

Part Two: (Number of Wins + Number of Losses) / 2 = Sum of ‘Rankings’ is Actually Iterative OWP, OOWP, and OOOWP, OOOOWP,OOOO…

This is where we call Colley ‘RPI on Steroids’. If you take the time to read the details of Colley’s work, you find that Colley merely engineers the ‘ranking’ and then iteratively injects rankings back into themselves across all teams and looks for a stabilizing solution for all rankings. The time this takes should be relative to the degree of connectivity across all teams.

Hence the first calculation is OWP. The first iteration is OOWP. Second iteration is OOOWP. And so on. These calculations are then weighted by 0.5. So this actually improves RPI and becomes what we call an ‘honest attempt’ at building a rankings model. However, it suffers from the same bias as RPI as in…

Part Three: Colley’s Method is Biased (0.500, 0.500); Just Like RPI (0.25, 0.5, 0.25).

Let’s take a closer look at that nifty equation from Colley again. Let’s set N to be the number of games played, Nw be the number of wins, and Nl be the number of losses. Then

Nw = 0.5*(Nw – Nl) + 0.5*(Nw + Nl).

This is a 50/50 split on WP and the resulting iterative OWP. But we could do other things… like instead write

Nw = 0.25*(Nw – 3Nl) + 1.5*( 0.5*(Nw + Nl) ).

This gives us a 25 percent weighting on WP while giving 75 percent weighting on strength of schedule. Immediately we see the 50/50 split is just because ‘why not?’ much like RPI was selected.

Thoughts?

Hopefully this evidence shows you how RPI is not as great of a criteria to use for determining teams. It’s still a measure, but not a strong one; nor even one based on statistical merit other than ‘I think this is a good idea.’

So what do you think?