A Basic Hierarchical Model of Efficiency
 3 Comment
In my last post on retrodicting team efficiency, I set a general baseline that can be used help determine if a new model of team efficiency makes better predictions than a naive model. This is important, as we want to know if added model complexity is worth the hassle.
This post will present a basic hierarchical model of efficiency, and we’ll determine how much better this model is in terms of year to year predictions.
The Model
Like the classical adjusted plus/minus efficiency model, this basic hierarchical model for efficiency is constructed so that we make predictions about the mean number of points scored on a given possession between an offensive and defensive lineup. With this model, though, we consider all player’s offensive and defensive ratings to come from a normal distribution, where we consider two distributions: one for offensive ratings, and one for defensive ratings.
Mathematically, we can write this model as follows:
Where estimates the average number of points scored on a possession on the road, estimates the home court advantage for when the offense is at home, and and estimate the offensive and defensive ratings for each player, respectively.
Also, because we have to use a Bayesian analysis, all of the parameters and hyperparameters of this model are given noninformative prior distributions.
The Player Ratings
The tables below list this model’s estimated top 10 offensive, defensive, and overall players from the 0607, 0708, and 0809 seasons.
Rank  0607: Name  ORtg  0708: Name  ORtg  0809: Name  ORtg 

1  S. Nash  5.0  S. Nash  6.5  L. James  5.7 
2  D. Nowitzki  4.9  C. Paul  5.7  C. Paul  5.4 
3  C. Billups  4.7  K. Bryant  5.4  D. Wade  5.3 
4  G. Arenas  4.5  D. Harris  4.8  M. Redd  5.0 
5  L. Barbosa  4.3  D. Howard  4.6  L. Odom  4.8 
6  C. Atkins  4.3  D. Nowitzki  4.6  S. Nash  4.6 
7  A. Parker  4.2  L. James  4.5  K. Bryant  4.2 
8  D. Lee  4.1  K. Martin  4.5  P. Stojakovic  3.9 
9  M. Ginobili  3.9  P. Stojakovic  4.4  C. Billups  3.8 
10  K. Bryant  3.8  M. Ginobili  4.4  A. Nocioni  3.8 
Rank  0607: Name  DRtg  0708: Name  DRtg  0809: Name  DRtg 

1  C. Hayes  5.8  K. Garnett  3.9  K. Garnett  5.3 
2  B. Bowen  5.6  C. Hayes  3.6  A. Bogut  4.3 
3  D. Harris  5.2  R. Wallace  3.5  L. Odom  3.9 
4  N. Hilario  4.3  J. Smith  3.4  M. Camby  3.6 
5  T. Duncan  4.2  J. O'Neal  3.3  Y. Ming  3.4 
6  K. Garnett  4.0  S. Pavlovic  3.1  J. Przybilla  3.3 
7  R. Balkman  3.9  J. Przybilla  3.0  B. Wallace  3.0 
8  D. Mutombo  3.6  M. Ginobili  2.9  M. Jaric  3.0 
9  J. O'Neal  3.6  A. Johnson  2.8  A. Johnson  2.9 
10  K. Perkins  3.4  R. Nesterovic  2.7  B. Knight  2.9 
Rank  0607: Name  Ovr Rtg  0708: Name  Ovr Rtg  0809: Name  Ovr Rtg 

1  D. Harris  7.1  M. Ginobili  7.3  L. Odom  8.7 
2  T. Duncan  7.1  K. Bryant  6.2  L. James  7.3 
3  K. Garnett  7.1  S. Nash  6.2  C. Paul  7.2 
4  M. Ginobili  6.6  D. Nowitzki  5.9  D. Wade  7.1 
5  B. Bowen  6.1  P. Pierce  5.5  K. Hinrich  5.9 
6  A. Parker  5.9  J. Moon  5.4  K. Garnett  5.4 
7  C. Hayes  5.5  P. Stojakovic  5.1  A. Kirilenko  5.1 
8  S. Nash  5.4  C. Bosh  5.1  M. Bonner  5.0 
9  D. Lee  5.4  T. Young  5.0  R. Lewis  4.9 
10  D. Nowitzki  5.2  K. Garnett  4.8  Y. Ming  4.8 
As far as uncertainty in the ratings is concerned, we wouldn’t say that any one of these players is better than the other. Also, there are some cases in which we wouldn’t say that a given player is better than the average player.
Retrodiction Results
Obligatory top 10 lists aside, the real interest is in terms of predictions. Using the retrodiction methodology laid out in the post on retrodicting team efficiency, the ratings from the previous year’s model are used to predict the offensive, defensive, and net efficiency ratings of each team in the next season knowing game location and the 10 players on the court.
The following table lists the results:
Year to Year  Efficiency  Mean  Std Dev  MAE  MAE Diff  RMSE  RMSE Diff 

0607 to 0708  Offensive  0.02  3.73  2.91  1.79  3.66  1.22 
0607 to 0708  Defensive  0.58  3.55  2.64  0.97  3.54  1.34 
0607 to 0708  Net  0.56  4.87  4.03  1.96  4.82  2.25 
0708 to 0809  Offensive  0.72  4.45  3.01  0.97  4.43  0.56 
0708 to 0809  Defensive  0.21  4.57  3.61  1.39  4.50  1.76 
0708 to 0809  Net  0.51  7.01  5.01  2.19  6.92  1.65 
The difference in the classical model’s mean absolute error (MAE) and root mean squared error (RMSE) is listed under the MAE Diff and RMSE Diff columns. These are calculated by subtracting the classical model’s result from the hierarchical model’s result. In other words, negative numbers mean our predictions are doing better than the classical model.
Again, one area of interest is in the net efficiency. The graph below shows the predicted net efficiency versus the actual net efficiency:
Unlike the classical model, this model does not predict teams to have very large or small net efficiency ratings, as most fall within +/ 10 points. This is probably a good thing, since most of the actual ratings fall within this range as well.
Summary
Based on the results above, I consider this basic hierarchical model to better predict efficiency over the classical model.
There is, however, still room for improvement, as we again have some weird results where we predict a net efficiency of 5, yet the actual net efficiency is 15.
Randomness will always give us imperfect results, but this result seems like one that is worth investigating in an effort to provide insight into how we can construct a better model of efficiency.
If you enjoyed this post, use RSS to get notified of new posts.
3 Comments on this post
Trackbacks

Kevin Pelton said:
Good stuff, Ryan. Two questions: Does the assumed normal distribution explain why these numbers are smaller than we see with classic adjusted plusminus?
Second, who’s that outlier at the top of the chart that this method actually predicted to be a below.500 team?
September 7th, 2009 at 10:02 pm 
Ryan said:
The assumed normal distribution for player ratings keeps us from estimating ratings that are extreme (too big or too small) for players with small sample sizes. So this naturally keeps us from making extreme predictions for the future seasons.
I believe the outlier you’re referring to is the one I mention in the summary. It is the 0809 Bulls team, where predictions are only made for ~900 possessions since we ignore possessions from guys like D. Rose. I want to better understand why this collection of players did so well, and if we can explain it in any way.
September 7th, 2009 at 10:14 pm 
Ryan said:
I’ve constructed this spreadsheet that lists all offensive and defensive ratings from this model, with associated standard deviations and 95% credible intervals (which are constructed with the 2.5% and 97.5% quantiles).
September 9th, 2009 at 9:55 pm