Alexis Lebedew on Evaluating Setters

I recently received an e-mail from Alexis Lebedew of the Australian Institute of Sport, bringing to my attention some of his writings. Lebedew's focus is the evaluation of setting, a skill that has gone relatively unanalyzed over the years. The statistic of a setting "assist" exists, but because it represents the number of balls leading to kills, it overlaps considerably with hitting statistics.

In a piece entitled, "A Reconceptualisation of Traditional Volleyball Statistics to Provide a Coaching Tool for Setting" (link), Lebedew proposes a way to rate the quality of sets by taking into account not just the spike attempt following the set, but also the pass preceding the set. In short, setters are most rewarded for making "lemonade" from a "lemon" pass. As Lebedew states more technically, "...the combination of a [high-quality] spike and a [poor] pass has the top Rating... within the ‘Excellent’ outcome."

In fact, sets can be graded on a scale of 0-12, based on combinations of quality ratings for pass and spike. Lebedew notes that coaches who are used to grading passing and hitting performances on a metric different from his own (e.g., rating hit attempts on a 3- rather than 4-point scale) will still be able to construct a meaningful scale for setting, although the top value may differ from 12.

Lebedew also attempted to validate his setting metric in two ways. He first showed that computer software designed to link passes and hit attempts within the same sequences to derive set attempts only rarely missed a set attempt when compared to video footage. Second, he charted teams' percentages of sets (games) won for different averages of setting proficiency. For example, teams won roughly 95% of time when their set quality averaged 9 or higher, roughly 90% of the time when it averaged 8.5 or higher, etc., down through roughly 55% when averaging 6 or higher on setting. Lebedew encourages coaches and setters to strive for setting-proficiency averages of around 7.5-8.

All of the data were from international beach volleyball, which qualifies the generalizability of the findings in some important ways. With two-person teams, of course, there's no way to assess the setter's savviness in choosing which hitting-eligible teammate to set (as noted by Lebedew). Also, at levels of play beneath international caliber, more realistic setting-proficiency aspirations than the aforementioned 7.5-8 may need to be established.

JQAS Article on Quality of Skill Performance and Winning Points

A recent issue of the Journal of Quantitative Analysis in Sports(Volume 6, Issue 2) contained an article by Michelle Miskin, Gilbert Fellingham, and Lindsay Florence entitled "Skill Importance in Women’s Volleyball." Access to articles is by subscription, but the journal has guest-visitor privileges for single articles.

Miskin and colleagues analyzed data for a particular women's Division I team (not identified by name) during the 2006 season. When the team played at home, play on its side of the net was videotaped and later coded. Serves, passes, and digs were rated by judges on quantitative scales (e.g., 0-to-5), sets were evaluated in terms of their distance from the net, and spike attempts were coded by area of the court from where they were hit.

Essentially, the authors appear to be looking at correlations (or associations) between characteristics and quality of skill performance, and likelihood of winning the point. As they state on page 2:

The importance score incorporates not only the impact of a specific skill..., but also the uncertainty associated with the performance... Thus, a skill whose association with scoring a point is less certain will be penalized when using this metric when compared to a skill where performance at a given level is more closely associated with a positive outcome.

The article throws a barrage of statistical terms at the reader (e.g., Bayesian analysis, Markov Chains, Dirichlet prior, Gibbs sampling, gamma distributions), some of which I was familiar with, but many of them not. Fortunately, the authors translated the complex statistical results into plain English recommendations for the team that was investigated:

1. Keep sets and passes away from the net.

2. Force the attack to the middle and right side if at all possible.

3. Devote a considerable proportion of practice time to transition offense.

4. Get to blocking positions more quickly following a serve.

Presumably, if a team wanted to apply the analytic tools described in the article in their full glory, it would need to hire a pretty high-powered statistical consultant (in addition to acquiring the videotaping and coding resources). Perhaps similar analyses could be done via more basic correlational and regression techniques, but I suspect that the resulting conclusions may be somewhat imprecise, compared to those from the fully sophisticated analyses.