Friday, December 21, 2007

Subtlety and Context in Interpreting Volleyball Stats

Over at the VolleyTalk discussion website, user “38 Skynyrd” started a topic the evening of December 15, after the Penn State-Stanford NCAA title match, about volleyball statistics. The initial salvo in the discussion essentially argued for more subtlety and context in interpreting volleyball statistics, given that:

“…absolute raw numbers in the box score do not always tell the true story of how good or bad a player did in a given game or match.”

The full set of messages is available here. A number of suggestions were made by the discussants for new statistics. Given the obvious relevance of this discussion for our mission here at VolleyMetrics, I have excerpted a number of these ideas (with the author of each one credited in parentheses). These are shown below:

“…a hitter may have hit for good numbers overall, but if they made 5-6 hitting errors at critical points in the match, then their overall hitting percentage may still look good in the box score, but they still had a crappy match.

There are also a lot of stats that aren't tracked in the box score. Missed blocking or defensive assignments, poor choices in out-of-system plays, crappy free ball passes. A blocker could not block a single ball during an entire match, yet that player's coach could praise her for having the most incredible blocking match of her career if she made every single blocking assignment and move correctly, and took away what she was assigned to take away, and the back row defenders end up with 300 digs” (38 Skynyrd)


“…a blocked attack is not the same error as an outright hitting error. And there's no way to tell what a "0" attack led to. Did it throw the opposition out of system or did it not stress them at all? Was that hitting error caused by the block?” ([R]uffda!)

“One of the easiest ways to stat an individual player's performance during a match is to assign a +1, 0, -1 scale to each touch that they get on a ball. For instance,

+1 = excellent pass
0 = marginal pass (i.e. passed to 10-foot line)
-1 = bad pass, aced or shank

+1 = kill
0 = ball kept in play by opponent
-1 = hitting error (blocked, out-of-bounds, into net)

If [you] grade each touch a player makes [and average the scores], by the end of the match you'll have a score that is between -1.0 and +1.0.

It can help judge an individual players performance better, but it's not weighted for situational dynamics (i.e. a +1 contact when the score is 0-0 should not be weighted the same as a -1 contact when the team is down 23-29). It also doesn't address bad plays when the player is not touching the ball - such as out-of-position on defense, missed blocking reads, getting in another player's way on the court, etc.” (38 Skynyrd)


“…there are still important many areas of play which are not covered in the stats for volleyball. IMO [In my opinion], the most important of these are passes which do not hit the target, missed blocking assignments (i.e., failure to close the block) and missed digs in the back row” (Traveler5)

“One stat that is completely objective and I think could be useful would be what % of a player’s attacks are handled and then killed for a point by the other team. So if the other team digs your attack, and winds up getting a kill on that 3-touch sequence immediately following your attack, that would count negative, and if they don’t score, either by sending over a freeball or having a legit attack dug up, it counts towards you…

You could also… break down how many blockers were on each of a player’s attacks, and how well they hit against 0, 1, 2, and 3 blockers. This could also be used by setters to see the average number of blockers their hitters had to go against.

Front row vs. back row would also be nice, since a player good enough to get a lot of back row swings is probably going to hurt their % some since that is a lower percentage shot” (Chance)


Suggestions for refinements of dig statistics: “…opportunities matter. Thus, I have calculated the "dig %", which is digs/non-error attacks (it doesn't make sense to penalize a team for not digging a ball that is blocked or out)… [and] digging compared to opponents… difference between digs and opponents’ ” (p-dub)

I and another user, Mike Garrison, both alluded to improvements in baseball statistics that have been fueled by the “sabermetric” movement. Some responses:

“Baseball is helped by the fact that it is such a discrete game. You are either out or safe. You are either at first, second, or third base. A pitch is either a ball or a strike. Etc.

The weakest part of SABRmetrics is the fielding stats… A player can just barely miss a ball or not really come close to it at all…

Most other sports, including volleyball, are more like baseball fielding than hitting or pitching. So I would start by looking more at fielding stat methods than at hitting or pitching methods.” (Mike Garrison)


“Even more important is the independence of events. Whereas the events in baseball are not completely independent, they are far closer than any sport. If a batter is batting with a runner on first, you generally know where that runner is going to be at any time... None of this can be said for volleyball (or most other team sports). Players are all over the place, and offensive sets can vary significantly.

The best example of baseball's stats independence is in offense and defense. Baseball (and like sports) is the only sport where a great defensive play does not improve offensive opportunities. In all other sports, the defense can help the offense. In volleyball, great defense can slow the opponents attack, and we always hear about "transitioning." Shoot, the defense can even score directly on a block...” (p-dub)


“[Unlike baseball, where the pitching rubber and batter’s box constrain the positions of the key players...] Volleyball is a game where the ball is struck from almost anywhere on the court, and there isn't enough information available…

My biggest pet peeve of the [n]umber crunchers are those who tout ridiculous stats or won-loss records without regard for the average competition faced” (Bear Clause)


Where feasible, I will consider gathering the data to produce the statistics suggested above. I invite you readers to do the same!

4 comments:

Anonymous said...

Thanks for posting this. They are rare, but we do on occasion have some very thoughtful discussions about analysis on VolleyTalk. Unfortunately, too often it ends up with "wouldn't it be nice if we had ..." instead of "here's what we know."

BTW, I have heard that Shane Reece from BYU presented an analysis based on a "touch-by-touch" database they have created at this year's American Statistical Association meeting. Man, wouldn't that be useful? It would kind of be like what Stats, Inc has done with baseball, but more impressive given the amount of action.

Anonymous said...

Alan - I know this is old news, but it is relevant.

Our VT discussion got me looking into digs a little more and I made an interesting discovery about dig correlations. The strongest correlation I found for digs was with ... opponents digs! Correlations (r values) for the Big Ten, Big 12, and Pac Ten were in the range .65 - .75, and the overall correlation for all three conferences is something like 0.77!

Ruffda pointed out that the way to get lots of digs is to have long rallies, and that means that both teams get lots of digs. That means a team like Penn St, who doesn't get dug very often, doesn't get a lot of chances to get digs of their own. As a result, they are near the bottom of the NCAA in digs/game ("292 out of 325"). Then again, they are at the TOP of the NCAA in terms of getting dug, and by a good amount (the closest team I have found so far allows 1 more dig/game).

From top to bottom, there is a difference of about 10 digs/game among D1. However, the difference between a team and its opponents is typically less than 3 (the max I have found is 3.3, and most are much less). So Penn St is near the bottom in D1, but their opponents are even less (and probably would be the bottom if they were a separate team). Similarly, whereas Chatanooga lead the country in digs at 22.6/game, their opponents, where they a separate team, would be second in the country at 21.5 or so, behind only Chatanooga.

Interesingly (at least for me ;)) is that there is basically NO correlation between Dig % (digs/(opp attacks - opp errors). Unfortunately, this is only really applicable at the team level, and I don't think it translates well to individuals. Perhaps the way to do that is by a WinShares-like approach?

Anonymous said...

Here is something I have posted on the Yahoo Big Ten discussion group:

Given our recent discussion of digs, I have thought about a new
approach for thinking about digs stats. Recall that I have found
that digs/game at the team level has a problem in that they are
strongly dependent on your own team's ability to not be dug. As
such, Penn St ranks very low in digs/game, despite the fact that they
are not in the least among the worst in the country in floor
defense. This is for the most part NOT attributable to their
blocking, as blocking doesn't correlate near as well with digs.

Alternatively, I have proposed the "dig percentage", which is digs/
(opp attacks - opp errors). This is a means for taking into account
dig opportunities at the team level. There are some differences in
team ranks of digs/game and dig %. For example, the top 5 in
digs/game (from the big ten website) are

1. Michigan 18.0
2. Purdue 17.9
3. Northwestern 17.0
4. Minnesota 16.1
5. Illinois 15.3

Penn St is last, at 13.6

However, in terms of dig %, the top 5 are

1. Michigan
2. Northwestern
3. Illinois
4. Purdue
5. Mich St

and Penn St is 7th. So much of Purdue's high dig count can be
attributed to the long rallies.

Now, a problem with dig % is that it is restricted to the team
level. We have discussed some sort of individualized stat trying to
account for who is responsible and assigning errors, but that
requires more detailed stats than we have. Instead, I propose a new
approach, what I call "Dig Shares." The concept is based on James's
baseball "Win Shares", and what I am basically doing is partitioning
the team's success among the individuals. Thus, for example,
Michigan's dig % is 51.8%. Meanwhile, Stesha Selsky accounted for
33.1% of Michigan's total digs. Therefore, we give Stesha 51.8%*.331
= 17.2 "Dig Shares."

So that is where I started. However, I noticed something while
looking at the conference leaders. For starters, there is a pretty
strong relationship between the leaders in dig shares and those in
digs/game (with some deviation). Second, and most surprisingly, I
found that dig shares were, on average, about 3 times the digs/game.
So ultimately, I have just divided the dig shares by 3 to put them on
the same scale as digs/game. Therefore, numbers you might find good
for digs/game will also be good for dig shares.

Now let's compare how they came out. Here are the top 20 in league
in digs/game
1 Selsky, Stesha 5.95
2 Nobilio, Kate 5.69
3 Edinger, Ashley 4.78
4 Tan, Christine 4.70
5 Wack, Jocelyn 4.70
6 Miller, Kelli 4.63
7 Hiza, Emily 4.52
8 Pierce, Juli 4.11
9 Holehouse, Roberta 3.95
10 Mastandrea, Anne 3.74
11 Trogdon,Miken 3.56
12 Stevens, Ami 3.29
13 Merlau, Danita 3.21
14 Schatzle,Ashley 3.02
15 Anderson, Lindsay 2.92
16 Paulus, Courtnie 2.78
17 Zimmerman, Lexi 2.75
18 Meyer, Danielle 2.70
19 Calhoun, Kaila 2.70
20 Hall, Kelsey 2.69

And here's what I get for dig shares
1 Selsky, Stesha 5.72
2 Nobilio, Kate 5.58
3 Edinger, Ashley 5.07
4 Wack, Jocelyn 4.92
5 Tan, Christine 4.69
6 Holehouse, Roberta 4.64
7 Hiza, Emily 4.38
8 Miller, Kelli 4.19
9 Pierce, Juli 4.19
10 Trogdon,Miken 3.85
11 Mastandrea, Anne 3.39
12 Stevens, Ami 3.29
13 Schatzle,Ashley 3.27
14 Hodge, Megan 3.01
15 Merlau, Danita 2.90
16 Anderson, Lindsay 2.86
17 Hall, Kelsey 2.74
18 Paulus, Courtnie 2.73
19 Meyer, Danielle 2.71
20 Calhoun, Kaila 2.70

So the top 4 are the same, but there is some minor variation below
that.

Now, the part that blows me away right now is how closely the
digs/game and dig shares agree with each other. Among the top 20,
the average difference is 0.1!!! In fact, this is the general trend
for all players with significant playing time (more than 75 games).
The players that have the biggest deviation are those back row
players who have played more middle range of games (like Jessica
Ullrich for Purdue and Emma Pollert for Northwestern). However, I
have discovered that they fall back on the line if I adjust their dig
shares by the fraction of their teams games they played (dig
shares/fraction). If I do that, remarkably, their is almost perfect
agreement between dig shares and digs/game (r^2 = 0.985). That
agreement is too good to be coincidence. In fact, for some teams,
the agreement is almost perfect. Look at digs/game and dig shares
for Gophers

Tan, Christine 4.70 4.69
Dieter, Brook 2.60 2.59
Hartmann, Rachel 2.22 2.22
Cowles, Hailey 2.22 2.21
Roysland, Kelly 2.08 2.07
Gibbemeyer, Lauren 1.02 1.01
Fallon, Kelly 0.79 0.79
Chin, Krista 0.75 0.75
Schneider, Caitlin 0.67 0.66
Hagerty, Rachelle 0.54 0.54
Roehrig, Kyla 0.45 0.45
Vatterrodt, Katie 0.45 0.45
Phillips, Charde 0.40 0.40
Jones, Jessy 0.35 0.35
Wilber, Michele 0.33 0.33
Schmidt, Kelly 0.03 0.03

These are almost exactly the same!

In the end, there has to be a reason for it. Give me some time to
sit and think about it and I will figure it out. It probably has to
do some team level stats (note there is a factor of 33 that shows up
in my dig shares (100/3) - coincidentally about the number of matches
played? I don't know).

Here's the short answer. This work shows that

1) We can refine the dig numbers for individual players, basing them
on dig %, which is not affected by absolute dig numbers

2) For the most part, it's not necessary. Digs/game for full time
players does a pretty darn good job of reflecting the dig shares.
Yes, there are some players who are slightly off - Roberto Holehouse
finishes 9 in the conference in digs/game instead of 6th, but
overall, the agreement is good.

My next thought to is to throw out the non-dug attacks. Perhaps dig
% should just be digs/(digs + opp kills)? Why should teams be
penalized for not digging balls that the scorer doesn't think needs
to be dug? Actually, that could go into the success side, maybe...

I'll let you know what I come up with.

Anonymous said...

Re: the relationship between dig shares and digs/game

The factor of
33 is opp Att/game - opp Err/game. The league average is 32.9.

If you break it down, the dig shares turns out to be the following

dig shares = (digs/game)/(opp att/game - opp err/game)/33.3

Thus, if the team's opponents attacks - err/game is close to 33.3, then
dig shares will be very close to digs/game.

Minnesota 33.4
Ohio St 33.3

QED

If the (opp attacks - opp errs)/game is more than 33, then the dig
shares is less than digs/game. If it is less than 33, then dig shares
is more than digs/game.

Purdue 36.8
Penn St 28.4

Cool. It all works!

2023 NCAA Women's Preview

Sixth-four teams are alive at the moment, but it sure looks like Nebraska (28-1) and Wisconsin (26-3) will meet for a third time this season...