Monday, November 26, 2007

"Natural Experiment" Compares How Texas Tech Hit Before and After Change of Setters


The women's volleyball team at Texas Tech University, where I'm on the faculty, just finished a Big 12 season that seems hard to describe as anything other than a disaster. After winning their conference opener against Colorado, the Red Raiders lost all 19 of their remaining Big 12 matches (Oklahoma State does not field a team in this sport, meaning that each partcipating school faces 10 opponents, twice each). In Texas Tech's final 11 matches, it only won one game (i.e., 10 times it was swept 3-0 and once lost by a 3-1 score). Above is a photo I took at Tech's home match against Iowa State, relatively early in conference play.

Among the misfortunes experienced by the Raiders, senior setter Emily Ziegler went out -- for the season, it turned out -- after the eighth Big 12 match with a foot injury that required surgery (here and here).

Replacing Ziegler for the Raiders' remaining 12 conference matches was Kourtney Dunnam, whose path to the Texas Tech starting setter's role was unusual, to say the least. After playing for one year at Wayland Baptist University, Dunnam transferred to Texas Tech, where she was the team manager for two years. Certainly, one must admire Dunnam's persistence in working her way up to a roster spot in a major Division I conference.

The research methodologist in me saw an opportunity to take advantage of this "natural experiment" (some might say "quasi-experiment") involving the Red Raiders' change in starting setters, a change that was brought about in more-or-less random fashion.

As shown in the chart below, there were eight opponents (shown in blue) whom Texas Tech faced once with Ziegler as starting setter, and once with Dunnam in that role. That aspect provides experimental balance. Less optimal for research purposes are the facts that (a) Ziegler's matches were in the early part of conference play and Dunnam's in the later part; and (b) identity of the setter was confounded with game location (i.e., for any given opponent, if Ziegler faced the team at home in Lubbock, Dunnam of necessity faced it on the road, and vice-versa).


Still, though, I thought it would be interesting to compare the Red Raiders' hitting percentages under the direction of the two setters, but only against common opponents (Dunnam faced Kansas State and Oklahoma twice each, listed in red above, the data from which are not included in the analyses).

As it turned out, Texas Tech's average hitting percentage was virtually identical against the eight common opponents, .145 under Ziegler and .141 under Dunnam.

What is also apparent from the chart is that the quality of the opponent seemed to be a much more important factor in the Red Raiders' hitting percentage. Nebraska and Texas tied for the Big 12 title, each with a 19-1 record. Against the Cornhuskers, Tech hit .075 and .079 in the teams' two matches, whereas against the Longhorns, Tech hit .047 and .034. Conversely, against the less-dominant conference opponents, the Raiders usually hit in the .10's and .20's.

For each of the eight opponents common to Ziegler and Dunnam, I then averaged Tech's hitting percentages for the two matches, and correlated this with the opponent's Big 12 winning percentage (the idea of comparing a team's hitting percentages against different opponents was suggested in the comments to my earlier posting on hitting percentage).

As shown in the plot below, the correlation was a robust r = -.70 (two-tailed p = .053). In other words, as an opponent's winning percentage went up, Texas Tech's hitting percentage went down. Correlation does not prove causality, but the result makes logical sense.


It would, of course, be a gross exaggeration at this point to say that the identity of a team's starting setter is irrelevant. This analysis was done on only one team, in one year, in a small number of games. Texas Tech's hitting percentages under Ziegler were already pretty low, so even with the less experienced Dunnam, there may not have been much room for them to drop further (i.e., a "floor effect").

If any of you readers know of further examples of a team abruptly switching setters midway through the season, please provide that information in the Comments section. I would be happy to attempt replications of the above findings with other teams.

1 comment:

Craig said...

Notre Dame switched from playing mostly 6-2 to playing mostly 5-1 partway through the season, but the Irish also didn't play any opponent besides the teams they saw in the conference tournament more than once — and that was just a repeat of the previous weekend.

It's sort of a change in setters, but I realize that there are so many other variables involved with the switch that it's unlikely to be of much use.

2023 NCAA Women's Preview

Sixth-four teams are alive at the moment, but it sure looks like Nebraska (28-1) and Wisconsin (26-3) will meet for a third time this season...