The October/November issue of the AVCA's Coaching Volleyball magazine features an in-depth article by Iowa State women's coach Christy Johnson, entitled "Taking Your Setter from Good to Great: Seven Qualities for Which to Strive." Many of the ideas in Johnson's article are amenable to data-recording and quantification, perhaps in ways that some teams are already tabulating for their internal statistical purposes.
Here are some examples, quoting from Johnson, that I feel potentially could inspire objective grading systems for evaluating setters.
"Great footwork allows your setter to contact the ball high in the middle of the forehead every time."
"If a hitter can take a great swing at the ball, then the setter has done her job..."
"To train this concept, I’ll toss balls all around the court. I’ll ask my setter to set a quickset, for example, on every set she can, unless she feels she can’t put up a good ball, in which case she should then set the outside set (or backrow set or backset, whatever she can do at that point). I want her to learn her range, and even though we’ll continue to work on expanding that range, she needs to understand what she is physically capable of."
"If our outside is hitting a high ball, the setter can afford to set her off the net a little bit. She’ll have time to adjust, and there will likely be two blockers waiting for her anyway, so better to keep her off the net. Middles need a ball that is traveling towards the net. They don’t have time to adjust to an off set, and they are often going against only one blocker, so we can keep their sets a little tighter."
Setting does not, of course, occur in a vacuum. Accomplishment of some of the above tasks will depend on the quality of passes a setter receives and the hitting abilities of the players she sets. If there were some way to record reliably the locations of suboptimal passes on serve-receipt, then different setters could be compared on the percent of time they put up a hittable set from a given location. Such attempts to quantify setters' range for "rescuing" errantly passed balls would parallel efforts among baseball analysts to quantify fielders' defensive ranges (see here and here for baseball examples).