Don't Be Average

Most managers are not expert statisticians so we let ourselves believe that averages are the way to go for metrics. Want to capture how long it takes to close a deal? Compute the average sales cycle. Want to see whether calls are answered promptly? Measure the average speed of answer. Want to check individual productivity? Calculate the average calls per rep per day. Want to know how long it takes to resolve support issues? Measure the average time to resolution. The problem with averages is that they may meet the criteria of ease of collection and ease of understanding, but they may not be very meaningful. Let's take a simple example from an inbound call center (either telemarketing or support) that uses average speed of answer as its main metric. The target response time is 30 seconds. On day one, the center gets 100 calls. Half the calls are handled within 5 seconds and the other half in 45 seconds (significantly over the target). The average speed of answer on day one is 25 seconds, which is under the target of 30 seconds. This is completely misleading since fully half of the customers experienced a significant delay compared to the target. Contrast this situation with day two, where the opposite problem with averages surfaces. On day two, 100 calls are received again. All calls are responded to within 25 seconds except for one call that waits for 9 minutes (suggesting a very patient caller and most probably a snafu in the rep scheduling algorithm). The average speed of answer is just above 30 seconds, dragged there by the one slow response and signaling a problem even though only one customer was inconvenienced—and, more importantly, even though the day one average did not show a problem! One can build similarly disturbing examples around customer satisfaction, cost of sales, or revenue by customer. So what's the alternative to averages? The solution is to set targets and to measure against the target. In the call center example, measuring achievement against the 30-second hold time target would yield a score of 50% on day one and 99% on day two, a much better indication of customer experience. Measuring achievement against targets is fine even if the customer does not know what the targets may be. For instance, telemarketing call centers rarely tell their customers what their target hold time is. That's fine; you can still set a target and measure against it. In some areas it doesn't make sense to set a single target, or you may not know enough to define a meaningful target yet. Try using several levels of target achievements and measuring against the multiple targets. For instance, if you are measuring the length of the sale cycle, try computing the percentage of deals that take less than a week, a month, and less than a quarter (or whatever time units fit your particular situation). The rule is: be wary of averages. Use achievement against target instead whenever a target can be defined.