Monday, October 02, 2006

Measuring the Unmeasurable

A theme I have seen over the years is "How do you determine whether your teaching/training is working?"

In my world - that is easy. How much time does it take you to do a task and how accurate are you?

However, for most topics, measuring success is a slippery thing. In a prior job, I had to facilitate customer service training. The students enjoyed being away from their desks (and getting a chance to vent), but I was never convinced that these trainings did anything to create long-term change in the desired direction and I had no idea how to measure that. Fewer complaints from patients? Increased worker retention?

I'm starting to see an anthropological approach to solving this issue of whether your training is working or not.

The paper that got me thinking about this is Most Significant Change (MSC) technique. Rick Davies and Jessie Dart designed this technique to allow international aid organizations track whether their programs worked.

The basic technique consists of collecting stories from the field and selecting some to send to the next group. The selection committee then explains to the storyteller WHY they selected the story (the feedback loop). This process is repeated with each level filtering the stories and explaining the selection to the level below.

The stress here is on cultural / social change. The intangibles that are difficult to measure using conventional techniques.

Brian Goodwin and Dave Snowden are also developing a measurement system using narratives. Their approach - from Snowden's description of it in this post - is more quantitative.

The debate over what the new knowledge economy will look like is worth the read.

No comments: