How do you know if something is working? How do you evaluate something you cannot fully understand? How do you convey your intentions and expectations to others?
These are tricky questions that anyone who works in complex environment will encounter. One option is to be rigorously quantitative. It’s all about the numbers. Hundreds of pre-defined KPIs to be captured, collated and analysed. There is definitely value in formulating KPIs but this approach has drawbacks:
- Unexpected Outcomes: The numbers you define up front may not be the numbers that matter. For example, you may run a programme that aims to increase organizational collaboration – but its biggest impact is actually in staff retention. How would you know that with a narrow set of KPIs?
- Measuring the Intangible: Many organizations focus on the numbers that are easy to collect rather than the numbers that matter – e.g. number of courses run, number of reports produced. These numbers often do not describe the impact of these actions because impacts are hard to measure.
- Making It Meaningful: The numbers on the page may mean a lot to you but they are dry and dull to everyone else Even worse, the numbers probably do not provide a clear guide for action.
One solution to these issues is a technique known as Most Significant Change (MSC). MSC is a participatory monitoring and evaluation technique that uses stories. It was first outlined by Rick Davies in the 1990s for the development sector. It is not a replacement for quantitative reporting but rather a complementary qualitative technique.
The basic MSC process will be described in the next post.