In this fourth article in the series, we will examine monitoring the results of the change process.

There’s an old management saw that says “you can’t manage what you can’t measure.” Although overused and frequently abused, there’s some truth to the old saying.

As technical professionals, most of us are both graced and cursed by the love of precision, and if something cannot be measured precisely, we dismiss it. Too often we like to pretend that things that feel vague simply don’t exist at all. So we use the old “can’t manage/can’t measure” as an excuse for failing to pay attention to “soft” issues.

This is one of the key reasons that technology organizations and projects are frequently troubled with productivity, morale, communications, attitudes, and politics in need of improvement. We all know that these things are real, but since they can’t be measured, we decide that they can’t be managed. So we become the victim of the unseen menace.

It is important to identify indicators or touchstones that can be monitored to gauge what effects the change efforts are having. For example, if you are a CIO and the directors who report to you can’t seem to make decisions without you, they constantly require your intervention to settle disputes. The improvement in the management team that you seek is to change their interaction to allow them to resolve their own issues. There is no specific measure of whether your team is wasting your time, but you can keep track of how much time you spend in meetings solving problems that you feel should have been resolved without you. Over time, your log, while not a true scientific measurement, should be a good indicator of whether your change effort has had the desired effect.

There are a number of ways that you can monitor these indicators. Selecting the appropriate method requires thinking carefully about the nature of the indicator, the cost of collecting the data, the frequency of monitoring the indicator, and the value of precision over simplicity. Methods of monitoring include:

1. Surveys - the old tried and true method. Surveys have many advantages: clear answers, inexpensive, and they offer the feeling of precision. The disadvantages include: difficulty getting responses, respondents often represent extreme viewpoints, and the quality of information is very sensitive to the quality of the questions asked.

2. Focus Groups – Although usually thought of as only a marketing tool, asking specific questions in a small group setting can be a helpful way to get information on what’s going on. Advantages: relatively inexpensive, better representation in sample population than surveys and unexpected information can be clarified and elaborated real time. Disadvantages: More expensive than surveys, results tend to be more qualitative than quantitative, and you can get a “groupthink” response in which no one in the group wants to contradict others, so just goes along with whatever others are saying.

3. Individual Interviews – If conducted by an independent third party, individual interviews can offer the most accurate and complete information about what’s happening in the organization. However, they are the most expensive way to gather data and also tend to result in qualitative information.

Of course, these can be used together as well. For example, surveys can be used to gather general information which is then confirmed and expanded through either focus groups or interviews.

Regardless of the method you choose, if you decide to put in the effort to monitor the progress of your change initiative, you are in the best possible position to understand the effects you are having on the organization and to maximize the return on your investment in organizational transformation.