What to keep in mind while designing your measurements

Out of the many good advice about measurement systems for software processes, I have benefited most from these four pieces of advice:

1. Set the goal before you measure.

It is tempting to set out a range of measurements, each costs some good time in calculation, while you don't need it. It is also very common that a project manager or a process engineer says: "we will need them later on, so why not collect them anyways?", another third idea is that "collecting many measurements may give you insight about the deficiencies in the project or the process, so the more you collect, the more insight you gain"!

I have lived with these philosophies for a while, and spent so much time collecting lots of measurements just for the sake of measuring, and I really think it was pointless.

The strategy I'm following right now is to set up the next objectives (one or two, at maximum), design the measures which I really need, and implement the infrastructure which will help me collect these measures automatically.

2. Distribute measurements results and analysis to the development team

Results of measurements may be very interesting for the team to visualize. Not only interesting, but also motivating. As a practise I continuously follow, I send the results of the measrements and analysis to the team, and discuss with them their implications. This helps me to:

• Motivate the team to work towards achieving better measurements.
• Motivate the team to record measurements more accurately.
• Get the team to the point that they trust the value of the measurements as a means for project management, and as a means for process improvement.

3. Minimize the time of collecting measurements.

This is a point which is very sensitive. If the cost of collecting a measurement is high, it is more likely that the whole measurement system will collapse! Developers like to develop, not to "waste" their time counting number of hours, bugs, lines of code, et cetera. If it constantly takes time to calculate measures, it will distract the team, and demotivates them.

To mitigate this issue, automate your measurement calculation. How? this depends on the measurement and the tools used.

4. Do not use measurements for evaluating individual developers.

Measurements is a means for project tracking and process improvement. It has also a side effect of motivating the team and achieving goals. But, a common mistake is to use these measurements to pinpoint productivity issues to individual developers.

I used to work in a strong measurements environment sometime ago, in which measurements are collected to monitor all software development phases. After the project ends, the same measurements are used to evaluate developers, and calculate their mid-year bonuses accordingly.

One of these measures was the number of review issues opened in peer reviews. In time, we found developers not opening any review issues, and prefer to communicate the review results either orally or by mail, not through the issue tracking system (which automatically calculates the measure). The reason they did that is no to affect the bonuses of themselves! I have to say that I personally found myself inclined not to open any review issues just like the rest of the team. It is a feeling that you cannot resist.

If you are interested to read more about such valuable advice, you will find these advice (and others) in the following references:

• More process Patterns, by Scott Ambler.
• Rapid Software Development: Taming Wild Software Schedules, by Steve McConnell.