Why collect measurements?

Two main reasons to collect measurements:

1. To track your project. A basic task of any software project manager is to track the project progress. Examples are the defect rate, the size of finished functionality versus not finished. A very good technique which reads this measurement and displays it in a graphical format is the burn-up/burn-down charts which are heavily used in Agile development.
2. To improve the process of producing software. Very similar to six-sigma projects, CMMI level 4 and 5 have techniques which starts with an evidence of a mal-functioning process (most commonly supported by measurements).Then, it digs into the root causes, and suggests corrective actions. Finally, it designs measurements to quantify the impact of corrective actions, and start over again. This ‘Continuous Improvement’ cycle is one of the techniques used in any engineering discipline.

Measurements are an invaluable means for management, especially for software development, which is a deep intellectual engineering discipline. I may safely say that without measurements, you may not reach good management decisions what so ever. One of the sayings that is said again and again in Balanced Score-Cards environments is that “you cannot manage what you cannot measure, and you cannot measure what you cannot define”, is really true.

What to keep in mind while designing your measurements

Out of the many good advice about measurement systems for software processes, I have benefited most from these four pieces of advice:

1. Set the goal before you measure.

It is tempting to set out a range of measurements, each costs some good time in calculation, while you don't need it. It is also very common that a project manager or a process engineer says: "we will need them later on, so why not collect them anyways?", another third idea is that "collecting many measurements may give you insight about the deficiencies in the project or the process, so the more you collect, the more insight you gain"!

I have lived with these philosophies for a while, and spent so much time collecting lots of measurements just for the sake of measuring, and I really think it was pointless.

The strategy I'm following right now is to set up the next objectives (one or two, at maximum), design the measures which I really need, and implement the infrastructure which will help me collect these measures automatically.

2. Distribute measurements results and analysis to the development team

Results of measurements may be very interesting for the team to visualize. Not only interesting, but also motivating. As a practise I continuously follow, I send the results of the measrements and analysis to the team, and discuss with them their implications. This helps me to:

• Motivate the team to work towards achieving better measurements.
• Motivate the team to record measurements more accurately.
• Get the team to the point that they trust the value of the measurements as a means for project management, and as a means for process improvement.

3. Minimize the time of collecting measurements.

This is a point which is very sensitive. If the cost of collecting a measurement is high, it is more likely that the whole measurement system will collapse! Developers like to develop, not to "waste" their time counting number of hours, bugs, lines of code, et cetera. If it constantly takes time to calculate measures, it will distract the team, and demotivates them.

To mitigate this issue, automate your measurement calculation. How? this depends on the measurement and the tools used.

4. Do not use measurements for evaluating individual developers.

Measurements is a means for project tracking and process improvement. It has also a side effect of motivating the team and achieving goals. But, a common mistake is to use these measurements to pinpoint productivity issues to individual developers.

I used to work in a strong measurements environment sometime ago, in which measurements are collected to monitor all software development phases. After the project ends, the same measurements are used to evaluate developers, and calculate their mid-year bonuses accordingly.

One of these measures was the number of review issues opened in peer reviews. In time, we found developers not opening any review issues, and prefer to communicate the review results either orally or by mail, not through the issue tracking system (which automatically calculates the measure). The reason they did that is no to affect the bonuses of themselves! I have to say that I personally found myself inclined not to open any review issues just like the rest of the team. It is a feeling that you cannot resist.

If you are interested to read more about such valuable advice, you will find these advice (and others) in the following references:

• More process Patterns, by Scott Ambler.
• Rapid Software Development: Taming Wild Software Schedules, by Steve McConnell.

Why process is important from day 1?

Sometimes we here some devastating (yes literally devastating) ideas like:

• Process is pure overhead.
  
• Process is discovered while working on the project (after we start the project).
  
• Let’s start coding, we will refactor anyways.
  
• Just jump in the code, things will get crystallized as you go.
  

The problems of such sayings is that it tempts poor developers to start a project with the unhandled risks inherent in the software development endeavor.

Famous processes like RUP markets for itself as “a risk removal process”. Agile methods assume to mitigate many risks of software development, even if not having a list of risks which we keep an eye on, which is true in my humble opinion.

The fact of starting a project without a clear view of how to do configuration management, how to do estimation, how to document requirements, how to involve stakeholders and get feedback on requirements, when and how to test, what are the bugs types, and which of them are to be fixed first, how, when and by whom architecture and design is going to be done, how to do tracking and what measures will indicate progress, how you will handle change, and many others. The fact of starting the project without a clear answer of such questions is pure act of failing the project, causing some persons to suffer sometime of their lives and get disappointed and demotivated at the end of the way (the developers), and causing others to lose some money, lose business opportunities, and lose trust in software development teams (the customers).

I hope this is a clear answer to the question. Please take some time and think of how you will develop software before you take the responsibility of any projects.

The next question is: Does process limit develop creativity? This needs a separate post.

All of them are victims, the customer, and the development team!

Very interesting situation. I’m now out-sourced as a project manager to manage some of the IT projects developed by the company I work for. The client is a big telecom customer of my company. In this case, I deal with the end users of the systems and see and hear how they think of the development team. On the other hand, I deal with the development team closely, and they consider me “on their side”, not on the “enemy/customer” side!

The customer is really suffering. He suffers from so many delays, unexpected results while testing, misunderstandings, unmanaged expectations, resistance of changes in requirements, et cetera. In summary, at best, the customer is suffering. The customer says that the IT was much better in the past, but now, they think that the development team is incapable, but they can’t escape dealing with them because simply speaking, all other vendors are alike or even worse!

On the other hand, the development team is also suffering. They suffer from the changing business at the customer side. They suffer from the large scale of the software size, while there is no central architecture team. They also suffer from other things, like the oppressing environment created by middle management who are also oppressed by the frustrated customer. They finally suffer from the fact that they are not appreciated by any means! All what they hear is that the customer is frustrated because so and so, but they are never rewarded of the tremendous amount of effort they exert.

Both of them, the customer and the development team, are victims. None of them should be blamed. Yes, none of them should be blamed.

The fact is that it is the methodology which should be blamed for such miserable situation. There has to be a change in the attitude, a paradigm shift in the operation of producing software. Techniques like iterative development, incremental delivery, automated tests, continuous customer involvement, and central architecture team are all critical success factors.

I’m afraid this situation is becoming the norm in software development. It started to be an accepted situation that software development is a difficult field and we have to accept all these issues as normal and cope with it; very similar to a situation where one has a disability and he has to cope with it!

I’m feeling so bad for these suffering people, both in business and in development.