Tragedies of Performance Evaluation of Software People

I'm constantly having conflicts with managers who insist on backward methods of performance evaluation for agile teams. It is one of the miseries of software teams, because in most cases, such methods demotivate individuals and undermine team values. The issue is serious to the extent that Professor Edward Lawler wrote a recent post in Forbes magazine titled: Eliminating Performance Appraisals. In this post, prof. Edward questions whether organizations should stop doing performance appraisals altogether, because they “do more damage than good”, and “when poorly done, they create a multitude of problems”!

I felt I have a moral obligation to write something in this regard. My contribution in this post is a collection of resources and an attempt to structure the topic into three high level subtitles:

1. What's evil about performance appraisals
2. Should smarter people be rewarded more money (as a means of motivation)?
3. Using process data for the sake of individual performance

If you are a smart person suffering from a performance evaluation scheme from the middle ages, send your HR manager this post, and let him read the resources mentioned below. May be (I'm not sure) it may help.

1. What's evil about performance appraisals

First, this is an article: Should I Rank My Employees? from the Wall Street Journal. It mentions the following interesting survey:

"Stanford professors Jeffrey Pfeffer and Robert Sutton are among the most fervent critics. They cited a survey of more than 200 human resources professionals from companies employing more than 2,500 people that found that even the more than half the companies used forced ranking, the respondents 'reported that forced ranking resulted in lower productivity, inequity and skepticism, negative effects on employee engagement, reduced collaboration, and damage to morale and mistrust in leadership.' … "

Another blog by the famous software engineering management author: Esther Derby titled: Should a ScrumMaster (or any coach) Give Performance Appraisals? In this blog, Esther asks how would it be good while "psychologists know that 80 percent of people believe their performance is above average"

The topic of performance appraisal are very controversial to the extent that one of the so famous books in management in the last years is called: "Abolishing Performance Appraisals: Why They Backfire and What to Do Instead"

2. Should smarter people be rewarded more money (as a means of motivation)?

We need to change our believes that a merit system will not work unless it ranks employees into classes and every class is motivated by some more money.

This is a must-see master piece by RSA Animate: Drive: The surprising truth about what motivates us:

In this short animation, they attack very strongly the idea about motivation using financial incentives when it involves cognitive and knowledge work. At minute 1:50 till 4:15, this animation sites the results of a study done by three top universities in Engineering and Technology: MIT, University of Chicago, and Carnegie Mellon university:

"As long as the task involve only mechanical skill, bonuses worked as expected (higher pay = better performance). But, once the task called for rudimentary [basic or simple] cognitive skill, a larger reward led to poorer performance!"

Also, one of the most important books in software engineering history is Rapid Software Development: Taming Wild Software Schedules by Steve McConnell. This book was named the best Software engineering book in the year by Software Development magazine's Jolt Award. Please refer to this book to understand what motivates Software engineers. Chapter 11: Motivation is the best reference I read about motivating software engineers. In page 262, you'll find this quote: 

"At least two-dozen studies over the last 30 years have shown conclusively that people who expect to receive a reward for doing their jobs successfully don't perform as well as those who expect no reward at all (Kohn 1993). The work itself is the greatest motivator, and the more a manager stresses an external reward, the less interested the developer becomes in the work itself, and the more potential motivation is lost."

The author makes a very clear alarm that incentives has many forms which are non-monetary like:

• Dinners with company executives
• Vacation-time bonuses
• Gifts of appreciation (theater tickets or dinners for two)
• Sincere praise directed at a specific accomplishment
• Team T-shirts, polo shirts, rugby shirts, watches, pins, mugs, posters,
• Humorous or serious awards in the form of plaques, certificates, trophies, and the like
• Special events to celebrate significant accomplishments; depending on your team's preferences, and event might be a dinner at a favorite restaurant, a show, a trip, a ski day, or a dinner at the boss's house (or, for maximum effect, the boss's boss's house)
• Exceptions to company policies for the team, such as casual-dress Friday, a Ping-Pong table in your team's section of the building, free soda pop in the team refrigerator, and so on
• Special courses (outside the local area)
• Sponsorship at conferences that the organization would not ordinarily sponsor
• Grade-level promotions

You can download this book from this location.

One more article called One More Time: How Do You Motivate Employees?, which sold more than a million reprints making it the most popular article in the history of the Harvard Business Review. In this diagram, notice the particular position of the salary and it's weight as a demotivator rather than a motivator. 

Motivators and demotivators - Notice that salary (money) is not among the motivators, but rather a demotivator in case salary is not sufficient

Actually, what this diagram tells us is the same exact phrase mentioned in the RSA movie above at minute 4:50:

"If you don't pay enough, people won't be motivated. Pay people enough to take the issue of money off the table"

3. Using process data for the sake of individual performance

Forcing managers to use process data, like feature estimates, number of bugs, reported actual hours, etc. to evaluate team members will blow up the process metrics themselves. This, in turn, has a drastic effect on the process itself, making the team process-blind (not being able to improve the process based on process metrics) Being process-blind is the direct result of exploiting process metrics and reuse them as Targets, and as per Goodhart's law: "When a measure becomes a target, it ceases to be a good measure."

Why this happens? To understand the dynamics, let's recall the general rule put by Tim DeMorko, the author of the great book: Peopleware: Productive Projects and Teams's, a book which had an impact on the whole industry; the general rule is known as DeMarko's Principle:

"Effort moves towards whatever is measured"

This can happen subconsciously or even consciously; and this is considered one of the major pitfalls of measuring, as explained the this article: The seven pitfalls of performance measurement which considers: Risk of manipulation of performance data as one of them. When it comes to individual evaluation, use of process metrics becomes a real risk, and opportunity of manipulation is so widespread. Writers have been noting this again and again, to the extent that some writers like Eli Goldratt, the father of Theory of Constraints says:

"Tell me how you measure me, and I will tell you how I will behave!"

In summary, using process metrics to evaluate individuals distorts the metrics and after a while, the metrics lose their meaning and cease to reflect real process data.

A Final Word

Finally, I have to say that this topic is vast and needs a lot of effort to make a shift in the mindset of project and HR managers of software in Egypt. It is not easy, but impossible is not on my dictionary :)

It’s none of the Managers’ Business!

A team is busy refactoring a very large and overly complex several hundred thousands lines of code. This team would hang on the wall this sign and instructions:

We are busy refactoring the product code for the benefit of all of us

It is a highly technical stuff which you will not grasp, even if you tried hard

We need to concentrate. Please do not keep nagging for status and end dates

When we finish, we'll let you know

Thank you for your understanding :)

Yes, this is a joke, but we've met teams doing this spontaneously! Here is the full story:

In a recent review of three companies which tried to refactor large products (hundreds of thousands line of code), we found that decisions were taken in all three companies to suspend the refactoring effort. Two of them decided to go for a full rewrite and the third decided to minimize further development in this particular product.

This appears to be impatient managers' decision. But, analysing the three cases we reached some very interesting reasons behind these costly decisions. The number one reason was the proposition by the technical team that: "it's none of the managers' business"!

Technical teams had this attitude. Managers had almost no involvement in planning refactoring effort, and the technical teams did not spend any effort to involve busy managers or report status to them. This attitude created opposition to further expenditure on refactoring and was one of the main reasons for management wanting to either discontinue the refactoring effort or rewrite the application from scratch.

We reached the following corollary about  how managers manage refactoring effort:

Managers will not sponsor any activity unless they can track and control it!

 Other successful refactoring attempts witnessed more support from managers. The basic difference was that managers used to receive monthly reports about the status of refactoring, and would accordingly decide on whether to accelerate or slow down refactoring.

Why process is important from day 1?

Sometimes we here some devastating (yes literally devastating) ideas like:

• Process is pure overhead.
  
• Process is discovered while working on the project (after we start the project).
  
• Let’s start coding, we will refactor anyways.
  
• Just jump in the code, things will get crystallized as you go.
  

The problems of such sayings is that it tempts poor developers to start a project with the unhandled risks inherent in the software development endeavor.

Famous processes like RUP markets for itself as “a risk removal process”. Agile methods assume to mitigate many risks of software development, even if not having a list of risks which we keep an eye on, which is true in my humble opinion.

The fact of starting a project without a clear view of how to do configuration management, how to do estimation, how to document requirements, how to involve stakeholders and get feedback on requirements, when and how to test, what are the bugs types, and which of them are to be fixed first, how, when and by whom architecture and design is going to be done, how to do tracking and what measures will indicate progress, how you will handle change, and many others. The fact of starting the project without a clear answer of such questions is pure act of failing the project, causing some persons to suffer sometime of their lives and get disappointed and demotivated at the end of the way (the developers), and causing others to lose some money, lose business opportunities, and lose trust in software development teams (the customers).

I hope this is a clear answer to the question. Please take some time and think of how you will develop software before you take the responsibility of any projects.

The next question is: Does process limit develop creativity? This needs a separate post.

Customer and Project Team Bill of Rights

The concept is brilliant. The first time I read it I said: “Yes, this is really what I always felt missing!”

Steve McConnell, in his excellent book Software Project Survival Guide, mentions a set of rights that should be respected for both the project team, and the customer who owns the product. Please spend some time reading them carefully; this is a prescription for project success.

Customer's Bill of Rights

I have the right:

1. To set objectives for the project and have them followed
2. To know how long the software project will take and how much it will cost
3. To decide which features are in and which are out of the software
4. To make reasonable changes to requirements throughout the course of the project and to know the costs of making those changes
5. To know the project’s status clearly and confidently
6. To be apprised regularly of risks that could affect cost, schedule, or quality, and to be provided with options for addressing potential problems
7. To have ready access to project deliverables throughout the project

Project Team's Bill of Rights

I have the right:

1. To know the project objectives and to clarify priorities.
2. To know in detail what product I’m supposed to build and to clarify the product definition if it is unclear.
3. To have ready access to the customer, manager, marketer, or other person responsible for making decisions about the software’s functionality.
4. To work each phase of the project in a technically responsible way, especially to not be forced to start coding too early in the project.
5. To approve effort and schedule estimates for any work that I will be asked to perform. This includes the right to provide only the kinds of cost and schedule estimates that are theoretically possible at each stage of the project; to take the time needed to create meaningful estimates; and to revise estimates whenever the project’s requirements change.
6. To have my project’s status reported accurately to customers and upper management.
7. To work in a productive environment free from frequent interruptions and distractions, especially during critical parts of the project.

It is top priority for managers to spread this atmosphere of mutual recognition of rights. Customer, as well as project team, sometimes denies other’s rights. The customer should ensure that he is available to answer the team’s questions. Many a time, I keep waiting many days to book a meeting with the customer or even get an email feedback from him. On the other hand, customers sometimes suffer from project teams not reporting to them the true status of the project. Many things like that take place, resulting at the end of an unsuccessful project end.