Tragedies of Performance Evaluation of Software People

I'm constantly having conflicts with managers who insist on backward methods of performance evaluation for agile teams. It is one of the miseries of software teams, because in most cases, such methods demotivate individuals and undermine team values. The issue is serious to the extent that Professor Edward Lawler wrote a recent post in Forbes magazine titled: Eliminating Performance Appraisals. In this post, prof. Edward questions whether organizations should stop doing performance appraisals altogether, because they “do more damage than good”, and “when poorly done, they create a multitude of problems”!

I felt I have a moral obligation to write something in this regard. My contribution in this post is a collection of resources and an attempt to structure the topic into three high level subtitles:

1. What's evil about performance appraisals
2. Should smarter people be rewarded more money (as a means of motivation)?
3. Using process data for the sake of individual performance

If you are a smart person suffering from a performance evaluation scheme from the middle ages, send your HR manager this post, and let him read the resources mentioned below. May be (I'm not sure) it may help.

1. What's evil about performance appraisals

First, this is an article: Should I Rank My Employees? from the Wall Street Journal. It mentions the following interesting survey:

"Stanford professors Jeffrey Pfeffer and Robert Sutton are among the most fervent critics. They cited a survey of more than 200 human resources professionals from companies employing more than 2,500 people that found that even the more than half the companies used forced ranking, the respondents 'reported that forced ranking resulted in lower productivity, inequity and skepticism, negative effects on employee engagement, reduced collaboration, and damage to morale and mistrust in leadership.' … "

Another blog by the famous software engineering management author: Esther Derby titled: Should a ScrumMaster (or any coach) Give Performance Appraisals? In this blog, Esther asks how would it be good while "psychologists know that 80 percent of people believe their performance is above average"

The topic of performance appraisal are very controversial to the extent that one of the so famous books in management in the last years is called: "Abolishing Performance Appraisals: Why They Backfire and What to Do Instead"

2. Should smarter people be rewarded more money (as a means of motivation)?

We need to change our believes that a merit system will not work unless it ranks employees into classes and every class is motivated by some more money.

This is a must-see master piece by RSA Animate: Drive: The surprising truth about what motivates us:

In this short animation, they attack very strongly the idea about motivation using financial incentives when it involves cognitive and knowledge work. At minute 1:50 till 4:15, this animation sites the results of a study done by three top universities in Engineering and Technology: MIT, University of Chicago, and Carnegie Mellon university:

"As long as the task involve only mechanical skill, bonuses worked as expected (higher pay = better performance). But, once the task called for rudimentary [basic or simple] cognitive skill, a larger reward led to poorer performance!"

Also, one of the most important books in software engineering history is Rapid Software Development: Taming Wild Software Schedules by Steve McConnell. This book was named the best Software engineering book in the year by Software Development magazine's Jolt Award. Please refer to this book to understand what motivates Software engineers. Chapter 11: Motivation is the best reference I read about motivating software engineers. In page 262, you'll find this quote: 

"At least two-dozen studies over the last 30 years have shown conclusively that people who expect to receive a reward for doing their jobs successfully don't perform as well as those who expect no reward at all (Kohn 1993). The work itself is the greatest motivator, and the more a manager stresses an external reward, the less interested the developer becomes in the work itself, and the more potential motivation is lost."

The author makes a very clear alarm that incentives has many forms which are non-monetary like:

• Dinners with company executives
• Vacation-time bonuses
• Gifts of appreciation (theater tickets or dinners for two)
• Sincere praise directed at a specific accomplishment
• Team T-shirts, polo shirts, rugby shirts, watches, pins, mugs, posters,
• Humorous or serious awards in the form of plaques, certificates, trophies, and the like
• Special events to celebrate significant accomplishments; depending on your team's preferences, and event might be a dinner at a favorite restaurant, a show, a trip, a ski day, or a dinner at the boss's house (or, for maximum effect, the boss's boss's house)
• Exceptions to company policies for the team, such as casual-dress Friday, a Ping-Pong table in your team's section of the building, free soda pop in the team refrigerator, and so on
• Special courses (outside the local area)
• Sponsorship at conferences that the organization would not ordinarily sponsor
• Grade-level promotions

You can download this book from this location.

One more article called One More Time: How Do You Motivate Employees?, which sold more than a million reprints making it the most popular article in the history of the Harvard Business Review. In this diagram, notice the particular position of the salary and it's weight as a demotivator rather than a motivator. 

Motivators and demotivators - Notice that salary (money) is not among the motivators, but rather a demotivator in case salary is not sufficient

Actually, what this diagram tells us is the same exact phrase mentioned in the RSA movie above at minute 4:50:

"If you don't pay enough, people won't be motivated. Pay people enough to take the issue of money off the table"

3. Using process data for the sake of individual performance

Forcing managers to use process data, like feature estimates, number of bugs, reported actual hours, etc. to evaluate team members will blow up the process metrics themselves. This, in turn, has a drastic effect on the process itself, making the team process-blind (not being able to improve the process based on process metrics) Being process-blind is the direct result of exploiting process metrics and reuse them as Targets, and as per Goodhart's law: "When a measure becomes a target, it ceases to be a good measure."

Why this happens? To understand the dynamics, let's recall the general rule put by Tim DeMorko, the author of the great book: Peopleware: Productive Projects and Teams's, a book which had an impact on the whole industry; the general rule is known as DeMarko's Principle:

"Effort moves towards whatever is measured"

This can happen subconsciously or even consciously; and this is considered one of the major pitfalls of measuring, as explained the this article: The seven pitfalls of performance measurement which considers: Risk of manipulation of performance data as one of them. When it comes to individual evaluation, use of process metrics becomes a real risk, and opportunity of manipulation is so widespread. Writers have been noting this again and again, to the extent that some writers like Eli Goldratt, the father of Theory of Constraints says:

"Tell me how you measure me, and I will tell you how I will behave!"

In summary, using process metrics to evaluate individuals distorts the metrics and after a while, the metrics lose their meaning and cease to reflect real process data.

A Final Word

Finally, I have to say that this topic is vast and needs a lot of effort to make a shift in the mindset of project and HR managers of software in Egypt. It is not easy, but impossible is not on my dictionary :)