Interesting quotes

Quotes that deeply influenced how I think!

“I hear, and I forget
I see, and I remember
I do, and I understand”
-- Ancient Chinese Proverb

“Leadership is the process of creating an environment in which people become empowered”
-- Gerald M. Weinberg

"When projects fail, it's rarely technical"
-- Jim Johnson, the Standish Group

Parkinson’s Law:
“Work expands so that to fill the time available for its completion"

"An approximate answer to the right problem is worth a great deal more time than a precise answer to the wrong problem" 

-- John W. Tukey

“Insanity is doing the same thing, over and over again, but expecting different results.”
-- Albert Einstein

“Two things are infinite: the universe and human stupidity; and I'm not sure about the universe.”
-- Albert Einstein

“If you can't explain it to a six year old, you don't understand it yourself.”
-- Albert Einstein

“Anyone who has never made a mistake has never tried anything new.”
-- Albert Einstein

“A clever person solves a problem. A wise person avoids it.”
-- Albert Einstein

“Everything must be made as simple as possible. But not simpler.”
-- Albert Einstein

“Failure at an organizational level seems to come  from the inability to customize processes and  make them their own. Trying to apply someone  else’s template to your organization directly  doesn’t work well. It leaves out too many  important details of the previous successes and  ignores your company’s specific situation. ”
-- Kent Beck Extreme Programming founder in a 2006 interview with InfoQ.com

"Nothing in this world was so powerful as an idea whose time has come"
-- Victor Hugo

"He who is good with a hammer thinks everything is a nail"
-- Maslow

"It seems that perfection is attained, not when there is nothing more to add, but when there is nothing more to take away."
-- Antoine de Saint-Exupery

"Leadership is the process of creating an environment in which people become empowered"
-- Gerald M. Weinberg

"We are being ruined by the best efforts of people who are doing the wrong thing."
-- Edward Demming

Redmine Arabic

A while ago, I worked with a client who is using Redmine Arabic interface. I found the arabic translation of Redmine very weak. It is missing many translations and miss-translates many others and causes a lot of confusion.

The good news is that I have reviewed Redmine arabization and adjusted the arabic translation to match the majority of arabic speakers in Egypt, the gulf area, and the Middle East. Also, I have fixed many incorrect translations and added missing ones.

The new translation is released with Redmine 2.4.0, and here is the patch that I have submitted.

It’s none of the Managers’ Business!

A team is busy refactoring a very large and overly complex several hundred thousands lines of code. This team would hang on the wall this sign and instructions:

We are busy refactoring the product code for the benefit of all of us

It is a highly technical stuff which you will not grasp, even if you tried hard

We need to concentrate. Please do not keep nagging for status and end dates

When we finish, we'll let you know

Thank you for your understanding :)

Yes, this is a joke, but we've met teams doing this spontaneously! Here is the full story:

In a recent review of three companies which tried to refactor large products (hundreds of thousands line of code), we found that decisions were taken in all three companies to suspend the refactoring effort. Two of them decided to go for a full rewrite and the third decided to minimize further development in this particular product.

This appears to be impatient managers' decision. But, analysing the three cases we reached some very interesting reasons behind these costly decisions. The number one reason was the proposition by the technical team that: "it's none of the managers' business"!

Technical teams had this attitude. Managers had almost no involvement in planning refactoring effort, and the technical teams did not spend any effort to involve busy managers or report status to them. This attitude created opposition to further expenditure on refactoring and was one of the main reasons for management wanting to either discontinue the refactoring effort or rewrite the application from scratch.

We reached the following corollary about  how managers manage refactoring effort:

Managers will not sponsor any activity unless they can track and control it!

 Other successful refactoring attempts witnessed more support from managers. The basic difference was that managers used to receive monthly reports about the status of refactoring, and would accordingly decide on whether to accelerate or slow down refactoring.

Well, Agile Teams Should be Appraised as a Team ... Hmmmm

It may be useful to think this way. Agile teams are cross functional, self organizing, and collectively accountable for work results. Therefore, they should receive one score at team appraisal, and hence one raise, and one bonus, and one etc.

Very true. But, appraisals have another merit other than calculating the yearly raise. It should be used an invaluable individual motivator for self improvement, specially for teams whose work products involve highly innovative and creative work .. wow!

Usually, what I have seen in enterprises is that individual appraisals were a source of all kinds of nuisances, for the individual, for his manager, for senior management, for all the organization. It is rarely accepted, and usually results in demotivation of 90% of the individuals who see the results as unfair and biased.

There are causes for such phenomena, and here are five of them:

1. Using process data in evaluation, like the number of bugs, or the code review issues, or the produced LOC per hour!
2. Using everyday events and situational happenings as evidence for good or bad evaluation
3. Mixing individual productivity and team productivity metrics, and using both interchangeably while evaluating the team.
4. Single-rater evaluation by direct manager.
5. Using the appraisal as a motivational tool. That is, let's appraise the team, and give them a raise so that they become motivated. This doesn't work, and the higher the raise become, the more demanding the team will be.

Ok, how to appraise Agile teams? I'll prepare a model for that. This model sets two objectives for evaluation: Evaluating the process, and evaluating the individuals. The metrics and techniques used to evaluate the process are completely different and disjoint from metrics and techniques to evaluate individuals.

This will be my work for the upcoming several weeks. Keep alert!

Detect Architectural Violations Using ConQAT

In the previous blog on how to divide code into component, I have outlined simple steps. One of them is to detect and count violations (invalid calls from one component to another).

The following are simple steps to detect such violations using ConQAT:

1. Add a new Architecture Analysis. Go to Project>New>Other 
   
2. Select ConQat>Architecture Specification: 
   
3. A new architecture specification file is opened. Start adding components and assign classes, packages or namespaces to it: 
   
4. Following the steps in the next screen shot to (1) Add component (2) Click on the component to open the properties window (3) Press the ‘New inclusion button’ (4) Write a pattern indicating the class or package name: 
   
5. Now, we should run the architecture analyzer to verify the dependencies and catch any violations. To do that, we will create a Run configuration. Open “Project>New> Run Config": 
   
6. Open “blocklib” you will find a long list of analysis blocks. Select the following: 
• For .Net code: “ILArchitectureAnalysisChain” 
• For java code: “JavaArchitectureAnalysis” 
  
7. Add the required parameters as in the next screen shot: 
   
There are some other useful properties like:

• exclude-dependency: to exclude the specified namespace or class pattern from analysis
• exclude-file: to exclude the specified file names from analysis
  

8. Press the ‘Launch ConQAT analysis’ link 
9. Go to the architecture specification file and press the ‘Switch Editor Mode’ button as in the screenshot below: 
   
10. Browse for the ‘architecture-assessment.xml’ file under the output directory and select it. The result of the architecture analysis is displayed as in the following screenshot: 
    

Some Tips and Tricks

• The ‘orphans view’ lists all classes which are not assigned to any component. Review this list and make sure that it does not include any classes which may impact your analysis

• You may use the ‘orphans view’ to incrementally build you architecture, as such:
1. Create all you components and keep them empty
2. Run the architecture analysis, you will find that all classes are orphans. 
3. Drag and drop classes or complete packages on to the component
4. If the package is not part of your code and need to be excluded, right click on the package and ‘Copy pattern’: 
   
5. Then, you may exclude this pattern using the exclude-dependency property

• You may right click the violations and take a decision to allow them, if appropriate:
  

A Simple Method for Dividing Code into Components

In the 3Q's method, the second stage of refactoring legacy code is 'Divide and Conquer'. In this stage, we discover and separate (or simply reorganize) the code into manageable components. Usually, this is done as per the following simple workflow:

What to do while dividing your code into components

This is rather a straightforward logical sequence of modularizing legacy code. What we are interested now is how each step is done.

In an experiment that I have done with one team, we have separated the classes into components and After the first step, this how it looked like :)

Analysis of violations between the newly specified components

Red lines indicate communication between components which should not be allowed. As you see, nearly there are violations between all components! This is very normal, because these components are specified for the first time; and previously, all classes were allowed to talk to all other classes in the application code.

But, how did we detect these violations? In a later blog: Detect Architectural Violations Using ConQAT answers this question.

I'm Presenting At the Agile Conference 2013

I'm presenting at Agile2013 inshaAllah :)

What I will present is the result of last year's work on continuous refactoring of legacy applications. Here is my session proposal details: Refactoring as a Lifeline: Lessons Learned from Refactoring.

The Agile conference is the biggest international conference about Agile Software development featuring more than 200 sessions delivered in 5 days. 

Detect Exact and Similar Code Duplicates Using ConQat

[Update: this post is related to ConQAT version 2011.9. However, the conQat blocks provided in this post will not work on the latest version which is 2014.1. Dr. Hummel (one of ConQat developers) has kindly shared with us the set of blocks for the newer version here. As per Dr. Hummel, the equality and similarity properties (described below) will be part of all future releases]

Part of my work on refactoring legacy code was to find a way to identify code clones (duplicates) in large data repositories; and during my experiments on refactoring, I found that it would be much faster to follow this order:

1. Remove dead code (CSRS ~ 600 LPH, also reduces 10% of duplicate code)
2. Remove exact clones (CSRS ~ 300-400 LPH)
3. Remove similar clones 

Where:
CSRS is the Cost Size Reduction Speed, measured in LOC per Hour (LPH)
Exact clones: are clones which are typically the same (probably they may differ in white space only)
Similar clones: are exact ones but with renamed variables.

The best tool I found was ConQAT, an excellent open source and very flexible tool. However, I had to adjust it a bit to enable detecting exact and similar clones.

Detecting Exact and Similar Code Duplicates:

If you would like to try this, download this adjusted set of blocks, then follow these steps to replace the old ConQat blocks on your machine.

1. Unzip the file “ConQat blocks - vX.Y.rar”, and replace the following folder: <conqat eclipse folder>\ conqat-2011.9\conqat\bundles\org.conqat.engine.code_clones\blocks
2. Press the “Enforce Full Model Rebuild”:
3. Open ConQat Runtime view and press “New”. You will find new parameters called “equality” and “similarity”: 
4. Add the “equality” parameter, and set the equality threshold to 1 (meaning that the clones should be 100% identical): 

See the Result!

This is an example of the result, only exact clones are listed:

Works for Which Language?

This applies on (and has been tested against):

• Java code using the JavaCloneAnalysis block
• .Net code using the CsCloneAnalysis block
• C/C++ code using the StatementCloneAnalysis block. Actually, the StatementCloneAnalysis may be applied on 10 different languages. Review the ConQat documentation for more information. 

Enjoy!

The First Retrospective

Being there while the team is doing their first retrospective is an invaluable and full of emotions experience. You notice the team members know exactly, for the first time, what they are doing: What they sought to do, what they did well, and what went bad. More important, you see the team get energetic, prepared, and all set for the next sprint, sometimes can't wait to rectify the side effects of the first one.

It is an experiment that I'm doing every now and then with new teams doing agile adoption. I see that it is important to study the first retrospective more deeply. Surely, it will give us insight about team dynamics and good practices of agile adoption.

Process Bugs

Context: A production bug is reported by 55 customers, causing a complete rollback of the new release in most of them, while some others halted their online services waiting for a hot-fix.

Adam (the PM), shouting: Who is responsible for this terrible bug?
Amr (the team leader): let's concentrate on resolving it and get these customers up and running again.
Adam, frowning: OK

After that the team spends more than 18 hours of continuous work fixing, testing, packaging, deploying, re-testing, etc. Finishing the last touches early next morning, Adam entered the team room:

Adam: Now, it was a horrible day yesterday, and I would like to know exactly what led us to this situation.
Mona (tester): developers are always careless about their code.
Sarah (developer): No way, it was due to your bad testing. If we are careless, Isn't it your responsibility to be careful and catch such tiny bugs?
Adam is listening ...
Yousef (developer): Hey, are we going to keep blaming each other and forget about the tight schedules, late notices, and minute changes, that we are always having?

Seconds of silence passed while all are staring anywhere not accepting any responsibility.

Amr: It was a process bug.
Adam: what?
Amr: A process bug. It has nothing to do with the team. One person or even the whole team cannot be blamed for such a recurring problem report. We keep looking for a person to shout at and forget about the root cause of the problem, which is a process bug.

This is a typical discussion that takes place in software development houses.  The last note is interesting and worth attention. Process Bugs are holes in the software development process which give way for bugs to slip into customer sites. Process bugs are what I always blame for bugs occurring at production. It is the development process which didn't help us prevent bugs from getting injected into our code. It also didn't help us discover these bugs and get them resolved before they reach customers.

It is very bad that we always keep thinking of which team members to blame. What is worse is our thinking that blaming really resolves the root-cause of the problem! If team leaders completely ignored these process bugs, and kept the deadly habit of team blaming, the following dynamic will definitely occur:

The deadly cycle of bug-blame-stress

If bugs occur and the team are blamed for it, they will lead to higher levels of stress; which will usually result in more bugs, more blaming, more stress, in an endless loop manner. Actually, this is an example of 'positive feedback loops' in which factors re-enforce themselves and result in system expulsion after a while. Examples such expulsions may be burn-out, getting out-ranged, quit development, and similar actions.

I have seen excellent software engineers quit the software industry because they lost confidence in themselves. Why? due to continuous public and personal blaming :(

In the bug case mentioned above (which actually happens every day), the following is a very modest listing of possible process bugs:

• Code review is never planned. The team hardly find anytime to review.
• Testing issues are reported by mail, and get lost every now and then due to complexity of managing hundreds of daily mails in the team leader's inbox
• Bugs are assigned to module owners, whom may be very busy responding to customers or developing new features in another project. 
• Team had several burn-outs, which resulted in more stress on the remaining team members!
• ...

The key take-away of this blog is to look for root causes in process rather than looking for someone to blame. Blaming may relieve managers' stress and bad temper, but it will lead to team expulsion very soon!

Instead of blaming, a sound management question for the team should be similar to the one I described in this blog title: Take your time to pay for this technical debt, but let me know how you would prevent it in the future! As I said in this blog, part of fixing a problem is preventing it from re-occurring. In other words,  fixing the process hole that led to it.