Saturday, May 28, 2016

Detecting and Preventing Code Clones Mercilessly - workshop guide

Please follow these steps:

First: Open the code project

We will be working on the example project provided by ConQAT:
  1. Open Eclipse
  2. Goto: New Project > Examples > ConQAT Examples > ConQAT Examples Project
Second: Run the code clones analysis tools
  1. Double click the clonedetection-example-junit.cqr 
  2. Click Launch ConQAT analysis
  3. After it finishes, goto Menu: Clone Detection > Open Clone Report, and select the clones.xml file in the output directory
Third: Detecting exact code clones
  1. Back to the clonedetection-example-junit.cqr page
  2. Scroll all the way to the right and click the 'New' button
  3. Add the field 'equality', and enter value 1 in the threshold
  4. Launch the analysis and reload the clones.xml file
Forth: Detecting similar code clones
  1. Back to the clonedetection-example-junit.cqr page
  2. Scroll all the way to the right and click the 'New' button
  3. Add the field 'similarity', and enter value 1 in the threshold
  4. Remove the 'equality' parameter
  5. Launch the analysis and reload the clones.xml file
Fifth: Detecting gapped clones
  1. Goto the 'ConQat blocks' section at the top right of the configuration file
  2. Click on 'Change' button
  3. Select 'Code Clones > JavaGappedClonesAnalysis'
  4. Add a parameter called 'errors' and enter the value = 2
  5. Add a parameter called 'gap' and enter the ratio = 0.2 
  6. Launch the analysis and reload the clones-gapped.xml file (notice the file name is different than the previous ones)

Tuesday, May 3, 2016

I'm Speaking at Agile2016

I'm glad to announce that I will be speaking at the Agile Conference - Agile2016 organized by Agile Alliance. This is the biggest agile gathering in the world, featuring more than 2000 attendees and 200 sessions.

I will be running a technical workshop titled: Detecting and Preventing Code Clones Mercilessly, at which I will go through different types of code clones and how to detect and prevent them.

Tuesday, January 12, 2016

Agile Egypt's 1st Open Space Experiment!

I like to call Agile Egypt's 1st Open Space Conference an experiment rather than a conference. It was really an experiment working in large group without having an agenda :) We were worried while preparing, anticipating and looking around during the upstream activities, but we had a lot of fun and learning throughout the day! It is awesome and unbelievable!

The dynamics of teams while cooperating and engaging in discussions and parallel activities were amazing. Some are moving around from team to another looking for something useful. One team closed up the topic in eager to participate in other one or two sessions before they time out. Two or three attendees had a lengthy side talk about a topic which wasn't on the original list. All of these are some notes of how dynamic and efficient the use of time was.

Let me summarize how it went:

Starting (30 minutes) 

First, we spent sometime explaining the concept of open space technology; the 5 principles and the law of two feet. Then, Mohamed Amr facilitated a group discussion about the topics that may be included during the day:

Sessions (2 hours)

We have split into 6 topics:
  1. Self-Organizing teams
  2. Agile and freelance developers
  3. Requirements in Agile
  4. Measuring technical dept
  5. Modeling Kanban issues in Diagrams of Effects
  6. Agile adoption success stories

Teams in parallel sessions

And this is me facilitating the Kanban modeling session:

To see more photos of this event, browse our event page on

Summary of Sessions' Key Takeaways (1 hour)

The last hour was very informative. Each group facilitator prepared a list of key takeaways which he/she shared with the whole audience in 5-10 minutes. Some members found this not interesting, because 1) If they were interested in the topic, they would have participated in the session and 2) the intent was to share key takeaways, while some facilitators narrated parts of the discussions instead :)

What's Next?

I have done a brief evaluation at the end of the conference. All Attendees would like to attend similar meetups again. Most of them (about 60%) gave 4 and 5 (out of 5) to their learning experience in this meetup.

I have observed how dynamic the meetup was. Learning is guaranteed! If you disliked a topic, you can instantly switch to another, which is several steps away. The fact that we are all co-located made the movement so easy.

We are planning to do it again inshaAllah with one minor tweak: we will have one or two sessions predefined and announced in order to attract more audience. One of the things that makes people reluctant to attend is the fact that he doesn't know what the topics are!

Although this is done on purpose (not announcing any topics), we think that announcing one or two topics would be a catalyst to attract more people. Once they are in, they can switch to any topic anytime!

Saturday, November 21, 2015

Is Scrum (and probably Agile) Dead?

Recently, I came across an interesting article: Scrum is dead: breaking down the new open development method. I usually do have big problem with such big declarative statements like 'Scrum is dead'. Actually, it's pretty easy to refute the whole rationale by bringing about one counter example!

But, to be frank, the article is good, with the author trying to put some basic principles for the profession of software development; principles that I fully agree with and promote. On the other hand, I strongly disagree with some very fundamental ideas in the article:

Agile is Not a Methodology

First, I find the author builds upon a fundamental idea that "Agile is a methodology". Well, this is incorrect. Also, the article conveys an idea that Agile and Scrum are two sides of the same coin (a misconception about Agile. May be the author doesn't mean that, but this idea is there in the article). Actually, Agile does not prescribe any methods. Rather, there are methods which are built while Agile in mind. So, what is agile? I always preach Ahmed Sidky's definition of Agile:

"Agile is a mindset, that is in the software world, established through 4 values, grounded by 12 principles, and manifested through many many different practices"

In this sense, there is no point of discussing whether Scrum, Kanban, or Open Development are useful methodologies or not. Because the whole idea of usefulness is very situational. What is useful in one situation is not useful in another, what makes sense for one team is totally irrelevant for another. 

Is Open Development an "Evolution of Scrum"?

The other statement which I find misleading is that Open Development is an "evolution of Scrum". Well, this is also incorrect. The Open Development description as appears in the article is totally in accordance with the Agile mainstream since its inception. The stream upon which ideas like Scrum and XP was built. In such mainstream, major advances in Software development were invented, such as automated tests and Continuous Integration and Deployment; and major ideas were revived or re-established like IID (Iterative and Incremental Development) or self-organizing and cross-functional teams.

Well, when to use which? or shall we use None?

Having said all of that, I do admit that Scrum (as per the official Scrum guide) is not suitable in many cases. A better statement is that: Iterative development is not useful in many cases. Some project types cannot be led using timeboxed iterations. Usually, the reason is that the demands of the business and the development pace are much faster than a one-week timebok, which is the smallest iteration timebox. The idea is very simple: If business needs something now, and development is capable of providing this thing at this now, why wait till end of iteration, which may be 2-3 or even more than a week ahead?!

On the other hand, I recently came to a situation where I couldn't use Kanban to manage my project. The unknowns of business are so much, and the availability of the business people is limited; besides great technical risks which needs prolonged time of team collaboration and deep thinking. In this specific case, we switched from Kanban to Scrum.

Managing Flow or Managing Iterations?
Actually, what we have done is to move not from Kanban to Scrum, but from Managing Flow to Managing Iterations. Kanban and Scrum nowadays are metaphors for two more fundamental management ideas: Managing flow versus managing iterations.

So, In this project instance, I found that putting constraints on flow is stopping us rather that helping us. I found that a timebox of one or two weeks gives the team more room to act, react, and innovate. In the meanwhile, it gives us some very high level indicators about team progress.

When shall we use None?
One final thought is that even if you're doing Scrum or Kanban, it's just a starter. It just brings about a sense of discipline and cadence which enables the team to measure it's progress and improvement over time.

So, if this is only a starter, you should expect that teams will move to other practices and probably invent their own process in the near future. At this time, I would say that this team is really Agile!

Monday, October 26, 2015

Things are Inverted in Agile!

It seems that many things in Agile are INVERTED !!

I noticed two important aspects (so far):

Project Startup Activities

We used to code then deploy. Just before deployment, we figure out how are we going to deploy, on which server, testing environment or staging environment. Usually, we may wait for sometime till these logistics are sorted out. 
In agile, things are different. We start by preparing the deployment environment, set it up, link it to our code deployment scripts using some Continuous Integration and Deployment tool. 

The Famous Design Phase

"In TDD, there is no design stage; design is inverted and distributed.", I quoted this from the online TDD training from Industrial Logic. It is fairly true and powerful concept. 
The idea is to code-test-design (or refactor) instead of design-code-test. Well, there has to be some upfront critical architectural decisions, but these are very few and takes several hours to several days at most.

Saturday, September 26, 2015

Dead Code; Is it Really Harmful?

The answer is YES, and can put you out of business in 45 minutes, like what happened with Knight Capital Group!

Recently, I have been watching uncle Bob explaining his famous SOLID principles, and he mentioned a very interesting case of a bug. The code mistakenly set a flag which enabled the execution of a piece of dead code.

The side effect of this execution is 440 million USD's losses which is 5 times the market value of the company :)

Further read about this case:

Tuesday, July 14, 2015

Lean CMMI - My Presentation at Agile2015!

First and foremost, I'm so grateful and thankful to Allah (swt) for helping me getting through this research paper. Not only that, I got it accepted in the Agile Conference 2015; the biggest Agile conference in the world.

I have published several papers before, but they were all industrial reports. Industrial reports (aka experience reports) document useful experiments done in the industry. As opposed to research papers, industrial reports do not necessarily use a scientific method to prove something.

My personal opinion, Industrial Reports are very useful, and probably more valuable than research paper. They reflect real experience and one can draw benefit from it right away. So, if your audience are software practitioners, then research papers are not the right means to communicate your experience to them :)

If this is the case, why did I write a paper? Two reasons:
  1. The first reason is that I had data! and found it interesting to experiment writing a paper.
  2. The second reason is to write a proof that incremental approaches in process improvement is better that holistic approaches. 
Here is a link to the abstract of the paper: Lean CMMI: An Iterative and Incremental Approach to CMMI-Based Process Improvement.

In this paper, I'm comparing results of process improvement in Configuration Management process area of CMMI. The first group of results are based on traditional process improvement effort (one process area at a time). The second group of results are based on the 'Process Increments' method, an iterative and incremental approach for software process improvement. This model divides improvements into small chunks as in the following model:

These Process Increments are dynamic. You can add, remove, re-prioritize them as needed. It's like user stories in software projects.

The results of the study are in the following diagram. It's a box plot of SP ratings in semi-formal CMMI appraisals. As you see, the ratings in the second group of companies are much higher than ratings in the first group of companies. is as follows:

Those who knows six sigma or has experience in research would know that these results are real proof that iterative and incremental methods works better!

Saturday, April 25, 2015

8 Months of Being Self-Employed!

Today is my 8th month after resignation from SECC and becoming self-employed at Agile Academy. This was a challenge full of new experiences, problems, obstacles, failures and successes :)

It's like any new startup experience, in which you're doing everything which you used to get them done by whoever else! Many things changed in my life like being with my wife and children for a longer time, setting in front of my laptop for a longer time, thinking about business and opening new opportunities. It's an experience in itself, not to be expecting any income at the beginning of every calendar month .. :(

One thing I noticed is that what Allah gives you cannot be compared to your effort. Of course you have to exert all what you can so that Allah give you, but it's always Allah's will, and I felt so relieved when I finally understood that Allah will not let you down as long as you are doing all what you can .. Al7amdulillah.

One very important thing to consider is that you have to always do what you like and what you know. Both are important. Then, الرزق will come along the way, whether through direct work you produce or through any other route.

One more thing .. It's not appropriate to work day and night. You have other obligations in life which you have to fulfill, like your parents, your wife and children, your own self, and before all of that, your Lord Allah SWT.

Friday, March 13, 2015

My Article about Technical Leadership

My second article published at infoq is for Technical Team Leaders. It's a post which I filled for many many years, and I have collected all my observations about what should a TTL be doing with the team in this article.

good read :)

Sunday, December 14, 2014

Tragedies of Performance Evaluation of Software People

I'm constantly having conflicts with managers who insist on backward methods of performance evaluation for agile teams. It is one of the miseries of software teams, because in most cases, such methods demotivate individuals and undermine team values. The issue is serious to the extent that Professor Edward Lawler wrote a recent post in Forbes magazine titled: Eliminating Performance Appraisals. In this post, prof. Edward questions whether organizations should stop doing performance appraisals altogether, because they “do more damage than good”, and “when poorly done, they create a multitude of problems”!
I felt I have a moral obligation to write something in this regard. My contribution in this post is a collection of resources and an attempt to structure the topic into three high level subtitles:
  1. What's evil about performance appraisals
  2. Should smarter people be rewarded more money (as a means of motivation)?
  3. Using process data for the sake of individual performance
If you are a smart person suffering from a performance evaluation scheme from the middle ages, send your HR manager this post, and let him read the resources mentioned below. May be (I'm not sure) it may help.

1. What's evil about performance appraisals

First, this is an article: Should I Rank My Employees? from the Wall Street Journal. It mentions the following interesting survey:
"Stanford professors Jeffrey Pfeffer and Robert Sutton are among the most fervent critics. They cited a survey of more than 200 human resources professionals from companies employing more than 2,500 people that found that even the more than half the companies used forced ranking, the respondents 'reported that forced ranking resulted in lower productivity, inequity and skepticism, negative effects on employee engagement, reduced collaboration, and damage to morale and mistrust in leadership.' … "
Another blog by the famous software engineering management author: Esther Derby titled: Should a ScrumMaster (or any coach) Give Performance Appraisals? In this blog, Esther asks how would it be good while "psychologists know that 80 percent of people believe their performance is above average"

The topic of performance appraisal are very controversial to the extent that one of the so famous books in management in the last years is called: "Abolishing Performance Appraisals: Why They Backfire and What to Do Instead"

2. Should smarter people be rewarded more money (as a means of motivation)?

We need to change our believes that a merit system will not work unless it ranks employees into classes and every class is motivated by some more money.

This is a must-see master piece by RSA Animate: Drive: The surprising truth about what motivates us:

In this short animation, they attack very strongly the idea about motivation using financial incentives when it involves cognitive and knowledge work. At minute 1:50 till 4:15, this animation sites the results of a study done by three top universities in Engineering and Technology: MIT, University of Chicago, and Carnegie Mellon university:
"As long as the task involve only mechanical skill, bonuses worked as expected (higher pay = better performance). But, once the task called for rudimentary [basic or simple] cognitive skill, a larger reward led to poorer performance!"
Also, one of the most important books in software engineering history is Rapid Software Development: Taming Wild Software Schedules by Steve McConnell. This book was named the best Software engineering book in the year by Software Development magazine's Jolt Award. Please refer to this book to understand what motivates Software engineers. Chapter 11: Motivation is the best reference I read about motivating software engineers. In page 262, you'll find this quote: 
"At least two-dozen studies over the last 30 years have shown conclusively that people who expect to receive a reward for doing their jobs successfully don't perform as well as those who expect no reward at all (Kohn 1993). The work itself is the greatest motivator, and the more a manager stresses an external reward, the less interested the developer becomes in the work itself, and the more potential motivation is lost."
The author makes a very clear alarm that incentives has many forms which are non-monetary like:
  • Dinners with company executives
  • Vacation-time bonuses
  • Gifts of appreciation (theater tickets or dinners for two)
  • Sincere praise directed at a specific accomplishment
  • Team T-shirts, polo shirts, rugby shirts, watches, pins, mugs, posters,
  • Humorous or serious awards in the form of plaques, certificates, trophies, and the like
  • Special events to celebrate significant accomplishments; depending on your team's preferences, and event might be a dinner at a favorite restaurant, a show, a trip, a ski day, or a dinner at the boss's house (or, for maximum effect, the boss's boss's house)
  • Exceptions to company policies for the team, such as casual-dress Friday, a Ping-Pong table in your team's section of the building, free soda pop in the team refrigerator, and so on
  • Special courses (outside the local area)
  • Sponsorship at conferences that the organization would not ordinarily sponsor
  • Grade-level promotions
You can download this book from this location.
One more article called One More Time: How Do You Motivate Employees?, which sold more than a million reprints making it the most popular article in the history of the Harvard Business Review. In this diagram, notice the particular position of the salary and it's weight as a demotivator rather than a motivator. 
Motivators and demotivators - Notice that salary (money) is not among the motivators, but rather a demotivator in case salary is not sufficient
Actually, what this diagram tells us is the same exact phrase mentioned in the RSA movie above at minute 4:50:
"If you don't pay enough, people won't be motivated. Pay people enough to take the issue of money off the table"

3. Using process data for the sake of individual performance

Forcing managers to use process data, like feature estimates, number of bugs, reported actual hours, etc. to evaluate team members will blow up the process metrics themselves. This, in turn, has a drastic effect on the process itself, making the team process-blind (not being able to improve the process based on process metrics) Being process-blind is the direct result of exploiting process metrics and reuse them as Targets, and as per Goodhart's law: "When a measure becomes a target, it ceases to be a good measure."
Why this happens? To understand the dynamics, let's recall the general rule put by Tim DeMorko, the author of the great book: Peopleware: Productive Projects and Teams's, a book which had an impact on the whole industry; the general rule is known as DeMarko's Principle:
"Effort moves towards whatever is measured"
This can happen subconsciously or even consciously; and this is considered one of the major pitfalls of measuring, as explained the this article: The seven pitfalls of performance measurement which considers: Risk of manipulation of performance data as one of them. When it comes to individual evaluation, use of process metrics becomes a real risk, and opportunity of manipulation is so widespread. Writers have been noting this again and again, to the extent that some writers like Eli Goldratt, the father of Theory of Constraints says:
"Tell me how you measure me, and I will tell you how I will behave!"
In summary, using process metrics to evaluate individuals distorts the metrics and after a while, the metrics lose their meaning and cease to reflect real process data.

A Final Word

Finally, I have to say that this topic is vast and needs a lot of effort to make a shift in the mindset of project and HR managers of software in Egypt. It is not easy, but impossible is not on my dictionary :)