Refactoring Legacy Organizations!

I have been involved in refactoring legacy code for so many organizations over the last 10 years. Also, I'm actively leading organization-wide transformation towards lean and agile. I've started seeing the similarities between the two "refactoring" efforts. And, here are some of the these interesting notes:

1. Stages are the same: First, do some quick-wins in code, which is the first step in the my roadmap in refactoring old code, then restructure it towards modular design with disjoint parts, and finally work on optimizing the whole and the parts inside each module. In organizations, there are very similar. First, fix the urgent issues, second restructure the organization towards disjoint self organizing teams, and finally, work on optimizing the relationships between teams and members in side teams. 
2. You can't see problems until you start resolving them
3. Problems build on each other. In code, you can have simple problem built upon another simple ones, and the chain starts complicating everything. In Organizations, a simple political issue leads to another simple issue, which may result in some dumb decisions and the chain continues. 
4. The earlier you tackle problems, the easier the solution is. 
5. Sometimes, you'll need to revamp part of your old code. This is exactly the same in case or organizations. Sometimes, it's even healthy to discontinue and redesign some departments, functions, groups, business lines, etc.
6. "Forking" code and refactoring it for long time without merging back is very risky. In almost all cases I've gone though the two branches become very different and the new forked branch is abandoned. Similarly, "forking" a shadow organization does not work. It increases the friction in the organization and ends with killing this new parallel structure which is, from the viewpoint of the "old guard", eating up the organization one byte at a time!

These are some thoughts. There are still many others, what do you think?

High-Performing Teams

In this keynote by Shane Hastie, agile product management guru and director of Agile Learning Programs at ICAgile. Shane mentions two very interesting studies about high-performing teams. The first one is related to psychological safety, and the second one is related to long-lived teams.

Psychological Safety

In work by google to know what makes a good team in google. They found that the single most significant factor is psychological safety. That is, you have support from your team to make mistakes and learn; in other words, your team will hold your back :) They were stunned to discover that technical superiority had no effect.

In sales teams, the presence of psychological safety resulted in an increase in revenue generated by this team above the norm with 21% and the absesnce of it resulted in a decrease in sales figures below the norm of 19%. This makes 40% difference, which is huge!

Long-lived teams

In an HBR article in December 2013 titled: The Hidden Benefits of Keeping Teams Intact, They say that "at a software services firm, a 50% increase in team familiarity was followed by a 19% decrease in defects and a 30% decrease in deviations from budget" just by leaving the teams intact.

Write a Book Exactly as If You're Developing Code

A book is a product and writing books is exactly the same as developing a product. I'm using XP and Agile technical practices and techniques to write my first book . Guess what? It works perfectly!

The story started sometime ago when I heard about LeanPub. It was cool and I thought I'll use it once. Now, I'm actively writing, or rather developing my first book: Refactoring to Clean Code:

The experience is very similar to developing a software product. Or, I can say it is exactly the same.

I'm not gonna talk about the book topics as features which may or may not suite your target customers, and how you should frequently pivot your strategy till you reach a solution market fit. These are very obvious similarities. Rather, I'll talk about the actual development of the book, producing book chapters, which is very similar to writing code modules.

I'm using version control, issue tracking, and full traceability (using github). I'm also using continuous deployment. Hah? yes, it is that black magic in software which enables features to be instantly deployed on production with very little overheads (for this, I use automated hooks which links githup with leanpub. Leanpub automatically generates a new version of the book upon every push from my side on github)

Finally, I'm using a full fledged development environment (Atom) which has realtime linter to check bugs on the fly (spelling and grammar mistakes). Atom in connects directly to github. So, I develop and commit changes on the same window.

Later on, I will prepare a test environment for my early adopters and reviewers. They will receive special editions from leanpub. They may open issues on githup, and even they may fix parts of the book and send me pull requests on github :)

Those who love coding, if they thought of the book as a software product, they will love writing as well :)

Would you like to become a reviewer (or early adopter)?

If you would like to become one of the early adopters, please let me know. Here is a link to the book with highlights about the contents:

https://leanpub.com/RefactoringToCleanCode

Guidelines for Reviewing my Book: Refactoring to Clean Code

• If you found a typo, change it right away and submit a pull request
• If you have a better wording or phrasing, you have two choices:
• Submit a pull request
• Open an enhancement
• If you open an enhancement, please link it to the section you're commenting on as such:
1. browse the file on github
2. hover on the section, you'll find a small link appearing as in the following screen shot. Copy the link and paste it in the issue. 

Lessons Learned from Mob Programming

Recently, I had enough time to experiment with mob programming, a revolutionary technique for developing software. It's like pair programming with 6-7 team members in stead of 2! Seems ridiculous, right? This is how extreme programming (XP) practices seam at first before they spread like wildfire!

For those who didn't hear about it, please watch this video:

To get a deeper dive, watch this lecture by Woody Zuill, the father of the idea.

Currently, mob programming is spreading so fast in the world, and people are starting to realize so much value out of it. In the Agile Conference 2017 (which I'm on the reviewers list of its program), we have two sessions on mob programming, one talks about this technique used for on-boarding large number of people, and the other talks about it in teaching colledge students.

Working With Mob Programming Onsite

Now, back to the topic. I worked with two teams to whom I introduced and used this technique. One of them in co-located, and the other is distributed. I'll introduce the setup of the onsite experiment, then will list some observations and lessons learned from both teams.

We have restructured the room, got a projector, and started the session. Here is what we've done:

1. We have dedicated an on-wall kanban board, other than the one on Jira
2. Started with three main tasks for the day, which are the most important ones so far
3. I have explained some basic rules for the game, one of which is that I'll act as the facilitator
4. Then we started!

Rules of the game

Several basic rules are put into action:

1. We'll break every now and then, probably once every 1-2 hours
2. It's ok that you leave the room for any side task, not for so long though
3. Any emerging unplanned task will be put on the backlog. We've started with three tasks on board, and ended up with about 13!
4. Work is time-boxed from 10 am till 4 pm. No other "significant" work should take place outside these working hours

How people felt about it:

"Some tasks would have never been carried-out without mob-programming them" - one team member

 It's very interesting seeing how people interact and collaborate during the session. Here are some observations I noted:

• Contrary to what I expected, technical leaders and development managers where happier with this technique. They felt more confident and more productive. Plus, they appreciated learning from other team members while mobbing!
• Learning took place in all directions. seniors taught juniors and vice versa. Old team members taught new one and vice versa. This was amazing! One can never think of what new ideas, techniques, even inspirations he'll learn from other team members unless he start working with them hand-in-hand
• Interaction and collaboration were high, definitely higher than solo-programming, probably a bit lower than pair-programming. Again, this is contrary to my expectation, which was that half of the team will take a long nap during the session :)

 Finally, here are some feedback points I got in the retrospective;

• It was fun! and joyful
• Some tasks would have never been carried-out without mob-programming them
• We used to do the same task so many times. This way, we all do it one time!
• Collaboration with the product owner is much better; we all ask him the same question, get the same answer, all at the same time
• It's a better approach to reach the best solution for a problem
• It's less suitable for small and easy tasks
• It may be impossible to work this way if we're working under pressure

All-in-all, it was an excellent experience. Right now, I'm engaged with a company doing remote mobbing as a means to train new team members in a faster and more effective way. Guess what? Still the old team members learn from new ones!

Ground Rules for Refactoring Legacy Systems

If you have a very poor code which is buggy, cluttered, ugly, scary, etc. If you have such code,  do one of the following:

1. Sunset this application
2. Ask your customers to stop asking for changes and fixes
3. Quit your job and switch to a more convenient one (other than software of course)

If you can't do any of these, then you're stuck with refactoring this code base. This is possible and even exciting.

Here is a set of ground rules to take into consideration before you start doing that:

1. Zero "more" defects policy: That is, stop working the moment you find the next bug, and fix it immediately. The rule is adapted from the Zero defects policy! which is a bit overwhelming at this stage.
2. Any "new" code must be covered by tests. Period.
3. Peer review is part of our DNA (till we're able to automate parts of this review, or all of it)
4. The Boy-Scout Rule applies to all of us. 
5. Automate all donkey work, one script at a time! That is, everything we do in a systematic way, can be scripted. What we should do is to automate all this work. 

If you do that, you'll find things slightly improving at the beginning, but exponentially improving later on, insha'Allah :)

How to find dead code?

Recently, I have read this interview with Kevlin Henney, an independent consultant and a great developer.

These are some very good insights about ways to find dead code:

1. Static analyzers are a good start
2. Search for files that has never changed since a while. "There are many reasons code may be stable — it’s just right, it’s just dead, it’s just too scary — but unless you investigate you’ll never know."
3. Runtime monitoring and Dynamic program analysis may be used to rule out parts of the code which is not dead code, effectively reducing the code under investigation

One very interesting note is that:

"Deleting dead code is not a technical problem; it is a problem of mindset and culture."

Furthermore, in this very interesting lecture by Kevlin, he tells the very famous story about Kinght corporation which lost hundreds of millions due some dead code left over. Also, narrates some very interesting stories alike. I highly recommend any developer to watch this lecture.  

Agile Outside IT - Agile and Sales

"By following the steps in Scrum, management learned that sales can lend itself easily to an Agile environment. Individuals found that this shared work model led to more consistent revenue and compensation in addition to a more even workload. No longer did the 'sales stars' work much harder than everyone else. Each team member focused on his/her own domain expertise."

This is a lesson learned mentioned in an article by Eric Kristfelt, a sales manager who worked for two Agile organizations. It gives a clear indication that an iterative agile process like Scrum was very helpful for a successful sales team.

Agile has strongly affected Sales and selling in two ways.

The Sales 'Iterative' Process:

First, sales is a process, sometimes described in 7 stages very similar to the waterfall process. This traditional process has become incompetent in dealing with today's fast market changes and fast learning customers. Scott Gillum discusses this deficient traditional process in his article at Forbes: The Disappearing Sales Process. Gillum relates a study which indicates that "57 percent of the sales process just disappeared", because today's knowledgable buyers already do this part on their own before they contact any provider.

Scrum, in this regard, has improved the sales process so much. Team started talking about adopting Scrum in sales, and doing marketing and sales sprints to improve learning about opportunities and customer needs, and improve fast response to customers. In this study, such fast response (when done within 5 minutes) makes the chance for a successful connection 100 times better, and the chance for successful qualification 21 times better!

Sales 'Cross-functional' Teams:

Second, sales traditionally focuses on the individual sales person. Now, thanks to the agile mindset, sales people are talking about cross-functional sales teams! Refer to Eric Kristfell article above for an example how sales cross-functional teams has created a better collaborative and successful environment, and resulted in the best target achieved in the organization. Also, this presentation is very useful example of how agile team roles maps to agile sales team roles.

Finally, I'm quoting Jeff Sutherland, the co-author of Scrum, telling us about this experiment adopting scrum with the sales team of ISence:

"Company revenue increased by 100% after Scrum implementation in the first quarter. Although market changes also contributed to this result, the general manager indicates that at least 50% of the revenue increase can be contributed to the adoption of Scrum"

The is enough for this episode, the next one to come tomorrow insha'Allah, about Agile in Finance and Budgeting :)

Agile Outside IT - Agile Education

Agile is penetrating more and more fields everyday. This is a list of areas which agile have very strong impact nowadays:

• Education
• Sales and Marketing
• Finance and Budgeting
• Strategic Management 
• Manufacturing
• ...

In short, any discipline which leans towards knowledge work will benefit one way or another from the agile values and principles.

This is the first post in a series. In this episode, I will talk very briefly about Agile Education.

Agile in Education:

The first reference is Steve Peha's Article at Infoq: Agile Schools: How Technology Saves Education (Just Not the Way We Thought it Would). What interesting in this article is the the very similar Manifesto for Agile Education which Steve came out with, with minimal changes from it original agile manifesto. Steve has not only mapped the 4 values and the 12 principles, he had also put helpful discussion of team practices which also applies to education!

Another attempt to mapping is done in this workshop by Hala Salah with some school teachers. After introducing them to the Agile Manifesto, Hala has facilitated an exercise with the teachers to come out with a similar value statements for Education. The audience have very smoothly came out with several versions of value statements which reveals the very close correlation between the Agile Manifesto and the field of Education and Study.

The second reference is www.agileineducation.org. Probably, the most notable thing in this website is the Agile Education Compass:

This compass "was created by a group of passionate Agile Educators who met face-to-face at the Scrum Gathering in Orlando, Florida between April 16-20, 2016". What's interesting about his compass is that it describes transformation as a journey from traditional eduction notions which has been in the mindset of educators since long like 'content', 'competition', and 'evaluation' to more appealing and agile notions.

The next episode will be about Agile and Sales. Keep Alert!

Detecting and Preventing Code Clones Mercilessly - workshop guide

Please follow these steps:

First: Open the code project

We will be working on the example project provided by ConQAT:

1. Open Eclipse
2. Goto: New Project > Examples > ConQAT Examples > ConQAT Examples Project

Second: Run the code clones analysis tools

1. Double click the clonedetection-example-junit.cqr 
2. Click Launch ConQAT analysis
3. After it finishes, goto Menu: Clone Detection > Open Clone Report, and select the clones.xml file in the output directory

Third: Detecting exact code clones

1. Back to the clonedetection-example-junit.cqr page
2. Scroll all the way to the right and click the 'New' button
3. Add the field 'equality', and enter value 1 in the threshold
4. Launch the analysis and reload the clones.xml file

Forth: Detecting similar code clones

1. Back to the clonedetection-example-junit.cqr page
2. Scroll all the way to the right and click the 'New' button
3. Add the field 'similarity', and enter value 1 in the threshold
4. Remove the 'equality' parameter
5. Launch the analysis and reload the clones.xml file

Fifth: Detecting gapped clones

1. Goto the 'ConQat blocks' section at the top right of the configuration file
2. Click on 'Change' button
3. Select 'Code Clones > JavaGappedClonesAnalysis'
4. Add a parameter called 'errors' and enter the value = 2
5. Add a parameter called 'gap' and enter the ratio = 0.2 
6. Launch the analysis and reload the clones-gapped.xml file (notice the file name is different than the previous ones)