Estimating software projects

My Twitter stream flooded with estimation related topics yesterday. We’re currently starting up a new project, so the timing was perfect and the topic highly relevant. Here are some recommended blog posts from yesterday on the topic:

Estimation is at the root of most software project failures
In this blog post Rob Bowley explains why we’ll never be very good at estimation, especially the long range high level estimates – mostly for reasons outside of our control. Still many people in the Agile community talks about estimation as if it’s a problem we can solve. The discussion after the post is also highly recommended:
http://blog.robbowley.net/2011/09/21/estimation-is-at-the-root-of-most-software-project-failures/

You Mean I Can’t Even Estimate? The Planning Fallacy in Action
A post by Jim Benson, down the same alley as Rob Bowley. Explains how we might deal with the fact that there will be a variation in every activity, and that we tend to underestimate a given task even if we’ve done that task many times before:
http://ourfounder.typepad.com/leblog/2011/09/you-mean-i-cant-even-estimate-the-planning-fallacy-in-action.html

Estimate the total cost of Agile projects
Kane Mar first describes why estimating the cost of software is, at best, an educated guess. Then he explains how to deal with this when estimating the total cost of Agile projects: 
http://www.scrumology.com/2011/agility/estimate-the-total-cost-of-agile-projects/

Story estimation on Agile teams
A comprehensive Q&A by Praful Todkar. Who should be involved in estimation? Should we time-box estimation? Is pre-analysis necessary before estimating a task? What should we include in our estimates?
http://computellect.com/2011/09/12/story-estimation-on-agile-teams/

On the previous project we started off estimating stories before taking them into the sprint. When we gradually moved into the maintenance phase, we discovered that we only selected stories by priority – not size. Also, the stories were quite small, so estimating them seemed like a major overhead.

Not giving any real value anymore, we decided to stop estimating stories. Later, when we applied Kanban, we removed the iterations as well. We’ve managed well without estimation for about eight months now, instead focusing on the flow and maximizing throughput.

So here we are, about to start a new project. I think the time is right for us to reflect on why we estimate stories, and why we (try to) estimate the total size of the project. After all, for this particular customer, Scrum is the “company standard” – they’re used to estimating stories before prioritizing and starting them. Therefore, I think this is where we will start, this is what the company knows. But who knows where we will end up?

Do we still need estimates, or do we eventually get into trouble anyway when using them? Do you know other relevant blog posts on this topic?

Kanban and Scrum after 500 tweets

I joined Twitter June 22, 2010, so, needless to say, I was not an early adaptor. Nevertheless, I just passed 500 tweets, and most of my tweets are somehow related to agile development, so did I come across something useful? The answer is yes, and I believe it’s an appropriate time to reflect – focusing on Kanban and Scrum this time.

During this period I’ve learned to enjoy Lean thinking, and Kanban in particular. The difference, you may ask? David Anderson puts it well:
“People ask me what is the diff between Lean & Kanban? Answer: Lean is a destination, Kanban is a means to get there.”

If you don’t know Kanban, here is a clean and to the point overview:
http://reaktor.fi/en/our_expertise/kanban/

And here is a first hand experience on what you can expect if you introduce it:
http://www.agilecoach.ca/2011/07/07/kanban-reboot-pt1

It’s not always easy to map a process to the Kanban board, so you might need some tips when you are stuck: http://blog.brodzinski.com/2011/08/map-process-to-kanban-board.html

After a while, you will know that keeping an inventory is not free of charge, and there might be an upside if you manage to keep it down:
”In 1987, while GM held 2 weeks of parts in inventory, Toyota held 2 hours.” (Jeremy Lightsmith)

By then you should know that you need to pull work items, not push them. This strategy, if you see the analogy, might even affect your leadership:
“Management is push, leadership is pull.” (Jim Benson)
…and what’s really the difference between a project leader and a project manager?
http://www.projectsmart.co.uk/project-leader-or-project-manager.html

Kanban takes an evolutionary approach, so big changes won’t happen overnight, but you should still be prepared for resistance in the organization. Here are 5 arguments against Kanban – and how to respond to them:
http://noostvog.wordpress.com/2011/06/29/5-arguments-against-kanban/

Even though I find Kanban very appealing, I still like Scrum. Recently, Scrum even got a major update to the Scrum guide! I found it really interesting that they now had removed release planning, cause it might compensate for poor development practices:
http://www.scrum.org/scrum-guide-updates
This doesn’t mean you have to stop all release planning, it’s just not mandatory anymore, and you should use it with care. This is an example of one small step you can take to improve the quality.

And quality, that’s what you need to focus on, no matter what agile process you are using. If you shorten down the feedback cycles, that would help you on the way. To get fast feedback, there’s a number of things you might consider:
…shorten the iterations:
http://www.itjoblog.co.uk/2011/06/the-iteration-is-too-long.html
…keep the releases small and deploy to production often.
http://kenschwaber.wordpress.com/2011/03/23/major-releases-are-a-failure/
…but maybe not 50 times per day?
http://www.thehackerchickblog.com/2011/02/continuous-deployment-for-continuous-learning.html

If you want to go fast, you need to move with a sustainable pace. You need to stay clean so you can keep going fast:
http://www.scrumalliance.org/articles/300-the-land-that-scrum-forgot

So, to be able to shorten the feedback cycles, you need to focus on getting working software at the end of each sprint. And if you don’t, you’re definitely not alone. Over 50% of Scrum teams can’t get shippable code at the end of each sprint. Research shows that this will slow you down between 2 and 24 times:
http://scrum.jeffsutherland.com/2011/01/optimized-scrum-getting-to-done-done.html

There are many topics I didn’t cover this time, so maybe there is room for another post?

Visualizing project policies in Kanban

When David Anderson visited us last month, he mentioned that we must make sure project policies are explicit by writing them down. Recently, I had a some sort of epiphany that some of these policies could be visualized as well – directly up on the wall.

I prefer a whiteboard and post-its for our Kanban wall, at least as long as the team is co-located. This gives us an instant overview of the flow, and we gather around it to discuss how to optimize the throughput. It’s simple, but flexible.

To keep track of lead time and cycle time we update the tickets with important dates as we move the tickets across the board. However, I noticed that tickets weren’t always updated with necessary information for my statistics, so we needed a better approach.

First we standardized the way the tickets look, with fixed places for different items, but routine glitches still happened every now and then. Then we started to visualize the criteria for moving a ticket to a new state.

First arrow in the image above shows the transition when the ticket is prepared and moved into the backlog. The person moving the ticket fills in the current date in the upper left corner (the red square). Similar happens in the next transition from backlog to analysis.

When moving from development to test, another person must test the new feature (or defect).

When tickets are done, the completed date is added, accompanied by a subjective well-being using :-), 😐 or :-(. The latter is later used in the retrospective.

Most of these visual elements were added just a couple a weeks ago, but so far the notation seems to be easily understood and effective.

Are you visualizing any of your project policies?

 

P.S.: Thanks to Jim Benson for a great tip on how to measure the subjective well-being, as an input to the retrospective.

Improve team performance – seek inspiration elsewhere

As a teacher, my wife had to step in to avoid a conflict among sixth graders at school. She told me that she would let her students come up with different solutions on how to solve this conflict the following day. I mentioned briefly that she, in this particular case, could use a retrospective technique I have used in several occasions, prioritizing with dots, and let her students vote for the solution they would prefer. 

It slipped my mind for a while, I didn’t even go into details when I mentioned it, but a couple of days later she told me she actually had tried this technique out on her students. Some minor adjustments was necessary to make it more suitable for sixth graders and a chalkboard, but in essence it was the same technique.

She found the results surprising. While she expected the majority to vote for asking an adult for help, and at lower age they probably would, the majority voted for a certain way of solving this on their own – without adult intervention.

Instead of taking the answer for granted she generated new valuable insight of how her students think – and she only spent a few minutes totally. In addition she might have found the tipping point when children start behaving like adults in terms of conflict resolution.

For those of us who embrace agile development, we might find some inspiration in this to bring back to your workplace.

Retrospective can be successfully applied at any age and with all kinds of people, just remember to adapt it to your audience. Even the smallest effort can generate valuable feedback, and you might be surprised of what you find. This can help you improve your team.

My wife tested a new retrospective technique amazingly fast – with almost no preparation. It seems to me that they have a great environment allowing them to experiment. If they make a mistake, they get immediate feedback from theirs students and take a different approach next time. This is an important agile principle, find a faster way to fail, recover, and try again.

Create a climate in your organization that allow people to experiment and innovate, make sure there’s time to do this. Break a few rules. Take a step outside your normal boundaries and seek inspiration elsewhere, for instance see how they use visualization in a first grade classroom – the prototype for a creative learning environment. Some elements might be useful for your agile wall.

Question: Have you ever been inspired by something you have discovered elsewhere, outside your normal domain, and brought it back into your team? And how did it go?

Further reading: