17 December 2009

Predictions for the New Decade

"Prediction is very difficult, especially about the future."
Niels Bohr
I was asked yesterday to provide any predictions that I might have for the computing industry for the next year. Initially, I wasn't sure I had anything significant to predict, but then I remembered a thought that has been rattling around in my head for a while now.

First, it's hard to believe that this decade is coming to an end already. Ten years ago in December 1999 I was working on that project, which was the absolute antithesis of anything resembling Agile. A year later, I had found out about Extreme Programming and the rest is history. So, onto my predictions...

This prediction isn't so much for 2010, but for the upcoming decade. Looking back at two previous game-changing trends in computing - Relational Database Management Systems (RDBMS) and Object-Oriented Programming - you see a very distinct pattern. There was some initial early work in both areas, but once marketable products became available, both of those trends took about 10 years to become entrenched enough in the market to be considered by mainstream companies. At about that same 10-year mark, there was also significant breadth in terms of the number of products and companies involved. Indeed, it could be considered that at the 10-year mark those respective industries were quite fragmented - different products and companies pushed different approaches and tools.

At the end of 2009, I see the same thing happening in the Agile Software Development world. We are at a point where the mainstream is beginning to consider Agile as the way they should be approaching their software delivery process. We are also at a point where there is a good level of fragmentation in the market - Scrum, Kanban, Lean, Extreme Programming/Industrial XP, Feature Driven Development, RUP and DSDM are all in use to various degrees. There is a level of competition among these approaches not unlike that which occurred between various vendors of RDBMS systems and OO languages. We're also seeing a burgeoning market for tools in the Agile space, especially for those for project management and those that facilitate communication among distributed teams.

For 2010 and on into the coming decade, I see a similar pattern to these other trends. Market leaders will emerge, even if their approach isn't necessarily considered the 'best' by purists (which is arguably already being seen with Scrum). There will be consolidation among approaches, companies and tools, and even a standardization of Agile approaches. For the latter, early work is already occurring for the technical practices in the form of the Software Craftsmanship and Agile Developer Skills movements.

By December 2019, I expect that the number of approaches to Agile Software Development will be down to just two or three. These may not look exactly like the approaches we see today, but their roots will be quite recognizable because they will follow the Values and Principles of 2001's Agile Manifesto. Agile as an approach will be firmly entrenched in the Late Majority, with even Laggards considering it.

Finally, the success rate of software projects will become so high that The Standish Group will cease to publish the CHAOS reports and shut down CHAOS University by around 2016 or 2017. OK, that's a stretch but, with no disrespect intended towards Jim Johnson and the people at Standish, it's a laudable goal nonetheless!

All the best to everyone in the new year and new decade!

[New addition based on feedback from Bob Marshall and Lisa Crispin]

Something I wasn't clear about above - the few processes or methods left in 2019 will likely incorporate facets from all of the ones that we currently see. There will be some variation in order to accommodate different types or sizes of projects or teams, but in the end all processes will be high-level enough to allow individual teams and organizations to tailor the process to their own unique circumstances.

Something Lisa mentioned that I hadn't was that the term Agile itself will slowly diminish in its use over the next decade. This will occur to the point that it will simply be the 'default' way that software is produced, much the same as how the Waterfall became the de facto standard during the 80's.

14 December 2009

On Estimation

My family moved over the past weekend to the lovely little town of Richmond, ON (actually part of Ottawa, but it's still a community of its own). We decided that for this move we would hire a company to both pack and move for us.

A couple of months ago I had a few moving companies come to our old home and give us estimates of the cost. Each company sent out a sales representative who walked around the house, attempting to determine how many people and how much time it would take to pack and perform the move. One company came out with a significantly lower estimate, and they had moved other people that I knew. So, we decided to go with them.

On 'packing day' the two packers showed up just when they said they would. Within 15 minutes, though, they said that the sales rep had underestimated how much was there. In the end, what was estimated to be an 8 hour job took 12.5 hours.

The next morning the movers arrived, also right on time. It only took them about 5 minutes to start saying that there was more than estimated. The lead mover then spent the rest of the day griping about how the move had been underestimated. I spent the rest of the day griping about how the sales rep said that the move wouldn't cost more than the estimate.

By the time the movers had the truck loaded, they had determined that several substantial objects wouldn't fit without a second trip (and the associated extra costs). So, I had to cut scope.

OOOPS... I think I just gave away the point of this blog entry!!

Here I was, the Customer, being told that I had to cut scope or face additional charges. I didn't get to choose what was going to be left behind for me - the movers told me what wouldn't fit.

It sucked. A lot.

After some reflection, I wondered if sending at least one other person to handle the estimation would be better. The sales rep assured me that he had 25+ years in the moving business, and I have no reason not to believe him. But this is the same failing as having a business or systems analyst go off and gather requirements and determine the size of a project:
No matter how much experience you have, estimating in the absence of the people who will actually do the work will lead to errors that affect the ability to deliver all that is required, a quality product, or both.
When estimating, include as much of the core Project Community as possible. If that means juggling some schedules to fit everyone in, then do it. Having that input is critical since it provides sanity checks on the amount and type of work to be performed. Involving more people will increase the likelihood that someone may have an idea for a simpler implementation of a particular feature. Involving more people increases the level of understanding about the problem to be solved for everyone. Involving everyone increases the cohesiveness of the community that will work together.

I knew intuitively that, from the Customer's or Product Owner's perspective, the traditional estimation process where people talked to you then went away and dreamed up a schedule just didn't work very well. Now, having been that Customer, I know for certain that it's the case.

8 December 2009

WAQB Plagiarism

For the second time, the World Agile Qualifications Board (WAQB) has plagiarized the work of Lisa Crispin and Janet Gregory. The WAQB's Agile Testing Foundations course outline is almost an exact copy of the Agile Testing book's table of contents.

The Agile community wants nothing to do with these people.