27 December 2008

False Advertising?

OK, time to stir the pot a little. In the blog posts Scrum is not Enough and Call It What It Is, I've been critical of the fact that Scrum by itself is inadequate for providing teams with a sustainable, long term approach to building software. I'm not alone in this opinion - Jeff Sutherland and Ken Schwaber also teach the use of XP practices in addition to Scrum's:
Ron Jeffries on the XP Yahoo Group
Also Ron Jeffries on the XP Yahoo Group
Video of Jeff Sutherland presentation
Jeff Sutherland's Blog
Interview with Ken Schwaber
InformIT Article with Ken Schwaber and Kane Mar
At the same time, Certified ScrumMaster courses are booming and Scrum is becoming the de facto standard for Agile - when most people talk about "Agile", they are in fact referring just to Scrum.

I want to make it clear at this point that I do believe that there is tremendous value in Scrum and its practices. It will allow teams to improve their productivity. It forces more direct communication and more visibility into the development process. These aren't just good, they're great! Again, though, the long-term viability of the Scrum practices alone is questionable. Teams need to use engineering practices in addition to the Scrum management and team practices in order to be successful.

So, why don't I see this in the courses? Why are those, such as James Shore and Mary Poppendieck, who say that Scrum practices alone aren't enough being called out? Perhaps it's as simple as the fact that basic, out of the box Scrum is a very easy sell to management and teams.

Let's frame this as "Dave the Manager walks into the Software Development Process Big Box Store":

My team is thrashing and we need some help. What do you have?
"Hey, Scrum is fantastic! It will help you right away with communication, and prioritization. The best thing is that we're not going to tell you how to do your work! Your team is self-organizing!"
Nice. I'll buy that! CHA-CHING!!
"Oh, by the way - you'll need these few other practices to make sure it all works."
Huh? But I just bought Scrum?
"Sure, and it's great!! You just need to buy these few..."
But I don't have any money left to buy anything else!
"Oh. That's a problem. I guess you can get away with just Scrum, but those few other practices..."
You said that Scrum would help me!!!
"Of course it will. And these few other practices will help more..."
That's false advertising!!
"I BEG YOUR PARDON, SIR?!! HOW CAN YOU SAY THAT?! I told you that Scrum will help your team... you just need these few other practices to make sure it helps for a long time."
OK, so this is a bit over the top. However, can you see the point? If you're selling, teaching, promoting, or otherwise suggesting that someone use Scrum for software development and you aren't pointing out that the engineering practices must also be examined, then you are partaking in false advertising. I would imagine that in the vast majority of cases it's entirely unintended, but it's still false advertising.

Again, Scrum is good. It just doesn't provide enough in its own practices to sustain the level of improvement that can be achieved.

1 December 2008

Baby Blankets

I'm participating in a process review exercise at a client at the moment, reworking a painful, disconnected document-centric approach into something much more lean and collaborative. Our project sponsor, a VP, instructed the group that there were no sacred cows and that out of the box thinking was not only allowed, but encouraged.

"Cool!", I thought, "We can really make some changes here!"

However, after a number of half-days of wrangling, the process wasn't looking, smelling, feeling or sounding much different than that which it was to replace. That fact prompted one of the participants to opine:
Baby blankets. Some people just can't give up their baby blankets.
Nice... and very appropriate. Evidently I wasn't the only person who was at the point of gagging every time I heard the word "template". I'm tired of hearing that certain documentation must exist to satisfy external auditors without having those auditors in the room to clarify exactly what they need. I'm tired of hearing that dedicated project teams are impossible because there are too many concurrent projects at the moment (there's a hint in that sentence!!).

I'm tired of dealing with other people's baby blankets!!

These concepts/thoughts/artifacts only exist in their current form to provide comfort - they serve no other useful purpose, or at least that purpose has not been adequately explained.

Making any significant change to an organization is difficult, and forces people into areas of discomfort. When people are uncomfortable, the baby blankets come out. However, no meaningful change can occur if people don't accept that they may need to grow out of their blankets at some point, i.e. doing something different in order to obtain different results.

26 November 2008

Embedded Collaboration

You keep using that word. I do not think it means what you think it means. - Inigo Montoya, The Princess Bride
In a meeting today at a client, people were talking continuously about how the IT organization and business have to improve their collaboration. The business feels a considerable amount of pain about the lack of collaboration with IT, and IT in turn is feeling some pain collaborating with the business during a transition to Agile.

The problem is, though, that what these people are saying is collaboration isn't really what they think it is. Get together for a meeting, come to some decisions, write up some action items and go away to do some more work until the next meeting. All good stuff, right? Sure, and there's communication going on and people were working together. But is it the type of collaboration that's required to create a true partnership between business and IT? No.

Someone at this same meeting talked about how business was the Customer and the IT group served them. That doesn't sound like collaboration to me. It's a model for doing business, but it's quite deficient with respect to the delivery of systems.

What's required is Embedded Collaboration. There is no functional separation of business and IT on a project - they are living and working together, breathing the same air, meeting at the same water coolers. I worked in this model from 1992 to 1994 at a government client (see Agile Circa 1988), and at a smaller scale in earlier work. Having the development team located in the same physical space as the business it supports is an immensely powerful way to promote and support the level of collaboration required.

I'm interested to hear about other organizations that have used this model. Is it universally good? Are there hidden challenges?

20 November 2008

Agile Ottawa Meetup - Nov. 24th

There is going to be a meeting of the Agile Ottawa group (or groups!) on Monday, November 24th at 7:00 PM. This is an initial session, intended to get some discussion going. Here are the details:
Agile Meetup 7-9PM, November 24, 2008
At the The Code Factory, 2nd Floor - 246 Queen Street, Ottawa, ON
Mayford Technologies is proud to be sponsoring this event, so there is no charge.

Hope to see you there!

19 November 2008

The Practices Do Matter... At First

With all the recent talk about the Decline and Fall of Agile, I've seen a number of comments about how we should focus on the Values and Principles of the Agile Manifesto and not on the practices of any one approach. I agree with this to a point. Once a team has matured within a particular process then absolutely they can really begin to tailor it to suit their local circumstances using the Values and Principles as their guide.

However, a team needs to learn to walk before it can run, and the Japanese martial arts learning concept of Shu Ha Ri is an excellent model to use in this case.

In the first stage, Shu, a team is given fundamental practices that they watch and perform. Everyone performs these practices in the same way, regardless of their own personal traits.

When the team reaches the Ha stage, they have integrated the practices into their own way of thinking and are good enough at them to begin to expand beyond simply mimicking what their master (or coach) has shown them. The literal translation of Ha is "to break free" or "to frustrate", a dual meaning that explains very well what happens at this stage. The team is learning how to tailor the initial practices to their own environment and personalities, and the coach does indeed have some frustration with respect to the team departing from using the practices as taught.

The final stage, Ri, is when the team has fully integrated not only the "what" of the practices, but the "why". The translation of Ri is "to break free", which reflects how the team no longer needs the coach to guide them with respect to the practices. The team has improved to the point that they know what works for them and what doesn't, and can work within themselves to self-improve.

Going back to the Agile Manifesto for a moment, you can equate the Shu Ha Ri model to integrating Practices, Principles, and the Values. The problem is, the Manifesto only speaks to the latter two, with no mention at all of practices. So, how can a team reach the Ri stage with an Agile process without having a Shu stage to start?

Well, there have been a number of discussions recently about the use of XP practices with Scrum. In a software development context, Scrum avoids prescribing any technical practices, preferring instead to let the team self-organize to complete their work. This lack of guidance contributes to both Scrum's higher adoption rate, and to the challenges encoutered by teams using Scrum that continued to use their existing practices that had led them to seek out a better development process in the first place!

So, teams are using Test-Driven Development. They Refactor their code, and this is something that Ken Schwaber has recognized is required. They're using Continuous Integration. Pair Programming is being used, or some other form of near real-time code oversight.

In short, teams are realizing that the Practices do indeed matter. Without them, a team will improve somewhat, but it will likely be a case of We Suck Less.

To be able to achieve all of the potential improvement, a team needs to take the Practices and do them by the book until they are good enough to decide whether they work for that team. At that point, and only at that point will a team be capable of making the decision to drop or modify Practices.

Shu = Learn the Practices as is until you're good at them.
Ra = Ask "why" about the Practices, and experiment with changes.
Ri = Forget the damned Practices and just build some software!

The last point is only partially tongue in cheek - when a team reaches that stage, they are able to use the Values from the Manifesto to frame what they do on a daily basis. They know what works for them to enable the sustained delivery of high quality software and what doesn't. Furthermore, they know how to (and do) seek continuous improvement.

A team cannot reach this last stage without starting at the first - the Practices matter.

18 November 2008

Waterfall vs. RAD vs. RUP vs. Agile (Scrum, XP, etc.)

When I broke into this industry in the mid to late 1980's, it was pretty well a Waterfall world. By that I mean the minimal definition of waterfall from the top of page 2 of Winston Royce's seminal 1970 paper Managing the Development of Large Computer Systems that showed the flowing steps of Requirements, Analysis, Design, Coding, Testing, etc., and not the more comprehensive definition that followed in the rest of that paper. I was a good soldier, defining and designing everything up front... well, OK, sometimes I cheated and did some coding work because I was impatient or bored and needed to get moving forward, but generally I followed those steps.

In the early 1990's, along came this cool concept called Rapid Application Development (RAD). The consumers of the systems we were building were also impatient - they wanted value NOW, and were willing to have corners cut with the design and documentation. So, RAD espoused getting something - anything - into production RIGHT NOW!! That worked fine, except that you didn't spend much time on any of the activities that promoted sustainable development.

So, we went from too much process to too little.

At the same time, Object Oriented languages were coming into prominence and the development models created by Grady Booch, Ivar Jacobsen and James Rumbaugh (with contributions from Philippe Kruchten) were combined into a single process know as the Rational Unified Process. The RUP dealt with many of the shortcomings of both Waterfall and RAD - it was iterative allowing the constant feedback missing in Waterfall, and was heavily weighted towards architecture and risk management which was missing in RAD. Cool, now the industry could move forward with a standard process.

But there was a problem. It wasn't so much the process contained in the RUP but how it was packaged. The RUP was put together as a process framework, which was not unlike a large pantry containing all the ingredients you would need to make any number of meals. Unfortunately, people didn't seem (or didn't want?) to realize that you didn't need all of the ingredients for every project. The older Waterfall mentality was used when attempting to tailor the process to local needs, and when faced with paring down a large process people seem to err on the side of not paring enough.

What resulted was many projects using way too much process, which in turn resulted in more documentation, more design and less software. This was not what the RUP's creators had intended, not unlike what happened to Royce's definition of Waterfall.

So, in the late 90's, people tired of the heavyweight processes started rebelling against them. Along came what eventually became known as the Agile processes such as Extreme Programming, Scrum, Crystal, Feature Driven Development, etc. They focused on delivering value, and value to them was running software. Many of the Agile processes also added technical practices intended to provide the sustainability missing from RAD. They also overtly identified the people as being as important as the process itself.

Success stories abounded, not unlike what happened with each of the prior waves of processes. There were also, though, some failures. Those have, in my observation, increased over time as more teams started to use Agile processes. When looking at the causes of these failures, they often show that teams weren't using all of the practices in a given process, or they didn't use good technical practices at all and just adopted the management practices.

So now, is the pendulum swinging back towards more process?

There was an interesting post from David Anderson on the KanbanDev Yahoo Group this morning. In that post he states,
At this point, it is impossible to take the 'David' or 'Jeff" or 'Israel' factor out of the achievement of the high maturity.
This is in reference to increasing the maturity level of organizations beyond just "Agile", and to do so he believes that we must remove the human element not from the process itself, but from the stewardship of the process.

I think this is the correct direction - acknowledge, retain and leverage the human factors of executing the process, while at the same time removing the human element from managing it. Any team could be successful using XP if Kent Beck, Ron Jeffries, Jim Shore, etc. were running it. What we want to achieve, though, is a point where any team could be successful with (insert process name here) regardless of who is running it.

To answer the question, then, yes I believe that some more process is required. However, I think the amplitude if the pendulum swings is decreasing and eventually we're going to find a happy medium that's sustainable both within an enterprise setting, and over a long term.

16 November 2008

Agile Circling the Drain?

Yesterday, James Shore posted an interesting blog entry entitled The Decline and Fall of Agile. In that entry, he opined as to how the lack of specific direction with regards to engineering practices in Scrum has secured its place as the leading agile method in use today. That lack of direction is also a leading reason why organizations have had challenges sustaining efforts to move to more agile methods of delivering software. In a nutshell, I couldn't agree more. I've said roughly the same things in the past in my own blog entries, Scrum is not Enough and Call it What It Is!

"Uncle" Bob Martin responded in the Object Mentor blog as well as on Twitter, indicating that he believes the notion that the process is the problem is hogwash. He's correct, but only to an extent in my opinion. Individuals do indeed need to be responsible - they need to strive to constantly improve themselves and their techniques. However, the problem isn't only with the software developers! Testers need to do this, business analysts, interaction designers, business people - they all have to learn to improve how they do their jobs in order to make any process successful over more than just the short term. However, a process that leaves people to their own devices to figure out how to attain these improvements is courting disaster. Common sense, as they say, isn't very common.

That's why, to me, the more prescriptive approach of XP (1st edition) makes more sense in that it follows the Shu Ha Ri method of learning. Initially, you must show everyone practices that work, rather than expecting the people to just find them. I once heard Scrum described as "the VB of Agile Methods", for this very reason. It's easy to pick up and start, but unless you already have excellent practices in place you're going to struggle after a while. Think of the fancy 3D fonts and drag & drop controls that got rave reviews in demos, but the overall system was shipped full of bugs, if it even shipped at all. Don't even get me started on DLL Hell!

My point behind the Call It What It Is blog entry was that saying you're doing Scrum when in fact a number of practices were added out of necessity does a disservice to Scrum and the other processes from which those practices were taken. If you're doing XP but using Scrum terminology, just say so!! If you say you're just doing Scrum, the next group in your company that sees your success is going to say, "Wow, let's adopt Scrum!" If they do so without the extra practices, and aren't disciplined enough to be using them already, they will struggle after some initial success.

So, assuming that James Shore is right and Agile is on its way out, what's next? With one client, I'm seeing tremendous synergies with the application of Lean in the business and Agile (or whatever you want to call it) in the software development group supporting that business. The business people understand what they have to do in order to support the development people, and vice versa. Even the approach of introducing Lean to the business was the same as my approach to bringing in Agile for the development team - a short initial training workshop, then right in the deep end with a a coach there to guide the team through the first few iterations.

I can see that two-pronged strategy as being much more effective than simply trying to convert the development teams. Certainly it's more difficult than just changing the software development method, but isn't that the case with most activities in order to make them sustainable over the long term?

2 November 2008

Learning from Dogs

Our 6-year-old Golden Retriever Hannah is pretty typical of that breed. She's big, happy, playful, lovable and thinks she's a lap dog. She and our cat wrestle as if they're littermates, and she's best buddies with our rabbit. A born predator if there ever was one! :)

Hannah has also had to have surgery on both her hips because of hip dysplasia, a genetic defect in the formation of the ball and socket joint that leads to severe, painful arthritis. Her first was when she was 11 months old, and the second when she was 3-1/2 years. Both of those were successful, and she recovered to being a normal dog.

Late last February we were walking in a dog-friendly park here in Ottawa, and had taken Hannah's brother (who lived around the corner) with us. The two dogs were running around and as best as we can tell Hannah was bumped from the side while both her back legs were planted in the deep snow. She ended up having a sprained MCL ligament in one leg and ACL in the other. The MCL sprain healed fine, but in early April she completely tore the ACL.

The ACL tear, unfortunately went undiagnosed for quite some time. We had been told it was just a sprain, and to keep her resting. She would improve somewhat, but then would start limping quite a bit. Finally, after taking her to another vet, the complete ACL tear was properly diagnosed. The only way to correct it was surgery.

I do need to point out, though, that the only indication that something was wrong with Hannah was the limping. She never whined, wasn't aggressive at all, and still got excited about the "W" word (that's 'walk' for those of you who don't have dogs!). In my 25+ years of playing basketball, I saw 2 people injure their knees with torn ACL's, and both screamed in pain. One, a man in his late 20's, was literally in tears from the pain. Hannah, however, would still climb stairs and jump up on the bed if we'd let her (and sometimes even when we wouldn't!).

This past Thursday, Hannah went in for corrective surgery for her bad knee. The procedure is known as Tibial Plateau Leveling Osteotomy (TPLO), and the surgeon literally cuts off the top of the tibia bone and repositions it to support the motion of the knee. This is done in place of a graft or synthetic replacement for the ACL ligament, and is generally better tolerated in dogs. Hannah came through the surgery just fine, and we picked her up from the vet about 6 hours after she was finished. She was still pretty groggy, and was understandably not putting any weight on the repaired leg at all. It's going to take 8 weeks for the bone to recover sufficiently for her to begin rehab, which I guess will be her Christmas present.

So now on to the point of this entry. This is a dog who has just had her third major surgery in 6 years. On the first night, she moaned a couple of times, but generally just slept. On Friday she was much more alert, but still not quite herself. On Saturday, she was almost back to being normal in terms of her behaviour, and she was actually putting some weight on the repaired leg. This morning (Sunday), you would think that she hadn't even had the surgery. She putting weight on the leg (which we're trying to dissuade), she's as alert and perky as ever, she was playing with the cat, and was begging for food at the table during breakfast.

I do realize that the medication she's receiving is helping to control the pain quite well, but this is still quite remarkable. When it gets right down to it:

Dogs don't know how to be wimps.

A human in the same situation would be lying in bed or sitting on a chair. They would be moaning and groaning about how sore they are, and how long they're going to have to convalesce. Hannah, meanwhile, seems to be saying, "Hmmm, my leg hurts. Anyway, LET'S PLAY!!".

Perhaps when your lifespan is only 12-15 years, you realize that you don't have any time to waste wallowing in your fate. I think there's a lesson to be learned or some inspiration to be gained from this. While we can't always live like today's our last day alive, maybe we can live like we're only around for 15 years.

Suddenly, the ache in my lower back isn't quite as bad.

22 October 2008

Cool Prototyping Tool

I can't remember where I heard about it, but I stumbled across a prototyping tool last week called Balsamiq Mockups.

It's a simple tool for creating screen mockups, and I had some UI work to sketch out so I thought I'd download it and give it a try. Thirty minutes later I had bought the product ($79 US) and continued to use it to flesh out a UI layout.

I love this tool! It's easier to deal with than using just HTML or a graphics editor like Photoshop. It's much, much more lightweight in both cost and megabytes than something like Visio.

What I really like, though, is the look of the diagrams - they look like sketches on paper or a whiteboard. If you're using the tool to show some UI concepts to end users, they don't look at the result as if it's the actual system, worried about the choice of colours or whether a drop-down is used for a particular field. You can create some more elaborate mockups, but I haven't gone down that road yet.

So, all in all, it seems to be a great little tool and I had no qualms about paying for it.

24 September 2008

Call it what it is!!

It's an oft-heard complaint: a team reports that they failed while using Extreme Programming, or Scrum, or another agile process. Some investigation reveals that they didn't use all of the practices. Some advocates of the process in question say, "You can't say you were using XP is you didn't use all of the practices!" A spirited debate ensues, with a decidedly religious slant.

What I have yet to see, though, is the opposite occur. Increasingly I read about teams that have implemented Scrum, for example, and it was a great success. Excellent! Wonderful! Then I read that those teams "added some technical practices" such as Test-driven Development, Refactoring, Continuous Integration, etc.

Uh, if it makes sense to some people to complain when practices are taken away, then shouldn't there be complaints when practices are added? Is Scrum really Scrum with TDD, Pair Programming, CI, 2-week Sprints, etc.? Isn't is just XP with different terminology?

As I mentioned in Scrum is Not Enough, Ken Schwaber admitted that teams should be using technical practices in addition to Scrum. He said,
In order to employ emergent architecture, every Sprint must leave behind clean, commented, refactored work.
Scrum deliberately avoids prescribing technical practices, which is fine. However, if you still require good technical practices in order to succeed with Scrum, are you still doing Scrum?

I may sound like I'm picking on Scrum here, but the same applies to XP. Industrial XP, for example, is very explicit about adding practices to handle enterprise issues. As such, it was rebranded to reflect the difference. Why is this not applied to other processes?

So, my main point is that we need to examine the actual process being used very carefully and call it what it is. If it's Scrum with extra practices, then call it that - don't state that it's just Scrum. If it's XP with practices to extend beyond a single team, great! Just say so!

For a team I'm coaching right now, I've incorporated aspects of Lean Software Development, XP, Industrial XP and Scrum. The difference is that I've told everyone that I've done it, and not just used a single name.

After all, open honest communication is a common value of all agile methods, is it not?

20 September 2008

Ho Hum

A team I've been coaching since June shipped their first release this week. They had the Release Demo on Friday morning, and it was just like all of the Iteration Demos that had preceded it. The demo took 30 minutes, there were a couple of questions and everyone broke for lunch. The only difference this time was that the demo was given from the production environment, and we all went out for a celebratory drink... er, Retrospective... later in the afternoon.

Essentially, the release itself was quite anti-climactic. There was no crunch, or people scurrying about in a panic trying to ensure that those last few bugs were fixed and all was ready. It was actually somewhat boring.

And that's exactly the way it should be!

The only thing that was concerning the folks on the team was that after another iteration's worth of work, there are some questions about the next bunch of functionality to be built. The primary concern is that the people on the team don't want to leave it - they don't want to go back to the "old way".

There are some other interesting aspects to this. First, the Customer had indicated that once the Stories that comprised this past iteration were in place, the system would provide enough value to the business to be useful. Second, the development team didn't even blink at the notion of shipping to production again after another 2-week iteration. Third, this was not an all-star team - they were quite representative of the people in the IT organization. The only person that was possibly hand-picked was one of the business analysts, but that was more because of her subject matter expertise.

So, give good people the right tools, environment and support from the organization, and they will kick butt compared to traditional development methods. Moreover, they will do it in such a way as to be, well, boring! And that's exactly the way it should be!

22 June 2008

Crunch == Poor Management

I was conversing via e-mail with an old colleague of mine about getting together for lunch sometime this coming week. He responded, "We've got a bit of a crunch on, so we may need to delay for a week or so".

Once upon a time, I was part of that his team and way back about 5 years ago it was a reasonably agile group. Over time that group lost its Customer, and over time became less and less agile. Now, about the only practice used that could be considered agile is testing by the developers. It has never been said explicitly to me, but I believe that management thinks that agile "failed" this team, whereas from my perspective the team failed agile. When the going got tough, the team fell back on old behaviours, and I was the lone voice saying to stay the agile course. So, I eventually "changed my organization".

What caught my attention about my former colleague's comment was that these crunches are expected and normal. To me, however, they are indicative of poor management. Agile methods are no longer new - they have been proven time and again to be effective when the team actually follows the practices and doesn't cut corners when they feel some pressure.

In the case of my former colleague's team, I know their situation and what led to it. I warned them last October that this was going to happen, and it did. It's not because I'm clairvoyant or some Agile Coaching superstar but rather because, given all the indicators 9 months ago, the outcome was absolutely clear:
  • Planning was performed mostly at the start of a release, and tracking during was lax at best because the features to be built were far too coarse-grained to be able to provide reasonable estimates for their construction.
  • The team did not use iterations, so there were no checkpoints during construction to verify the team's progress.
  • Interaction between the team and the Customer was sparse, often once every 3 weeks.
  • The Customer very rarely saw the actual system during development - possibly a demo near the end of the development cycle - so there were few opportunities for them to steer the project to suit the business goals.
  • Acceptance testing was performed solely by QA people with little involvement from the Customer (although there was some from the Business Analysts), and the acceptance testing was 100% manual. While this was feasible 5 years ago with the initial embryonic system, it is currently the single largest bottleneck for the team's progress.
  • The code base has a very high level of technical debt. While the developers aren't at a standstill, they do still have to deal with unintended consequences of making changes. Fortunately, they have tests that cover about 40% of the code base which does give them something of a safety net.
  • The developers don't stop production when unit tests are failing. On more than one occasion, releases were performed when "the bar was red" because it was felt that it would be too much work to update the tests to make them pass.
  • Retrospectives are very rarely performed, and when they are very little is actually done based on the recommendations that come out of them. I wasn't the only one who made this observation - even some people who could be called the team's worst critics of Agile felt the same way and said so.
So, it's actually quite easy to understand why the team is in a crunch mode right now. There was a point a little while back where management declared that no one could take vacation over a space of a couple of weeks surrounding a release. While that may not sound like much, the mindset that went into that decision is telling. They are living in fear and don't know how to work any other way.

The truth is, though, they did know another way, but they abandoned it. If perhaps they had the guts to admit that the way the team used to work was better, then they wouldn't have these problems, and people could live normal lives.

Good management implements, inspects and adapts. Poor management scratches their head wondering why things are so tough, and says things like, "It's like this everywhere". Well, I'm here to tell you, it's NOT like that everywhere.

21 June 2008

Defining Moment

When coaching a team new to Agile, you're always looking for a defining moment. This is a point in time when the proverbial light bulb goes on over the team's head about not only how they should be doing Agile practices, but why. It roughly equates to the Transforming Idea in Virginia Satir's Change Model, and is a reasonably good indicator that the team is about to move from the Chaos stage to Integration and Practice.

This past week I was giving an introductory workshop to a team I'll be coaching for the next couple of months, and I used Jean Tabaka's Doggy Day Care Brochure example as a simulation of an Agile project. To my utter astonishment, this team had that defining moment within the first 30-45 minutes of a 90-minute exercise. The project planning was carried out with one person as the Customer, a couple of devlopers as Testers, and the actual Customer, a manager and a business analyst as developers. Everyone latched onto the concept, and immediately liked using index cards for planning. During "implementation" on a whiteboard, the developers noticed that their estimates were sometimes wrong, the testers found some implementation issues (including an edge case in the pricing!), the customer realized that a new story was needed. For the second iteration, someone who had to step out to another meeting returned and joined in as a second Customer. This really streamlined the process of feeding work to the developers.

During the retrospective, the person initially acting as the Customer highlighted that she felt overwhelmed and that she was a major bottleneck, which is something that many have pointed out as a weakness or risk area of Agile. It was interesting, though, to see someone experience that issue so acutely within a simple exercise!

I usually hope to see a defining moment during the first or second iteration, and certainly don't expect to see it in the initial training! The upshot of all this is that I believe this team is destined for success in their transition to Agile.

20 June 2008

Scrum is not Enough

A post by Ken Schwaber on the scrumdevelopment Yahoo group a little while back caught my eye. It was a response in a thread entitled "Scrum and Architecture", and Ken's response ended with,
In order to employ emergent architecture, every Sprint must leave behind clean, commented, refactored work. Otherwise emergence hits the wall within six Sprints (or sooner, depending on how bad the morass is).
I was rather surprised by this, since it's tantamount to an admission that the Scrum practices alone are not enough to provide a team with a process that's sustainable for anything other than the short term. It's not the content of the comment that's surprising - many people have said the same thing, myself included - it's the person making the comment!

Ken has been vocal in the past about the fact that Scrum deliberately avoids prescribing any technical practices. However this comment belies the fact that he already knows that without solid technical practices Scrum is simply not enough.

Is that the message being conveyed to groups new to any Agile development? I don't believe so. How may teams have adopted Scrum because it's the lowest agile common denominator, only to find out within a few months that "this agile stuff doesn't work"?

So, for the sake of anyone considering a move to an Agile process, let's make this perfectly clear:

Scrum is not enough!

19 February 2008


Scott Ambler of IBM was here in Ottawa last week, and I had the opportunity to chat with him over dinner before he spoke to the Agile Ottawa group. Our conversation triggered a lot of thoughts on my part about the state of Agile Software Development, and certainly my place in it after over 7 years as a practitioner.

Perhaps it's owing to my work in the federal public sector, but over the past couple of years I find that I have become somewhat disillusioned with Agile Software Development, or more appropriately its widespread adoption.

Many of us have experienced tactical-level success with some form of Agile. That success may even have been quite significant, but I question whether it's sustainable. Is the success a function of the process or the people who implemented the process? If key people who drove the successful adoption move on, does the team still adhere to the agile method they had adopted? Even after a successful adoption of an agile method or methods, can that success be sustained within one group, let alone be propagated to the rest of an organization?

There will always be organizations that seem to get it with regards to being agile. That doesn't happen by accident, and you most likely will find that those organization have inspired leadership with people who, regardless of the software development process, can be considered agile.

The question remains, though, what about the other 95% of organizations in the world? Are they simply going to be left out in the cold? Will they jump on the latest buzzword bandwagon, only to abandon it when their results are less than expected? Will they spend thousands upon thousands of dollars to adopt this new way of building software, only to drop it when they discover that it requires at least as much effort and discipline as other more prescriptive processes?

In the conversation I had with Scott, and in his presentation later, there was a lot of discussion about the notion that Agile has to "grow up". We can't rely on the shock factor of calling something Extreme Programming. We can't rely on the marketing hype of Scrum and the Scrum Master certification. We have to look beyond the clashing egos of the Agile thought leaders. The principles and practices behind Agile are sound, but they do need to be more realistic about the world in order to be usable in a practical sense. We can't always have co-located teams with a single on-site customer working on greenfield projects, as nice a scenario as that is. The myths surrounding the agile perspective on documentation and software architecture exist for a reason - we need to be explicit about their place in the agile world.

For me, the bottom line here is that Agile Software Development as a whole is in danger of becoming the next RUP... or RAD... or (insert favourite acronym here), doomed to relegation to the scrap heap of software processes. OK, so that's a bit harsh, but while we may have crossed the chasm, we may not like what we find on the other side.

5 February 2008

Abandon Hope!

In a meeting at a client today, a developer was giving a status update. He mentioned that he hoped to have something completed by the end of this week.

He hoped.

I've joked with colleagues before that there is simply no place for hope in the software development industry. When I said it, I meant "hope" in the "abandon hope all ye who enter here" sense, but when I thought about it, though, there really should be no "hope" in software development.

Progress should be based on reality, i.e Yesterday's Weather, and not on some optimistic, rose-coloured glasses view of what could be if the planets all align correctly. Software developers, their management, their families, pets and minor acquaintances are all notoriously optimistic when estimating how long it will take to accomplish a task. Therefore, when estimating, we need to draw upon experience and a proven track record. Where there is uncertainty, use a Spike to gather enough information to be able to provide an estimate that's devoid of hope, and full of evidence!

The developer could have said, "At my current pace, I will be done by the end of the week", or, "Without any other distractions, I will be done by the end of the week". Even better, he could have said, "The last time I did something like this [or this size], it took me 3 days. So, I should be done by the end of the week."

Semantics? I think not.

16 January 2008

Agile Ottawa Event - February 13th - Scott Ambler

The Agile Ottawa group is very pleased to announce that Scott Ambler of IBM will be our presenter for our February event on Wednesday, Feb. 13th.

When: Wednesday, February 13th at 7:00 PM

Where: The Theatre at BitHeads, 1309 Carling Ave. (Westgate Mall), at the entrance on the East side of the building.

What: Scaling Agile Software Development

The majority of organizations have gotten their feet wet with Agile software development techniques and are now hoping to take it to the next stage. However, they're discovering that the simple methodologies they initially adopted aren't sophisticated enough to address the complex situations the find themselves in. This presentation describes strategies for taking an Agile approach to distributed development teams, for addressing regulatory compliance requirements, for large teams, and for leveraging legacy assets. Techniques from the Rational Unified Process (RUP), Agile Modeling (AM), and Agile Data methodologies that are required to scale Agile will be described in detail in this presentation. Once you go beyond the Agile rhetoric you will find that you can in fact scale it to meet the complexities of the real-world situations you find yourself in.

Speaker Bio:
Scott W. Ambler is the Practice Leader for Agile Development at IBM Corporation. He works in the IBM Methods group developing process materials and travels the world helping clients to understand and adopt software processes that are right for them. A prolific author, Scott has received awards for several books, including those focused on the Unified Process, agile software development, Unified Modeling Language, and development based on the CMM (Capability Maturity Model). A widely recognized expert on Agile Process, he is a regular speaker at international IT conferences and a senior contributing editor for Dr. Dobb’s Journal. Scott also writes the Agile Software Development at Scale blog on IBM DeveloperWorks.

Prior to working for IBM, Scott led the development of several software processes, including Agile Modeling (AM), Agile Data (AD), Enterprise Unified Process (EUP), and Agile Unified Process (AUP). He holds a BSC in computer science and a MS in information science from the University of Toronto.

14 January 2008

Perfection Obsession

My wife & I recently traveled to Calgary, Alberta to attend the wedding of my wife's niece. We stayed with my sister and brother-in-law, who are the parents of the bride, so I had a front row seat for what can only be described as the insanity of the final gyrations of the process of putting on a wedding. Being tasked with nothing more than carrying stuff, driving here & there and remembering how to tie a necktie, I had the opportunity to see the association between the staging of this event and the delivery of software to production in a less-than-agile way.

First an assumption - I'm referring to a standard, North American, Christian-based wedding with a church service and reception. For the families of both the bride and groom this was the first wedding of their respective children, so they wanted to make sure it was perfect.

Wait a second... perfect? That term came up repeatedly during the 3.5 days we were there? The bride wanted such and such to be perfect. The mother of the bride wanted such and such to be perfect. The father of the bride wanted such and such to be perfect. The groom wanted such and such to be perfect. The parents of the groom wanted such and such to be perfect. People were at the point of physical and mental illness over the stress of this event. Why? Because they were absolutely obsessed with ensuring that the whole event was perfect.

My father-in-law (the grandfather of the bride) and I opined about this while driving to the church. There had been an explosion of emotion (relatively small - perhaps a couple of kilotons) the day before the wedding when it was discovered that they couldn't find all of the proper holders for the table markers. Uh, what exactly is proper? Well, they were short two (out of 15) of matching holders and that just wouldn't do. Tears and bad feelings ensued, and eventually the father of the bride drove 45 minutes across the city (one way) to a place where matching holders were found (which may actually have been an excuse to escape the insanity! :). My father-in-law wondered aloud what difference it would make, and he was told that "every girl dreams about her wedding, and wants it to be perfect". I asked if anyone had ever been to a perfect wedding, which immediately prompted a sense of déjà vu (more about that in a second). Actually, the more correct question would have been, has anyone ever noticed that a wedding wasn't perfect, or if so, did they care? The response was silence, but I'm not entirely certain if it was because anyone other than my father-in-law agreed, or that they were quietly plotting my painful demise. :)

The déjà vu to which I referred came from many presentations on XP and Agile Software Development I've given over the past number of years. I almost always started by polling the audience on their experience level. "How many people have less than 5 years in software development? How many have 5 to 10?" etc. I then ask how many of them had been on a project where nothing had changed from start to finish. There's always a knowing chuckle from the audience and I launch into my presentation. That helped me to see that there are some lessons to be learned from this obsession with perfection at a wedding, and in the delivery of software to production.

First and foremost, nothing is ever perfect. Ever. It's that simple. Any non-trivial system will have defects, just as every wedding will have a hitch or two somewhere. Some will be larger than others and can't be ignored - not having a marriage license or a bug that corrupts data fall into that category. Others will reduce the quality of the overall experience - my wife wanted Gospel singers at our wedding and would have been quite disappointed if we hadn't been able to get them, while an application may have a memory leak that forces the user to close it and restart every so often - but the show can still carry on. Still other problems may have what seems to be a tremendously inflated impact - the table marker holders or perhaps a query that doesn't run as quickly as we would prefer - but once 'production' occurs the 'user's don't even know that there's a problem.

The bottom line is that we can reduce stress in very significant ways by determining what's good enough for your given circumstances. At a wedding reception, that may mean having a DJ play quiet music during the dinner rather than hiring a string quartet. In software, that means performing risk and reward assessments on all defects and enhancement requests. Some stuff can simply be ignored, while others may need to become the sole focus of the entire team until they're fixed.

Another way that weddings and software development are similar is that nothing ever goes completely according to plan, so you need to either have contingencies or simply be able to go with the flow. When my wife & I were married, halfway through the ceremony my son (from a previous marriage) leaned over to me and said that he had to go to the washroom!! Well, when he says that, what he really means is that he had to go about 5 minutes ago and now he really, really has to go! So, I looked at one of my groomsmen, and he nodded and took my son to the washroom. They were back in two minutes, and everything went along smoothly and everyone in attendance had a good chuckle. This happens in software too, and often we need to deviate from the plan in order to accommodate an unforeseen circumstance. We don't have time to perform an impact assessment or have the Change Control Board provide their blessing. We simply have to deal with the issue right there and then.

Finally, the industries surrounding both weddings and software development have a tremendous impact. We are inundated by information from the media, colleagues, friends, etc. about what are the best approaches and tools. This happens in software too! :) Seriously, though, our expectations are inflated by these external influences and we can come to ignore our own feelings and experience about to approach putting on a wedding or building software.

I could also draw parallels between how it requires a collaborative team to handle both a wedding and software, how children can result from both, and, most regrettably, that they both have a similar failure rate. I think you get the point though! Just for fun, try describing your last project or two and replace all software development terminology with that which would be used for a wedding. You may be surprised at just how similar they are!