25 June 2012

Test-Driven Development as a Design and Learning Technique - Introduction


The company for whom I'm now working is a Ruby on Rails shop, and prior to starting with them I hadn't done much more than Hello World in Ruby and its equivalent in Rails.  So, once I started I decided to use TDD as an approach and tool for learning the Ruby language and Rails framework, as well as a driver for everything I needed from an environmental perspective.  I wouldn't add or start anything new until I had a concrete need in the form of a failing test!

One of the first things I did with the company was to run a TDD demonstration session using the Prime Factors kata, which required me to learn a little Ruby and test-unit, and I used a combination of text editor and the command line on my Mac.  I had already done that particular kata loads of times in Java with JUnit, so it was quite straightforward to implement using Ruby.  Did I write the best Ruby code for the exercise?  Perhaps not, but I'm sure over time I'll improve.

So, I was past Hello World, but now needed more.

My next step was to do something 'real'.  I use public transit to get to work, since it's actually quicker, cheaper and less stressful than driving to the location in which I work.  The local transit authority recently opened up a REST API for obtaining live information about bus trips for specific routes and stops, and it sounded like something interesting to play with that had some concrete value, so I dug in.

I had a vision of an application that could take a geographic location, find the nearest bus stops within a given range, and give me a list of bus arrivals at those stops.  Ultimately, the system would be driven from Android and iOS apps using the Location Services on the device to provide the current location of the user.  I also envisioned having the user be able to select favourite bus routes and stops in order to filter the list to show only the buses that were of interest to the user.  I'm sure someone else has already done something similar, but I wasn't trying to build a system for commercial purposes.  The idea here is to learn while creating something with a real world application!

Here's a high-level view of the system:

This system appealed to me because it was going to slice through many areas that were new for me.  I had to learn the Ruby language sufficiently to retrieve, process and push data to a client of some sort.  I also have done nothing beyond Hello World in Android, and nothing at all in iOS.  This system provided many dark corners in which I could shine a light, and get my hands dirty again in code!

Again, though, the whole philosophy I will follow is to take a TDD approach - nothing new added from a code, library, environment or tool perspective until required by a failing test!

Stay tuned for more about my first steps into this adventure!

4 June 2012

Stoop and Scoop

We have a dog... a big dog.  Hannah is a very active Golden Retriever who is programmed to be one of two states - on a walk or wanting to be on a walk.  She accepts swimming as an acceptable substitute for walking, but that's another story.  So, each and every day in practically any weather we take her out for her exercise.

Being a dog, Hannah isn't particularly fussy about where she, um, takes a bio-break.  Now, we could simply look the other way and pretend that we didn't see what just occurred but, again, she's a big dog.  Since we follow a couple of well-worn routes around our neighbourhood and in the park behind, the accumulation of doggy doo would be quite significant very quickly.  Add to that all the other dogs, large and small, in our area and you would have piles upon piles of... I think you get the picture.

So, being the good citizens that we are we stoop and scoop, cleaning up the mess right there on the spot.  We diligently take it back home in a plastic bag and toss it in the garbage.  Most of our neighbours do the same, and the area is the better for it.  There have even been times when we've cleaned up the mess from other dogs whose owner were not quite so diligent, all for the greater good of our neighbourhood.

Imagine for a moment that the code in which you work is your neighbourhood.  When you are adding new code or making changes, do you want to be stepping in all the crap that other dog... er, code owners have left?  Do you think they want to step in the crap you 'neglected' to see because you looked the other way when your dog decided to stop?

Of course not.

When you're writing code, do the equivalent of stoop and scoop.  Ask yourself if the code you're writing, be it production code or tests, is adding crap to the neighbourhood.  If it is, clean it up!  I'm OK with the idea of getting the code to work and then refactoring to improve the design, but I can say from experience as both a programmer and dog owner that the smaller the pile of crap you have to clean up, the easier it is on the senses.

For the 'how' part of cleaning up the code, Joshua Kerievsky's Refactoring to Patterns is an excellent place to start.  His company, Industrial Logic, also published a cheat sheet for Code Smells to Refactorings that is a very helpful starting point for identifying what's smelly about the code and patterns for how to remove the smell.  These are small, stoop and scoop steps that will help you to prevent the construction of a large pile of crap that does nothing but smell!

So remember, other people have to live in the same neighbourhood as you.  Be a good neighbour - stoop and scoop your code!

(Please note - any "this is a crappy blog post" comments will be summarily marked as spam! :)

29 May 2012

Systems Thinking in the Back Yard

A few years ago I wrote a post called Rounding the Corners, which discussed my early foray into process improvement, disguised as my teenage desire to spend as little time as possible doing yard work!

Last fall my (now favourite) uncle was given a new riding lawn mower for his birthday, and he asked if I'd like his old one.  We have a pretty big yard so I jumped at the chance!  I only had one opportunity to use it last year before winter, but have been consistently mowing the lawn (and almost enjoying it!) this year.

This past weekend I noticed something interesting.  Since the riding mower doesn't have the ability to make the precision turns that a push mower can, I did spend time going over areas that I had already cut in order to turn.  At first glance this could be considered wasteful - after all it's rework is it not?  It is indeed rework, but in the grand scheme of things I'm still done cutting the grass much faster using the riding mower and able to do as much of nothing as I want afterwards.

Could I improve further by reducing the rework and tightening up my grass-cutting pattern?  Probably.  Right now, though, the pain caused by the inefficiency of the rework is insignificant compared to the system-level improvement provided by the riding mower.  At some point my time may become even more expensive such that I will want to save the few minutes currently wasted by driving over areas already cut.  For now, I'm satisfied with the large improvement!

This is an approach that we should take when working with teams, whether they're new to agile or seasoned practitioners.  What are the improvements at the system level that are going to make some of the current inefficiencies seem trivial?  Ask yourself questions like these:
  • Do we really need to force-fit 2 or 3-week iterations onto a group who is constantly interrupted by work coming in from the outside?
  • Do we really need to improve testing performance to enable TDD when the real improvement needed is with feature identification and prioritization in Product Management?
  • Do we really need to create cross-functional teams across a 500-person business unit when the real need is for a core group of about 50 to focus on the most important work?
There are always small wastes that can be removed in order to provide some improvement.  Sometimes these are referred to as "low-hanging fruit".  However, they often aren't the issues that need to be solved right now, and attempting to solve them diverts people's attention and the resources they use away from the problems that do need to be addressed right now!

So, when faced with what looks like some waste in a team's or an organization's process, stop and ask if that waste is really causing pain.  Is there something else that needs to be addressed first?  If so, work on that.  If not, then figure out how to make that riding mower even more efficient!

1 May 2012

Registration Open for Agile Coach Camp Canada - June 22-24, 2012

Registration is now open for Agile Coach Camp Canada, to be held June 22-24, 2012 here in Ottawa at the Travelodge Hotel on Carling Ave.

Registration is FREE, but space is limited so get your name in now! More details can be found on the Agile Coach Camp Canada web site, or you can register directly with the button below.

Eventbrite - Agile Coach Camp - Ottawa, Canada

10 April 2012

Pragmatic Agile - Remember the First Value!

Everyone, I'd like you to meet Al.  He's a real person, not a conglomeration of people or a fictional persona created for this post.  Al is a ScrumMaster at a recent client... a ScrumMaster with a secret.

I coached Al and his team quite a bit for the first few months I was engaged at that client, and then to a lesser extent for the next 18 months.  That team was very high performing, avoiding a lot dysfunctions that many teams encounter when they first move to any Agile process.  Their standups were reasonably effective, their defect rate was ridiculously low, they shipped meaningful work at the end of each sprint, and they sought constant improvement.  I started working with Al's team during their second sprint, and I was told that their first sprint was an "unmitigated disaster".  From my perspective as an objective observer, it really wasn't, but such was the attitude on that team that they believed they could do much better.  And they did!

This team had worked together as a group for years, many of them going back a decade or more together.  I often saw this team sitting together having lunch in the cafeteria, management included.  That's where Al's secret lies... he was the team's functional manager as well as their ScrumMaster.

I hear gasps in the audience...
You can't have a functional manager to whom the people on the team report as the ScrumMaster!  That's a conflict of interest!
Sure, in a good number of cases I'm sure it is.  Not with Al, though.  You see, although I hadn't met Al before he became the team's ScrumMaster, I could tell that he was a servant leader.  He comes from a  technical background, but doesn't meddle with the work of the team.  Indeed, he was originally appointed as the ScrumMaster so that the team members wouldn't be burdened with the role.  Al did whatever he could to remove obstacles and ensure that the team's flow of work would be as uninterrupted as possible.  Al was (and still is) a very effective ScrumMaster.

How could this be possible?  How could a functional manager possibly be an effective ScrumMaster?  After all, the functional manager has a position of authority over the team members.  He handles their annual reviews that affect their salaries and position within the company.  There's no possible what that such a relationship could work, right?

Yes, it can.  It has worked in many teams and many organizations long before the 17 good folks at Snowbird coined the term Agile.  Are there situations where the functional manager shouldn't be the ScrumMaster?  Absolutely!  In fact, I would suggest that in those situations, though, the functional manager probably should not be the functional manager either!

Many people have assumed that there is a rule stating that a functional manager can't be the ScrumMaster for a team.  Even Al and the Product Owner with whom he worked were somewhat sheepish when I arrived, apologizing for the fact that they weren't organized "properly".  There actually is no such rule in the Scrum Guide, and the list of "services" that the ScrumMaster provides to the Product Owner, Team and Organization sounds an awful lot like the things that an effective functional manager would have been doing prior to adopting Scrum.

It all comes down to the very first and most basic of the Agile values:
Individuals and Interactions over Processes and Tools.
When it comes to determine who should fill what roles in Scrum or any other process, it should be the individual people who are evaluated, not their job title.

In another business unit at this same client, a development group decided to remove the entire layer of first-level management, with all team members reporting to the director.  The former managers became the ScrumMasters for the teams, and there was no longer a reporting relationship.  That certainly dealt with the conflict of interest issue, but wouldn't the team members still view the ScrumMasters as their superiors?

Actually, no.  Those four ScrumMasters ranged from good to excellent in their role.  They were mindful of their "previous life" as a manager, and one told me that he had to actively suppress his project manager tendencies at times.  Another said that the transition from functional manager to ScrumMaster was easy for him, because facilitation and impediment removal was already a part of his job prior to the change.  So, again, people who are already servant leaders should not be discounted or disqualified from becoming a ScrumMaster simply because of their previous title.

I have seen organizations attempt to apply Agile - be it Scrum, XP or some other flavour - as a recipe without any understanding of the values and principles.  Those are situations that are ripe for failure because the values and principles form the basis for any and all things that are considered to be practices in Agile.  This sort of blind faith in practices is what causes problems.

Consider this TED talk by Barry Schwartz about our loss of wisdom:



So, if you find yourself in a situation where a decision about a practice makes you feel uncomfortable, reflect on what value or principle the practice represents.  Question if that .  Question when a practice is being omitted, what other practice or practices will be put in place to serve the value or principle.

Don't be afraid to question why the values and principles aren't being followed. After all, it's not the practices alone that will enable you to be agile - they are the means to an end. To be agile, you have to actually think about what the values and principles mean and how they apply to everything you do.

5 April 2012

Agile in its Second Decade

For quite some time I've been working on a post to commemorate the 10th anniversary of the Agile Manifesto.  I suppose that, as the 11th anniversary has now passed, it would be rather silly to continue to celebrate the 10th! :)

Part of my foot-dragging has been the feeling that I wasn't adding anything to the discussion about a decade of Agile.  Every now and then something would spark my interest and I might add a bit to the post I had started, but it just wasn't resonating with me.

In January, though, Dave Nicolette returned from a blogging hiatus with the post “Agile” considered harmful.  His thoughts about "post-chasm Agile" really shook loose a lot of feelings I've had for the past number of years.  More recently, I met up with a couple of friends in the Bay Area for dinner.  They too were disenchanted with what the Agile world has become.  They used the term "ashamed" to describe how they felt about being associated with Agile, and those conversations triggered my Occupy Agile blog post. In that post I also talked about yet another discussion in 2008 with Scott Ambler about the state of the Agile movement.

All of us shared this common feeling that the Agile world as it is today in 2012 is not what we learned years ago.  Each year it seems that there are calls to rewrite the Agile Manifesto, a new certification scheme pops out of the ground, more automated tools are created and marketed to simplify collaboration, and we move further and further away from the values and principles that defined the movement in 2001.  The shame wasn't a reflection of the original purpose of Agile being wrong, but of how the people in the community have behaved.

I agree that the Agile Manifesto was a statement, a line in the sand, made in the context of the world of software development of the 1990's up to the dot-com implosion.  It specifically focused on the issues within software development, which is one of the key problems cited by people people who believe a new Manifesto is needed.

Many of those people feel that we have solved the software development problems and now the issues are elsewhere.  Scott Ambler wrote about this in Reworking the Agile Manifesto in 2010, Bob Marshall in Agile Coaching is Evil, and Adam Yuret in Working Software isn't Enough.

I agree with all of those posts that there is more to fix than just the software development process.  I also respectfully disagree, especially with Bob, that we needn't focus on just the software.  If I didn't see as much really crappy code as I do, I would be more open to the suggestion that we can declare victory in the software practices war.  Of course, I'm not hired to come in and look at how wonderful everything is, but I would like to be pleasantly surprised just once by clear, expressive code following Kent Beck's 4 Rules of Simple Design, integrated continuously with effective and efficient automated checks running constantly.

Actually, I wouldn't be surprised, I would be shocked.

I suppose, in deference to the opinions of people like Scott and Bob, I should stop looking at the software trees and view the whole organizational forest in order to address the bigger issues that lead to developers writing crappy code.  I wish, though, that I had confidence in the belief that making the silly organizational pressures disappear would magically solve the software problems.

I don't.

Yes, we need to solve the big problems around matrix management, siloed business units, focus on efficiency and utilization vs. effectiveness.  I try to do that in practically every engagement I have.  I also deal with developers who really, truly don't believe that they should be testing their work.  They don't believe that integrating more than once a month is important.  I deal with testers who only want the final product to test.  The don't believe that an investment in automated testing is important.  I deal with architects who believe their job is to design the hell out of everything so that the developers can't screw it up.

In other words, I have to deal with the very problems that the Agile Manifesto set out to solve in 2001.

Are there bigger problems to be solved?  Absolutely.  Should we focus on them?  Absolutely.  Have we truly solved the problems in software development?  In some cases, yes, but in what I would argue is a vast majority of cases, not by a long shot.

So, to me, over 11 years after it was created the Agile Manifesto's values and principles still apply.  In some situations they are less important than in 2001, but in many more situations very little has changed since then.  I will certainly work to improve the organizational environment in which teams exist.  I will certainly work to foster much higher levels of collaboration (and ideally integration) between the development teams and the business people for whom they are building systems.  I will certainly work to help organizations integrate Lean thinking into how they approach not just the software being built but the whole process from "concept to cash".

I will do so, however, while also working to ensure that the way they build software is as good as what I learned from XP (and later Industrial XP) over a decade ago.

4 April 2012

Occupy Agile

I understand that the "Occupy" meme and movement is "so 2011", but after some conversations with friends over the past few days I've begun to realize that I'm not the only person who thinks that the Agile world has been moving further and further away from its origins in the values and principles of the Agile Manifesto.

I've had multiple people now say that they were beginning to feel that they no longer want to be associated with the term Agile - they feel ashamed by the behaviour they see in others in the Agile world.  They feel that there were far too many people who were in it just for a buck and really had little or no qualification to be training or coaching teams, and indeed were doing as much or more harm than good.

While I know many very good people in this part of the software industry, I had to agree with my friends.

Back in 2008, I wrote in a post called Disillusionment about another dinner conversation with Scott Ambler in which we commiserated about the state of the Agile world at that time.  I concluded with the suggestion that, as it was currently progressing, Agile was "in danger of becoming the next RUP... or RAD... or (insert favourite acronym here), doomed to relegation to the scrap heap of software processes".  Unfortunately, I can't say that what I see now is any better than in early 2008.  I'd actually say that it's worse.

Where I feel we've been led astray is in the pursuit of money over the delivery of value.  Let's face it - the fact that the CSM program from the Scrum Alliance helped bring Agile to prominence was a side-effect of a scheme to make money.  The jury is out on whether ICAgile is just another group to peddle certifications, and the PMI now has the PMI-ACP program offering certifications in Agile Project Management.  I'm undecided about Lean Kanban University, but you can be sure that there's a business plan behind it.

I'm not at all averse to people making money - I'm as much of a capitalist as anyone else.  The key difference is that I believe that if you deliver value to customers, the money will follow.  If you simply chase money, you lose sight of the value you're trying to deliver because you have to compromise your own values and principles in order to "make the sale".

I don't want to do that anymore, and I'm encouraged that others feel that way.  I'd like to find out how many people feel the same, and perhaps we can Occupy Agile - bring it back to focusing on values and principles and in doing so delivering value.

If you haven't already seen the little poster I created on Twitter and Facebook, here it is:


If you weren't a fan of the X-Files back in the 90's, I apologize for the obscure reference.

1 March 2012

The Power of Community Agreements

I recently attended and spoke at Agile India 2012 in Banglore, India.  It was a wonderful experience - the conference was excellent, and my first trip to India was memorable to say the least.  I'm quite sure that I'll be returning to see more of the country and experience people and the culture!

Coming from North America, there was one particular aspect of Indian culture that was significantly different than what I've experienced so far - the traffic in Bangalore!  From my 30+ years of experience driving in Canada, the U.S. and France, Indian traffic appears to be complete chaos.

Here's a famous video from YouTube that shows traffic flow in India.  Needless to say, crossing the street as a pedestrian is not for the faint of heart:


If we were to drive like that in Ottawa, for example, the carnage would be indescribable!!  However, it all just seems to work, and I saw fewer collisions of any sort than I would have seen during a similar 2 week period in Canada.  Why is this the case?

Community Agreements.

In Canada we expect people to stay in their lanes at all times except when safe to change.  We expect people to (roughly!) follow the speed limit.  We expect people to follow all the rules as laid out by Transport Canada and the provincial transportation ministries.  People generally do this, although speeding is one thing that I saw little of in India.  In fact, I didn't see a single police car on the road enforcing traffic laws.  In a 1 hour period this morning here in Ontario, I saw three!  In Canada, honking your horn at another driver is something done as a last resort, and can trigger episodes of road rage.  People have had their vehicles vandalized and have been beaten up in fights as a result of honking.  If you blink your lights at someone on a highway, it's an angry or impatient signal to "get out of my way".

By contrast, in India drivers don't even seem to notice that there are lane markers on the road.  They at least respect that (roughly!) you drive on the left side of the road.  Drivers are constantly honking and blinking lights... as a way to communicate with other drivers.  Honking at someone is a way of saying, "I'm here!"  Indeed, many larger trucks have a sign on the back encouraging you to honk to indicate your presence.  Blinking your lights at another vehicle is another way of showing that you exist, and that you're 'coming through'.  There were several occasions when our driver passed other vehicles on the highway when there was oncoming traffic, and we had 3 vehicles across a two-lane road.  No problem - everyone knew that everyone else was there, and we all made enough space to pass safely.  Try that in North America, and you would likely have a multi-vehicle head-on collision with major injuries.

The fundamental difference between traffic in Canada and India is that the "community agreements" of how we drive are different.  In Canada those agreements are based on well-defined and rather well-enforced rules, and that every driver follows them.  When a driver in Canada deviates from those rules, the likelihood of a collision increases greatly.

Traffic in Bangalore, IN - photo by Eirik Refsdal 
In India, though, the community agreements are based on constant communication, such as honking and blinking your lights, and respect for others on the road.  I saw many, many more instances where a driver would allow another driver to move into the space in front of him to keep traffic flowing.  The two drivers would make eye contact, possibly honk at one another to acknowledge their existence, and the other driver would move into the space.  I can recall only a single time over two weeks that the person driving us threw up his hands in exasperation at another driver.  In a two week period in North America or Europe, how many times would you do that?  In India, there are rules, but they are only followed in spirit and aren't strictly enforced.  The use of speed bumps and speed humps provides a low-tech way to control speeding without the same need for patrolling police cars as I see in Canada and the U.S.

Another key difference between India and North America is that the Indian road infrastructure (at least that which we experienced around Bangalore and Mysore) is not built to handle the same volume of traffic as that in car-crazy North America.  However, even though it may be slower to drive in India than in Canada, what I experienced was that there was still constant flow regardless of the number of vehicles.  Given the same infrastructure in a Canadian city with drivers following Canadian community agreements, you would have gridlock.

Community Agreements in Teams
Teams of people are similar in that they also have Community Agreements - founding principles on which they work.  Those principles are usually only implied, although some are occasionally explicitly stated.  They will be similar from team to team, just like drivers in both Canada and India adhere to the rule that you drive on one side of the road (the right side in Canada, left in India).  They will also be different from team to team, sometimes even in ways as fundamental as the difference between principles behind driving in Canada and India.  The key point, though, is that regardless of whether a team is even conscious of the existence of their community agreements, they exist.

Each team will over time evolve a way of working that suits them.  When I start coaching a new team, during training I ask them to discuss their particular agreements and try to make them explicit.  This can take the form of very concrete items such as "our working hours will be such that we're all together between 10AM and 3PM each day" or the coding standards to which the team will adhere to more abstract value-based agreements such as "we want to have fun while working".

The best teams I've observed as a coach and worked on as a developer have been the ones that had the fewest rules.  They were more similar to Indian traffic where respect and constant communication resulted in fewer fender benders than a long list of rules to which the team had to adhere.  There would be the same honking and light-blinking, but not in the angry North American sense, but rather in the spirit of traffic in India.  The area in which the team worked had a 'buzz' to it as people talked, but not to the extent that it was disruptive.  For example, if someone needed to communicate something, there was a discussion instead of an e-mail.

Contrast that with a "rules-based" approach where communication is routed upwards to a supervisor who contacts the appropriate supervisor of the other group, when then communicates within her team.  The communication reaches its destination, but how effective is it?  Was anything lost in translation?

Communication between people is a flow not unlike traffic.  As such, I would suggest that the only way to handle the chaos of working effectively as a team of people requires us to embrace the same level of communication and respect as that of Indian drivers.

Trying to enforce rules as a means to control that communication would result in gridlock!

6 February 2012

Pragmatic Agile - Be Principled!

In my previous post on Pragmatic Agile, I mentioned the 6th and 8th principles of the Agile Manifesto.  Just to be clear, I'm not a religious zealot who has the principles memorized chapter and verse and can recite them in order... naming them by their 'number' simply provides an easy reference without having to restate the principle every time!

In that previous post I related how I have encountered numerous people and teams who avoided changing their existing ways of working in the name of being pragmatic.  I can understand to an extent why they felt they were being pragmatic, especially given the first definition above - to them Agile is a theory, and they are afraid to make changes based on theory rather than their own experience.  However, that is NOT Pragmatic Agile, that's simply... well, doing what you have always done and saying that it's Agile.  If that is working for you, then great!  Just don't say that you're Agile.

So what does it mean to be Agile, then?

Around a decade ago I would have told you that you had to be doing Extreme Programming.  Others would have said Scrum.  Others, Feature-Driven Development.  Some, DSDM.  Yes, those are Agile methods that each have their own set of practices, which often overlap from one method to another.  When Agile was defined at Snowbird in 2001, the people behind those various methods boiled down their practices into what was common among them.  That led to the Agile Manifesto's Values and Principles.

The values are a statement of where the software industry was in the late 1990's and where the people at Snowbird felt it should be going.  In North American culture, we would refer to the values as "Mom and Apple Pie" statements - no right-minded person would disagree with them!  By being values, though, they were deliberately vague on how one would implement them, and they also represented a spectrum that could vary from organization to organization, team to team, even person to person.  That's good, but not enough to really be able to make concrete changes.  "No more documentation?!  Hell yes!!  We'll just bash out code until the cows come home!"  Well, it doesn't really work like that.

The real meat of what came out of Snowbird was the 12 principles of the Agile Manifesto.  They are what truly define Agile and what have endured over the ensuing 11 years.  Interestingly, if you were around the Agile world back then, you can also see the compromises between the various methods reflected in the principles. ;)  What isn't a compromise, though, is that those principles are the benchmark for saying, "We are Agile!"

The first principle states,
Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
Since first reading that, I've heard almost every excuse in the book against implementing that principle.  We can't possibly ship something incomplete!  Our users can't cope with changes every few weeks!  Our system is so large that it takes our customers weeks to install and validate - just when we'd be releasing again!  We have a certification process to follow for each production delivery that's very expensive so we can't afford to delivery early and often!

Yes, those are valid points... when you are working from older assumptions.  There are plenty of ways to address those concerns and still deliver early.  Scrum has spoken for years about completing a Potentially Shippable Product Increment each Sprint, although you may wait until you have a Minimum Viable Product (MVP) before you release to production.  Defining the MVP is a key part of the Lean Startup movement.

As a concrete example, consider credit card billing integration.  In one case I encountered, a system looked like it was taking a credit card and having it authorized automatically, but in reality it was storing the card information in a secure area and generating a quick e-mail to a mailing list.  Someone in the company would then take the information and manually call the card company for authorization.  It worked just fine, the system's users weren't aware there was a manual process in place, and it meant that the credit card billing portion of the system could be put into production much sooner than if it had been integrated with an automatic authorization service.  Bear in mind this was a number of years ago and today there are numerous SaaS solutions for handling credit card processing, but it illustrates the point that you can think outside of the "it all has to be automated" box in order to ship your Minimum Viable Product sooner.

Perhaps you don't have to deliver to production at all, but rather an environment that people can test the system as if it were live.  Another approach is that people could also still use the old system, but be redirected to the new functionality when they needed it.  As for users being overwhelmed by changes, the shorter the release cycle, the fewer changes that will be introduced and the more whelmed your users will be!  If you don't believe this, ask Google and Amazon how overwhelmed their users are when they release new functionality every 2 weeks (I suspect that, like me, you are one of those users!).  Sometimes that new functionality is only released to a small population of users in order to get quick feedback and minimize the risk that all users will be bothered by the change.  There is a well-known concept known as A/B Testing in which a system's users will be presented with two different presentations of a system in order to determine which works better.  The ultimate form of this kind of fast feedback mechanism is the Feature Fake, in which you don't actually implement anything, but just a button or link that pops up a form telling the user that the feature doesn't exist yet, and possibly asks some questions about whether they would actually want that functionality.  In other words, a simple, cheap concrete experiment that obtains real results to help determine if further investment is required.  That's very pragmatic, is it not?

What I find interesting is that the Lean Startup movement has taken this principle, embraced it, and moved to a new level.  Eric Ries' assertion is that following this principle is almost the only way you can be successful as a startup.  Yes, we hear about the really successful companies that started as a couple of people putting some code together, but we don't hear about all the failures that take untold amounts of money down with them.  Eric argues that you must release as early as possible in order to receive and act on feedback in order to ensure that you're meeting the market need you thought existed.

That sounds like it makes perfect sense, right?  Steering your work and your investments of time and money using what Eric calls Validated Learning... actual practice instead of theory.  Pragmatism.

The second principle follows this pattern as well,
Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage. 
If you're following the first principle, then you have to be ready to accept that the requirements from which you've been working will change.  Not might, not could, but will.  Indeed, even if you have been following a traditional serial development process when have the requirements not changed?

I wrote last year in Whither Requirements that the real goal of any requirements process is to achieve a shared understanding of what is to be accomplished.  This is a really, really tough thing to do and requires much more time than people care to think it does.  The biggest issue with serial processes is that they work from the assumption that once the requirements phase is done, well, it's done.  There is no room for clarification.  There is no room for learning while implementing the requirements.  There is no room for "oh, well I actually meant...".  Change Control Boards are created explicitly to prevent change, because any change is considered scope creep which represents a threat to the budget and schedule.  Budget and schedule are indeed important, but not at the expense of innovation and the inevitable learning about the real requirements of a system while it is being built.

If you stop for a moment and look at the second sentence of this principle, you'll notice that the Manifesto authors went out of their way to explain why changing requirements are important.  If you are working on a project with a 12-month release cycle that locks down the requirements after 3 months, even if that system ships on time the requirements will be 9 months out of date.  That might be acceptable in some domains, but in my experience it just isn't good enough even in public sector domains!

On many teams with whom I've worked, an early release plan contained everything but the kitchen sink in terms of features that just had to be delivered.  Once those teams started working in a mode where they delivered early and often and accepted changing requirements and priorities, an interesting thing happened - many of those "kitchen sink" requirements simply fell off the list.  Once the system was being touched by its users, the true priorities and truly important requirements became clear, and the pet requirements or wish-list items simply were no longer considered necessary.  One group dropped a full 20% of their initial items after 5 iterations... think about the cost savings that represented for the business!

As before, the Lean Startup folks have taken this approach to the next level.  They start from the position that you may have an idea for something that people might want, but until you turn that idea into an experiment, run the experiment and receive feedback, your idea is simply a hunch with as much chance of success as randomly picking a feature an implementing it.  In the Lean Startup world, each experiment will likely create changes in your requirements and thus embracing that change is required if you want to be successful.

So again, success is tied to actual validated practice vs. some unproven theory.

It's pragmatic.

2 February 2012

"Pragmatic Agile"

I've never been one you could call academic.  I'd like to think that I've always had my feet firmly planted in the real world, however you define it.  My focus has always been on delivering value rather than adhering to some theoretical principles.

For example, when I first learned about Extreme Programming in 2000, I didn't read Kent Beck's Extreme Programming Explained first, but rather Extreme Programming Installed by Ron Jeffries, Chet Hendrickson and Ann Anderson.  While both books described XP, the former wandered more into the theory while the latter was more along the lines of, "This is XP and here's how we implemented it."  That approach of applied theory appealed to me, and I was able to apply it myself shortly thereafter.

I'm a huge fan of the Pragmatic Programmers, Dave Thomas and Andy Hunt.  Their first book, The Pragmatic Programmer, is a must-read for anyone entering into the software development profession (and a should-read for most who are already developing software professionally!).  Again, their real world approach appealed to me, and they have turned the whole Pragmatic thing into a franchise of its own.  The books that they publish for other authors all have that feeling of providing concrete advice that you can use immediately.

However, I do have to admit that I cringe every time I hear the word pragmatic.  While Mssrs. Thomas and Hunt have what I believe the right approach to pragmatism, time and again I see that word abused.

Cherry Picking
I can't begin to count the number of times that, as a coach, I've heard programmers, testers, business analysts, line managers, product managers, directors, executives, and probably even the cleaning staff say that they couldn't possibly use some Agile practice.  Instead, they would insist on continuing to do what they had always done.  After all, we have to be pragmatic in how we apply Agile, right?  Well, yes... and no.

This notion of taking what you like from Agile methods such as Scrum, XP and Kanban is known as cherry picking, and in my experience it leads to serious problems almost 100% of the time.

Agile Values and Principles
I'm OK with people substituting practices.  I honestly believe that there are many ways to accomplish a task.  However, if you don't link the practices back to the Principles and Values of Agile, then you are going to experience difficulty.

For example, consider the 7th Agile principle:
Working software is the primary measure of progress.
What that means is... well, I think it's pretty clear what it means.  It does not mean a colourful Gantt chart generated by MS Project by someone who attends the standup meetings and tracks each task by each person down to the hour.  Even if that is done in a spreadsheet, completion of tasks and even the completion of Stories or Backlog Items simply represents activity.  True progress is shown when software is working and in the hands of those intended to consume it such that you can receive concrete feedback on that software's suitability to the business task.

The use of Velocity - Story Points or some other measure of the size of items completed over time - as a means of measuring progress can be (and is often) easily abused.  Despite that, I've always loved that the original XP team chose that term.  Speed is a simple scalar value, denoting distance over time, whereas Velocity is a vector, which has a directional component.  If you aren't receiving feedback on whether what you're building is actually what is supposed to be built, you could be going in completely the wrong direction.  In other words, you aren't measuring Velocity, you're measuring Speed!

The use of the term working also means that the teams don't simply hack together some spaghetti code in order to get the current stories completed.  That leads to the 9th principle:
Continuous attention to technical excellence and good design enhances agility.
I love asking groups of developers to whom I'm speaking if anyone there is a crappy developer.  Of course, there's always one or two who put their hands up and we all have a chuckle.  I then put up some of their code that I had asked for prior to the talk, and start to talk about all the Code Smells in it.  The excuses then absolutely pour in:
  • That was written by the people in (any site but theirs)!
  • We had to get that done before the release!
  • I wanted to clean that up, but my manager said I didn't have time!
Sorry, but a 3,000-line function with a Cyclomatic Complexity of 500 that grew to that size over 10 years is inexcusable.  Apparently, though, it isn't uncommon whether you're working in the procedural or the object-oriented world.

I see developers constantly taking shortcuts in the name of expediency, being pragmatic, about meeting a deadline vs. creating solid, maintainable code.  The ironic result is that rush to meet the deadline creates defects and opaque code that causes them to miss the deadlines even in the short term!

The same applies to automated testing of code.  I agree fully that it's neither practical nor necessary to ensure that you are hitting 100% code coverage in your low-level tests.  That's especially the case when you are applying tests to existing legacy code - you need to evaluate the risks associated with the work you're doing, and test appropriately.  However, for greenfield work it's relatively simple to start testing immediately and maintain those tests over time.  That applies to languages from C to Ruby, Java to Python, and almost everything in between.

Regardless of whether you're dealing with a legacy Big Ball of Mud or working from a clean slate, making the investment in automated tests will make you go faster.  That is the pragmatic approach!

So, if you are choosing to change or omit some practices from Scrum or XP, you need to ask yourself if you can still satisfy the Agile Principles that are part of the foundations for those practices.  If you can, then you are probably being truly pragmatic in your approach.  If not, then you need to look deeper at what needs to be done in order to improve how you work.

Stay tuned - more on pragmatism within the context of the Agile Principles in a later post!

31 January 2012

"Guaranteed to Pass"

On Twitter today I saw a tweet that mentioned a CSM course that "guaranteed" that you would pass the CSM Exam.

I already believed that the whole CSM thing was a bloody scam to make scads of money, although I'm willing to admit that it helped make Agile (or at least Scrum) more visible.  However, I said several years ago that I didn't believe in these sorts of certifications because they eventually became meaningless.

First, there is no way to truly certify that someone understands Agile or even Scrum.  That takes months if not years of constant practice.  Second, the moment the test was introduced to verify that people didn't sleep through the CSM course, I knew that the trainers would start to do exactly what I saw today - make the course material fit the exam.

I have a little experience in that area.  In 1995 I was working with PowerBuilder, and a colleague suggested that I should get my CPD - the Certified PowerBuilder Developer - certification.  His logic was that there weren't that many, perhaps a dozen, in Ottawa at the time and it would be advantageous for me as an independent consultant.  So, I took his advice, studied hard for a couple of weeks and took the two-part test.  I actually failed (got 74% instead of the minimum 75%) on one of the exams, but passed it on the second attempt.  I learned a lot while studying, but I also learned that there were certain ways of answering the questions that were based on what the manufacturer of PowerBuilder wanted you to answer rather than what I knew to work.  I was very careful on the second attempt to answer "what they wanted to hear", and passed easily with a mark over 90%.  I was proud of the certification and the work I put into obtaining it, and it did lead to better contracts for a couple of years.

Fast forward to 1998, and the organization in which I was working brought in a couple of new entry-level people.  Both of these people had just graduated from one of the business school programs where you learned computer programming in 8-12 months, and came out with certification in Visual Basic and PowerBuilder.  These were smart, very nice people who had come from backgrounds that weren't technical.  They decided, however, that the jobs were in technology and they signed up for the program at the business school.  These people graduated and came into the job with the same certification I received in PowerBuilder, meaning that they knew the material well enough to pass the exams.

The only problem, was that once they deviated from the material they had covered to pass the exam, they were lost.  They had no idea how to write well-factored object-oriented code in PowerBuilder, which was a challenge at best.  They had no idea how to deal with error conditions.  They had no idea how to handle concurrency issues in any sort of elegant way.  They had no idea what the difference was between synchronous and asynchronous calls.  They had no idea how to handle internationalization.  They had no idea how to optimize system performance.  They didn't have to know these in order to pass the exams, and yet I had already learned them before I took it because the other people with whom I worked who had more experience than me knew that those things were important.  Studying for the exams added perhaps 5 to 10% to my existing knowledge, and yet it represented 100% of the knowledge of the people from the business school.

Congratulations, Scrum Alliance and CST's who "guarantee" passing of the CSM test, you are about to create even more people who will be lost once the real world deviates from the material you covered.

25 January 2012

Agile Transitions and Human Nature

Like many people, I've struggled with my weight since my late 20's.  I'm currently in the process of losing weight yet again, and have passed the 20 lb. (approx. 10 kg) mark.  This isn't the first time that I have lost that much weight - I attained a loss of 24 lbs. about 15 years ago, but each time the loss wasn't sustained (or even sustainable).

This time, however, feels different and it has given me some (rather obvious) insights into some of the issues I have encountered with Agile transitions over the past few years.

Intrinsic Motivation
First, and foremost, you as an individual have to want to make the change.  Yes, that falls squarely unto the "Duh!" category, but a significant part of wanting to make the change is understanding clearly why the change in necessary.  People often have sudden health issues or a similar crisis that triggers their desire to lose weight, exercise, or make some other behavioural change.  Indeed, starting the change is relatively easy - my father used to say,
Quitting smoking is easy... I've done it hundreds of times!
The tough part is making the change last.  That requires motivation that is more than superficial - the type of intrinsic motivation that Dan Pink speaks of in The Surprising Science of Motivation.  Put simply no change, whether it be losing weight or moving from a serial product development lifecycle, is going to work unless you really, really want it to work.


This means that the leaders in an organization must not only talk the Agile talk, but they must walk the walk as well.  A senior manager cannot just say, “Thou shalt be Agile”, she must get to the gemba and attend Daily Standup Meetings, go to Sprint Review / Product Demo’s.  She must let go of matrix management practices and allow long-lived teams to be created.  She must eliminate silos in her organization in order to allow groups to come together determine the best way to deliver their product or system.  She must be willing to eliminate a “fail-safe” environment in favour of a “safe-fail” one.

Leaders don’t have to be managers.  Respected software developers, testers, infrastructure people and project managers, for example, also have to embrace this new way of working.  For a transition to succeed (or even just get off the ground) these people must be engaged and willing to give new practices a try.  They must be given the time to learn not just the practices, but the underlying principles and values.  The people who look to these leaders for guidance need to be given time & support to learn the values, principles and practices.

At the grassroots level, though, a certain amount of cynicism can be expected.  After all, Agile is just the latest fad that management is foisting upon them and, like all the other fads, it will be gone in 6 months to a year.  The key to success is to expect that there will be a short term hit in productivity as people learn a new way of working.  Providing support in the form of on-going training and coaching will help minimize that productivity hit, and ideally maximize the improvements.



Support
In order to obtain the required intrinsic motivation, one of the first key steps is to ensure that you have the support of others around you.  Yes, you can lose weight by yourself while everyone else in your household is continuing to use the same eating patterns, but your chances of long-term success are reduced.

This applies equally to an Agile transition.  A single development team tries an Agile approach and sees quite a bit of improvement.  When people try to take that improvement and get other teams to adopt Agile, there isn't as much success.  Why?  Often there isn't the support required in other teams or management to make the change stick.  Sometimes that comes from within the team itself, sometimes from their management, and sometimes from both.  For example, how motivating is it for the pilot team members who worked really well together to be split up and used as "seeds" for making other teams Agile?

There are many stories of attempts to introduce Agile methods as a grassroots movement from development teams.  In almost all of those cases, the attempts didn't lead to substantial change beyond the original team.  There just wasn't the support outside the original team to allow Agile to grow, and there were likely many non-Agile principles and practices that continued to be used that stifled any attempts to change.  That's similar to someone putting a huge bowl full of pasta in front of me for lunch and dinner, with some rich chocolate dessert after each meal.  If others don't support the change, the change likely won't stick.

So, this does require some education of those who must support the change.  The truly successful Agile transitions I've seen have been a combination of executive and grassroots movements.  At the executive level, managers understand that their previous ways of working are no longer sustainable and are willing to incur the shorter-term costs of change in order to achieve medium and long-term gains.  At the grassroots level, the people "in the trenches" understand why the executives want the change to take place, and know that they have the support to learn how to make it work.

Both levels, and everyone in between, needs to understand that the pace at which each individual will change is going to be different.  Indeed, there will be some people who will refuse to change regardless of how much support there is, and how much sense it makes to put the change into effect.  If you don't believe this, simply look at how many people smoke almost 3 generations after the landmark 1964 Surgeon General's report.

Discipline
OK, here's the tough part.  Making the change, whether you're a CIO or a programmer, requires discipline. Yep, the 'D' word.  That often turns people away immediately.
Sorry, I don't have time for that.
While psychologists have long said that it takes 21 to 28 days to make a new practice a habit, new research from the UK suggests that value is actually 66 days.  If you assume those are working days, that's over 3 months.

From an exercise and weight-loss perspective, I can say that it took about 2 months to change my routine and diet from a point of "this is different - I have to work at it" to "this is the new normal and I don't want to change".  For the longest time, I didn't want to exercise first thing in the morning because I have always found I'm more stiff.  However, trying to exercise in the evenings was running face-first into the wall of excuses - I'm tired, I need to get 'x' done, I ate too much, etc.  So, when I too a realistic look at my schedule, I had to admit it just wasn't working and changed my workouts to the morning.  Yes, I was stiff, but I loosened up quickly enough once I got going.

Some Agile practices can be relatively painless and easy to pick up, such as iterative development and showing working software at the end of each iteration.  This is especially so if your initial foray into Agile is with a greenfield project.  However, some practices are going to be more difficult and will indeed feel like you're writing with your wrong hand when you first try them.

Retrospectives are a good example of this - many teams have held "post-mortems" after projects, but that assumes that something is dead.  Traditionally, not many teams have taken time during development to review how their are performing.  For many people I've encountered, the Retrospective seems very "touchy-feely" and uncomfortable.  Over time, though, people learn to decouple a critique of the process from team implementing the process.  In that respect, the Retrospective becomes a very valuable tool and even a social event!

I can also speak from personal experience with Test-Driven Development that it took a couple of months of daily use to move from "this feels like I'm writing with my wrong hand" to "I don't want to write code any other way".  I was lucky in that I had a partner who was also willing to give TDD a solid try before passing judgement on it.  We paired almost 100% of the time as well, to the point that we both prefer to pair with others now, over 10 years later!

A common issue I've seen with Agile adoptions that falls into this category is "cherry-picking" of practices.  In many cases, any practice that is seen to be difficult or requiring discipline is discarded from the start.  That would be OK if the people doing so understood the Agile principle or principles behind the practice and they substituted practices that supported the principle.  However that usually isn't the case.  One of the reasons why Scrum is so popular is that it didn't include any technical practices.  Teams that have been truly successful with Scrum have been the ones that learned and understood the Agile principles and ensured that either their existing technical practices supported the principles, or they chose new practices that did.

The key is to ensure that the type of support mentioned above is going to be there for the time required for the behavioural changes to truly become new habits.  You must be willing to invest 3-4 months in a team to make those changes stick - even longer if you're dealing with multiple teams.

Tools
As I mentioned before, this isn't the first time I've tried to lose weight and exercise.  I played a lot of sports when I was in high school, and still played competitive basketball into my late 30's.  I was also a long-distance runner on the track team back in the day, and running has always been something I liked.  Since running is something I can still do, it has become my primary tool for cardiovascular exercise.

I also invested in a treadmill some 15 years ago.  I'm an expert at finding excuses to not go outside and run, so the treadmill was bought to prevent me from saying that there's snow in the Rocky Mountains (3000km away), so I'd better not run today!  Even when I was a competitive runner, I hated running in the winter and in the rain.  So, I bought a good quality treadmill to remove that excuse.

Over the years I accumulated some dumbbells, a weight bench and even fashioned my own device to exercise my forearms.  A couple of years ago I bought some surgical tubing for exercising after seeing it during a stint in physiotherapy.  Most recently, I bought a large exercise ball and a 10 lb. medicine ball to augment my "tools".

Finally, I went to Home Depot and picked up a bunch of packages of the soft foam floor mats and created a decent workout area in our unfinished basement.  The cost of all the items wasn't insignificant but it was literally spread out over 15 years, with most coming in the last 5 or 6 years.

All of these things came together to create a critical mass of having a good area to work out and enough exercise equipment to allow me to do many different  exercises and to run inside away from winter in the Great White North.

From a diet perspective, I had learned about making smoothies from frozen berries, milk and protein powder a few years ago (more about that in a moment), but it required the use of a blender.  Our traditional blender did the job, but it was loud and a pain to clean afterwards.  During an unrelated trip to a local Costco store, I noticed that they had the Magic Bullet on sale for something like $40.  I took a small leap of faith and bought this seemingly redundant tool.  I now have a smoothie almost every day for breakfast using the Magic Bullet.  It's quieter than our blender and much, much easier to clean.  It enabled me to start and continue my diet because it lowered the bar for using it.

Shortly after I started running again, I bought a new pair of shoes.  I went looking for ones that would provide some more cushioning to replace shoes that were about 5 years old and left my knees aching after an outside run.  Was there anything 'wrong' with my older shoes?  Not really, and they certainly weren't worn out.  However, with them I was experience an adverse reaction in the form of sore knees that was providing an excuse to not run!  I was fortunate enough to be able to afford to buy those new shoes, and my knees haven't ached since.

So what tools does a group entering into an Agile transition need to support them?  I advise taking the same approach - what do you have now, and what simple additions can be made?  What can be done to provide the proper environment, like I did by getting the foam pads for the concrete floor in our basement?  What subtle changes can be made to existing tools to remove demotivating factors, like I did by buying new shoes?

Remember, the goal is to provide an environment in which the proper behaviours become easy to do.  Yes, the treadmill was a significant expense in 1997, but it has enabled me to run regardless of the weather.  What can you do to remove your team's excuses for sticking to unproductive behaviours?
  • Eliminate cubicles in favour of an open workspace for your teams
  • If that change is too radical, rearrange tiny Dilbert-esque cubes into 'pods' of 6 to 8 desks where the central space is open, and people can simply turn around if they need to have a conversation with a colleague
  • Invest in as many whiteboards as you can possibly fit in any given work area
  • If you absolutely, positively can't have all of your team members in the same physical space, invest in tools that allow them to be there 'virtually' - place a web cam so that anyone can view the card wall, for example
  • Examine your existing tools and practices with those tools - something that made sense when you were working on a 6 month cycle may no longer make sense on a 2 week iterative cycle
  • Modernize your development and testing tools - open source software is your friend in this case... you can get almost everything you need for free, and it's often better than something homegrown or purchased for many thousands of dollars!
  • Provide easy access to snacks and refreshments - don't underestimate how important that is
Leverage Existing Knowledge
When I started this latest journey, I didn't simply discard everything I had learned before and start from scratch.  As I said before, I like running - I believe I first started when I was 13.  I'm not big on cycling, and I pretty much hate cross-country skiing outright.  So, since I already had the equipment to support it, the decision to use running for my primary exercise was simple.

However, that alone wasn't enough.  I had also learned about 10 years ago about how the human body reacts to the consumption of carbohydrates.  That was back in the days of "No Carb" diets, and I wasn't interested (probably my Irish heritage and thus love of potatoes!).  One variation of those diets I did see that was interesting focused more on controlling your insulin output by avoiding carbs except for one meal per day.  While I didn't apply the diet, that idea stuck with me.

A few years back I started going to Greco Lean and Fit, a local physical fitness franchise that advocates a holistic approach by advising you on diet and providing high-energy circuit workouts.  It was there that I learned about the protein smoothies and supplements, and how to manage food intake better.  Their workouts also showed me how variations can keep you interested, i.e. don't always do the same old thing!  For a number of reasons, I stopped going to those classes, but again it was always in the back of my mind.

When I started working out again, I originally thought that I would literally train for a while so that I could go back to Greco without injuring myself when I was there!  However, I found that with a few tweaks and a couple more tools I was able to create my own exercise space in our basement and do my own 10 exercise/3 repetition circuits, followed by a run.  It has worked wonderfully, and I've removed the excuse of having to drive all the way to the gym, or that their schedule didn't work for me!

What knowledge, what parts of your existing environment can you leverage to make an Agile transition work?  Are there tools that people are using that are working really well for them?  Are there existing practices that everyone says, "Yeah, that works really well!"

If your group has an existing monstrous build, don't simply get rid of it but rather look for ways to make it faster.  What are the bottlenecks?  Can newer, faster hardware be thrown at it?  Are there ways to build portions in parallel?  Are there ways for developers to run local builds that may not represent the production grade product, but those builds are very fast and enable the developers to use Test-Driven Development?

Similarly, if your existing test suites take hours or days to run, what can you do to reduce that cycle time?  If you do decide to change testing environments, don't throw out the old one immediately.  Instead, use the new environment for any new development and for validating bug fixes.  Gradually replace the old test environment, rather than a big-bang cutover.

From a people perspective, ensure that before starting a transition and certainly during that everyone involved is consulted and asked for their input on "what works".  Leverage the collective knowledge of your people, which is a much more productive approach than simply tossing out everything that was done before the transition.  You don't have to spend months doing this, but certainly investing a few days finding out what the people who have to live in the testing and development environments really like and don't like is well worth it.

You have to be somewhat judicious, though.  One key factor in my diet that seems to be helping with my motivation is that I eat whatever I want for dinner.  I have my smoothie for breakfast, some nuts and a piece of fruit for lunch, and water or coffee to drink.  At dinner, however, I can eat anything.  When we go to my Mom's place, I'll eat whatever is for dinner as well as dessert.  So, I don't feel like I'm suffering by dieting the way I am.

So, although I can eat whatever I want for dinner, I no longer have grab a soft drink in the afternoon like I once did.  I don't get a muffin or a doughnut with my morning coffee.  I made changes, but just didn't completely change everything.

How can you and your teams take that approach?  How can people such as Architects and Build Managers incorporate themselves into the teams, leveraging their deep knowledge of specialized areas?  Can the teams use their existing tools to move to Acceptance Test-Driven Development, or even true TDD at the microtest level?  Can they continue to their existing electronic tool for holding requirements and live with duplicating them on index card or stickies in order to have a real, tactile, card wall?  Can their current workspace be rejigged slightly to make it more open?

When working with groups that are in the early stages of a transition, I often hear the phrase,
We can't just throw the baby out with the bathwater!
I couldn't agree more!  Look for what has worked and worked well, and look for ways to incorporate it into the new way of working.  However, be aware that when using an Agile approach what was 'efficient' before may be a local optimization now.

Short, Medium and Long-Term Goals
I didn't become overweight and out of shape overnight.  To get that way takes years of work and effort. :)  Similarly, it takes a lot of time and effort to get back into shape.  If you're dealing with a legacy system or product that is becoming more and more difficult to change, you aren't simply going to spend a few days writing some tests and making it all better.  Similarly, if you have had a serial approach to product delivery for decades, you aren't going to change in the blink of an eye.

Whether you're trying to get back into shape or change how you deliver products, it's helpful to have short, medium and long-term goals to provide focus for your efforts.

For example, I gave myself a short-term goal which had nothing to do with my weight, but was to simply run 5 kilometres (without stopping to walk) and live to tell the tale.  To do that, I had to dial back the running pace that my brain thought it was capable of sustaining for that distance.  I achieved the goal on my 10th workout.  Cool!  I could do it, and now I just wanted to improve my conditioning level so that I could do it faster.

I set a medium-term goal of reaching 10 lbs. in weight loss and running 5K in 30 minutes.  I achieved the running goal about a week before the weight goal, but I was now into a mindset where I was positive that, as long as they were realistic, I could set the goals and achieve them!

So, I set a long-term goal of 30 lbs. weight loss and running 10K in 50 minutes (a pace of 5 minutes per kilometre).  To test my ability to run that pace, I did a short 15-minute run at that pace last week.  I was gasping for air by the end - I obviously still have a lot of work to do!

What this means, though, is that you must also accept that there is going to be some pain involved with this fundamental level of change.  You are going to be gasping for air.  There are going to be sore muscles.  There are going to be blisters.  It will hurt.  Whether you are wanting to exercise and lose weight or change from a silo-ed, plan-driven development culture, you have to accept that you didn't get to the point of wanting and needing to change overnight, and thus you won't be able to make that change overnight.

If you have a pile of legacy code with minimal or ineffective automated testing, you must understand that part of the pain is going to be reduced capacity to deliver new features while you get the level of automation in place to allow you to move faster.  You can't simply go from minimal or no exercise for years to running a 50-minute 10K - you have to build up the capacity to do so.  If you simply try to make the change in the blink of an eye, you will injure yourself.  Similarly, trying to move from low levels of test automation to 100% coverage will likely not work, and end up with detrimental effects as people experience the pain of applying tests to existing legacy code.

So, everyone involved in the transition must understand that this isn't a quick fix.  It will take time, there will be some pain, and you will need to stick with it long enough for the new mindset and practices to become habits.  The larger the transition, the longer it will take.  This is on the scale of many months to multiple years!

No Finish Line
While a 10K race has a finite distance, my efforts to improve my weight and fitness levels do not.  Running for a fixed distance or fixed time is a means to an end, not an end in itself.  If I simply said, "When I hit 30 lbs., the diet is over!", then I've missed the point.  The real goal is to establish a lifestyle that is going to improve and sustain my health for decades, not some arbitrary weight.  In other words, for what I have started, there is no end!  If I assume there is, then I'll regress.

This is the same approach you need to take with an Agile transition.  The goal isn't to get all your teams using Scrum.  The goal isn't to reduce defects to 1 per KLOC.  The goal isn't to ship on time, on budget.  The goal is to create a sustainable environment in which your people can do their best work, be innovative, and enjoy themselves!

Over time, the benefits you see will diminish in size.  I'm certainly not going to lose weight at my current pace indefinitely... I would reach a weight of 0 sometime in 2013.  Similarly, teams that transition to Agile may see great improvements in productivity and reduced defect rates initially.  Eventually, though, the rate of those improvements will decline, even on the best teams.  That's OK, though, and expected.  As long as you understand that you should continue to learn, continue to look for ways to improve, then you're doing just fine.  If you once had 50 field-reported defects every 6 months and now you have 2, you should continue to look for ways to get to 1 or even 0, but don't do so to distraction!  Don't knock yourself out, or berate the teams.  The chances are that your customers are now delighted at the new level of quality!

Conclusion
Dan Pink boiled motivation down to three key points: Autonomy, Mastery and Purpose.  My current weight loss and exercise program is working because I'm not just another student in someone else's class (Autonomy), I have the equipment, environment and the capability to know how to change my diet and how to do the exercise (Mastery), and... what was my Purpose?  My 11-year-old daughter had joined the cross-country running team at her school and wanted me to practice with her.  I couldn't say no, but I was in such poor shape I had to do something to be able to run with her.

If you don't know why you are embarking on an Agile transition - you don't know that Purpose - then don't  start until you do.  Don't start until each and every person involved knows the Purpose.  Don't start until you can look yourself in the mirror and say, "This is absolutely worth the effort it's going to take."