26 March 2015

Changing the Tires

You're driving down the highway trying to reach a distant destination. You've had delays such as traffic along the way, and you know that you're going to have to "push it" in order to have any hope at all of arriving on time. You start to feel something strange in the steering wheel. The car seems to be pulling to one side and the steering is rather "mushy". It dawns on you that you're losing air in one of the tires.

Ugh. You've already been delayed, and you really need to get to the destination on time.  Perhaps there's a store that will be closed. Perhaps an important meeting that requires you to be there. Regardless, a tire that's becoming flat will certainly delay you more than you can afford.

So, what do you do?

Naturally, you'd continue to ignore the leaking tire, knowing that you simply don't have the time to stop and fix it. You would try to think of every possible trick that would allow you to continue on your way without stopping. And, of course, you wouldn't call ahead to warn anyone that you could possibly be late. That would be admitting defeat, and this battle against the tire has become personal!

Eventually, though, the tire becomes flat. Still not willing to admit that you need to stop, you forge ahead with the now deafening sound of the flat tire starting to shred itself on the rim.  Once the tire has been consumed, the rim itself disintegrates and a whole corner of your vehicle drops to the ground in a shower of sparks as your journey comes literally to a grinding halt. You definitely have to deal with the tire now, and while you're at it you'll need a new rim and probably a substantial list of parts from your braking and suspension systems!

In technical terms, you're hosed.

This story sounds absurd, and outside of Hollywood movies with the requisite chase scene, it truly is absurd. Once you feel the tire going flat, you know that you have to pull over and deal with it. You may be lucky and have run-flat tires that allow you to make it to a service station, or maybe you're these guys who can change a tire while still driving:
Otherwise, you have a spare tire that will at least allow you to get a certain distance until you can get your faulty tire repaired.

There's one industry, though, where the absurdity of this story seems completely lost on all the participants. As you may expect, given the nature of this blog, that industry is software. How many times are rapidly growing defect lists ignored or deferred with a "we'll deal with those after we ship"? How many times have teams swept technical debt under the rug because they don't have time to clean up right now? How many times have people worked ridiculous, unsustainable hours until they crashed?

In how many of all those cases did the people involved not know that a tire was going flat? Probably close to none. We know when work isn't going well. We know when we aren't going to make deadlines. Collectively, though, we're in such a damned hurry to "get there" that we ignore all these things hoping that the tire will hold together long enough so that we can fix it after we arrive. I have some pretty clear feelings on the use of hope in the software industry.

I've heard plenty of explanations for this approach.

We Have to Ship by (Some Date) or We'll Miss Our Target Market!

If you have very strict date constraints, such as the Christmas shopping season or a specific trade show then, yes, you need to plan accordingly. There are times where it can make sense to incur some technical debt in order to take advantage of a market opportunity. In this scenario, your tire has a slow leak, but you can manage it by topping up the air periodically. You're able to reach your destination without any problems other than stopping for air.

At some point, though, you have to either repair or replace the tire. In software terms, you must pay off that debt after shipping. If you don't, you'll see your ability to take advantage of those market opportunities erode very quickly.

You can also avoid incurring technical debt in the first place by ruthlessly avoiding bloat on the features you're shipping. I spoke of this in Pragmatic Agile - Done is Better Than Perfect, where you whittle, slice and prune features to the bare minimum that provides value to the people consuming the software. When you can do this, you give yourself an enormous amount of flexibility in terms of deciding what to ship by a certain date.

That gives you all the time necessary to build a clean solution, avoiding most technical debt.

But We Can't Stop Shipping!

That's an interesting statement. It's a lament that I've heard a number of times from management, teams and individuals when I suggest that they need to pull over and change the flat tire.

One manager stated outright that if they simply stopped working on features to address technical debt, they would be out of business in 8 months. My response was that if they didn't change that flat tire, they would be out of business in a few years at best. That group had early success with Scrum but weren't dealing with technical debt, and their ability to complete features was eroding over time. About 3 years after I made my statement to him, the product was cancelled without ever shipping.

The rubber on the tire was gone, the rim disintegrated and the car came to a screeching halt in a shower of sparks.

That said, you can still ship while addressing technical debt. The hint in the previous paragraph is that you're already feeling the pain of technical debt due to the erosion of your ability to complete work. For example, every new feature and every change becomes more and more difficult because a change to code in one place is breaking features all over. Your testers can't perform full manual regression testing due to the increasing size of the system. Simply getting anything done is painful and the business people are becoming restless.

The best way to approach this while still delivering value is to simply admit that you can't ship as much as quickly as you once did. Admitting you have a problem is always the first step. Then, clean up the system as you need to touch it, i.e. when adding something new, changing existing functionality, or fixing defects.

When doing this, apply the Boy Scout Rule ruthlessly - always leave the code or system cleaner than when you came in. This can take the form of repeated small refactorings. It could be adding microtests where there were none. It could be adding higher level tests or adding more scenarios to existing tests as you learn more.

This is the equivalent of having run-flat tires. You can still drive, but your maximum speed is limited to 80 km/h / 50 mph.


While there are some contexts in which it makes sense to push features sooner without addressing technical debt, those are few and far between. Just like with the flat tire analogy, you will reach your destination faster in the long run (and often in the short term) by dealing with technical debt within our systems sooner.

I've work on or around several systems that have existed for over 10 years, including one that celebrates its 40th birthday this year. While those system do have some technical debt, their longevity suggests that they're actually relatively clean, allowing them to receive new functional to adapt to new business needs. They also represent excellent returns on the money invested in them.

Not every system needs to consider existing for 40 years, but even systems whose lifetime may be measured in weeks or months will benefit from minimizing technical debt from the start. The practices, such as Test-Driven Development, Simple Design, Refactoring and Pair Programming that help minimize debt will also allow you to deliver faster from the start.

If you use that approach, you may never need to change the tires again.


Will said...

The "Abandon hope" link is broken.

This is a working link: http://blog.daverooney.ca/2008/02/abandon-hope.html

TiTi said...

Yep, really well pictured.
You nailed it regarding the way we work sometimes. And the realization of it.

Realization is step one.
Then sometimes it's hard to question the process. We might have become so used to it...
Or convince collegues of the mess we're in.
Or provide a good path to improve the situation (hard to measure).

But software developers are the primary primary sufferers of technical debt.

It's interesting that you've never used the word "quality" in your post, but rather classified the issue as "technical debt".

I think it's also about serenity. Some things needs time to get well prepared and be resilient to (hostile) environment.
There is a lack of preparation, a lack of responsability.

In case of being the one with some responsabilities, one idea is to explain managment about this.
Make sure to detail the risks and be clear that if some things need to be shipped ASAP, maybe some other won't be possible in the meantime. Ask managment with choices: what won't be possible at the same time? Should priorities be redefined?
You could also make a deal that shipping will be effective for the near to come agenda, but that it will require an extra time later, dedicated to cleanup things after the rush.
(See: Do You Have Too Much to Do?)

Utimately, I think somebody has to say "No", be the guardian of the software, of the car.
Question is: who is it at your place? Yourself, the car owner ? The police ? or ... the tow truck driver ?

I'd be interested to know how most industries (outside software development) deal with that: tire change, things getting out of order, product incidents, client happiness, worker happiness, process to keep things working, ...
And more importantly, what differs between the good guys and the others?

myrtokamen said...

What a great metaphor! The truth is that it is very rare to find organizations not working like that. However, as soon as you point out issues, options and redefine priorities if needed, the better the result.