14 December 2009

On Estimation

My family moved over the past weekend to the lovely little town of Richmond, ON (actually part of Ottawa, but it's still a community of its own). We decided that for this move we would hire a company to both pack and move for us.

A couple of months ago I had a few moving companies come to our old home and give us estimates of the cost. Each company sent out a sales representative who walked around the house, attempting to determine how many people and how much time it would take to pack and perform the move. One company came out with a significantly lower estimate, and they had moved other people that I knew. So, we decided to go with them.

On 'packing day' the two packers showed up just when they said they would. Within 15 minutes, though, they said that the sales rep had underestimated how much was there. In the end, what was estimated to be an 8 hour job took 12.5 hours.

The next morning the movers arrived, also right on time. It only took them about 5 minutes to start saying that there was more than estimated. The lead mover then spent the rest of the day griping about how the move had been underestimated. I spent the rest of the day griping about how the sales rep said that the move wouldn't cost more than the estimate.

By the time the movers had the truck loaded, they had determined that several substantial objects wouldn't fit without a second trip (and the associated extra costs). So, I had to cut scope.

OOOPS... I think I just gave away the point of this blog entry!!

Here I was, the Customer, being told that I had to cut scope or face additional charges. I didn't get to choose what was going to be left behind for me - the movers told me what wouldn't fit.

It sucked. A lot.

After some reflection, I wondered if sending at least one other person to handle the estimation would be better. The sales rep assured me that he had 25+ years in the moving business, and I have no reason not to believe him. But this is the same failing as having a business or systems analyst go off and gather requirements and determine the size of a project:
No matter how much experience you have, estimating in the absence of the people who will actually do the work will lead to errors that affect the ability to deliver all that is required, a quality product, or both.
When estimating, include as much of the core Project Community as possible. If that means juggling some schedules to fit everyone in, then do it. Having that input is critical since it provides sanity checks on the amount and type of work to be performed. Involving more people will increase the likelihood that someone may have an idea for a simpler implementation of a particular feature. Involving more people increases the level of understanding about the problem to be solved for everyone. Involving everyone increases the cohesiveness of the community that will work together.

I knew intuitively that, from the Customer's or Product Owner's perspective, the traditional estimation process where people talked to you then went away and dreamed up a schedule just didn't work very well. Now, having been that Customer, I know for certain that it's the case.

5 comments:

Anonymous said...

I'm interested in hearing more about your experience as Customer/Product Owner. Were there are parts of the metaphor that fit? Sounds like the project ran 50% over estimate, etc. I wonder if it would help moving companies to have several smaller trucks instead of one big one? Iterative moving?
@BillyGarnet

George Dinwiddie said...

Dave, I notice you said "one company came out with a significantly lower estimate." Did this estimate give an indication as to whether it was less effort or cheaper for the same estimate?

I wonder if there isn't a lesson in there for us. It's very tempting to trust the most optimistic estimate. I've heard that, even when an estimate is expressed as a range, it's most likely that the optimistic end of that range will be what's remembered (and often taken as a commitment).

When we get an outlier in a group of estimates, we should probably investigate further. I've no idea, however, how to get other people to take this advice. It's hard enough to take my own advice.

Dave Rooney said...

Heh... yeah, the parallels are almost scary. The sales rep estimated 9,000 lbs. of stuff in the house. The movers said that, based on how loaded the truck was, there was about 15,000 to 16,000 lbs.

Fortunately, the 'lead' mover brought an extra person , which at least kept the move on schedule. I guess Brooks' Law doesn't apply to moving. :)

Otherwise, I've moved myself before and thus know a bit about the amount of work involved (semi-educated Customer). Beyond that, my role as the Customer seemed to follow very closely with that of the Customer in a traditional software delivery project.

Dave Rooney said...

@George,

In the end, the cost of the move was about $200 more than the estimate ($3400 vs. $3200). I was OK with up to a $500 difference.

I had a look at the other estimate I received by e-mail, and while it was closer to the actual 15-16K lbs., it was 13K lbs.

I agree with respect to outliers in estimates. When I'm facilitating Planning Poker, I often explore outliers on both the high and low side. I approach this from the perspective that the person making those estimates may know something that the others don't!

In the end, the lower estimate for the move was a firm one (+- $500), which is why I went with it. Considering that it was $2000 less than the others, I was willing to live with the movers' griping. :)

I suspect as well that they discounted the price in order to get the business during a slow point in the season. This was a relatively big moving company, and they only had two moves scheduled in all of December.

The experience, though, did open my eyes a bit to what a Customer on a traditional project must face. They're near the end of the schedule, and suddenly the development people are telling them (rather than working together with them) that not everything is going to be finished.

Not a real blog said...

It's interesting that George mentions investigating further when you have estimates out of the average. This is exactly what you do when you do poker planning (at least how I've seen it done): if you have one or more people giving higher or lower estimates than the rest, you ask what makes them think the task or story will take as long or as short as they're thinking. Very often I've seen estimates adjusted that way, when it turned out they had thought of details the rest of the people estimating hadn't thought about. It's also the main reason we had as a rule in that team never to estimate a story by just one person.