1 June 2011

Microtests and TDD Aren't Sufficient

First a couple of clarifying definitions:
  • "Microtests" is a term coined by Mike "GeePaw" Hill - it's a synonym for unit test, but avoids the baggage of that term which can mean anything from manual running of code to automated tests that take weeks to write.
  • "Tests" in this context are synonymous to the term checks used by some people.
I remember back a decade ago when I was a neophyte in this radical new approach to developing software called Extreme Programming.  At this time in 2001, I had spent about 6 months applying what I had learned from the books Extreme Programming Installed, XP Explained 1st Edition, Refactoring, etc., and hanging around Ward Cunningham's C2 Wiki, and was becoming proficient at the radical new approach to design and development called Test Driven Development.  I have to admit that it took about a good 4 months of dedicated practice to first get over the complete change in paradigm of writing tests first, to being comfortable writing tests, to letting the tests lead the design.  Today, I can't imagine working any other way, although it has been a struggle in a couple of legacy code situations.

I also remember that there was a buzz in the small XP community of the time that we wouldn't need testers anymore because our microtests would solve world hunger, cure cancer and probably fix the southern hemisphere ozone hole as a nice side effect.

Prior to joining the project where I learned about XP, I had the privilege of working with some really, really good testers who worked directly with me - we paired without knowing that it was called pairing. :)  Those people helped me to think much more critically about what I was coding, and that translated directly to much better quality from my code.  Adding the automation aspect of frameworks such as JUnit only served to improve that further.  However, it wasn't enough!

It didn't matter how well I (or anyone I worked with) had tested an individual class or even small collection of classes, the higher level interactions in the system would still have some issues that the microtests may not have covered.  That wasn't a shortcoming of microtests, but rather it was never the intention of that level of tests to cover everything!  Microtests and TDD show that the code is doing what the programmers intended it to do, which of course is very good, but those tests don't and aren't intended to show that the system is doing what the Customer or Product Owner wants it to do.

Fortunately, there were voices in that "we don't need testers anymore" wilderness such as Lisa Crispin, who co-authored Testing Extreme Programming, and ensured that the "programmers only" attitude was as incorrect as the "no documentation" myth.

Microtests and TDD are excellent, excellent practices and enable teams to drive towards zero defects in a very cost effective manner.  You still, need, however, other levels of tests and testing to achieve that goal and to sustain a team's ability to be agile over any extended period of time.

No comments: