When Should a Test Be Automated?
Summary
I want to automate as many tests as I can. I’m not comfortable
running a test only once.What if a programmer then changes the code and
introduces a bug? What if I don’t catch that bug because I didn’t
rerun the test after the change? Wouldn’t I feel horrible?
Well, yes, but I’m not paid to feel comfortable rather than
horrible. I’m paid to be costeffective. It took me a long time, but I
finally realized that I was over-automating, that only some of the
tests I created should be automated. Some of the tests I was automating
not only did not find bugs when they were rerun, they had no
significant prospect of doing so. Automating them was not a rational
decision.
The question, then, is how to make a rational decision. When I
take a job as a contract tester, I typically design a series of tests
for some product feature. For each of them, I need to decide whether
that particular test should be automated. This paper describes how I
think about the tradeoffs.
|