When to Focus on Manual Testing
High per-test or maintenance costs are one indicator that a test should be done manually. Another is the need for human judgment to assess the correctness of the result or extensive, ongoing human intervention to keep the test running. For these reasons, the following tests are a good fit for manual testing:
· Installation, setup, operations, and maintenance. In many cases, these tests involves loading CD-ROMs and tapes, changing hardware, and other ongoing hand-holding by the tester.
· Configuration and compatibility. Like operations and maintenance testing, these tests require reconfiguring systems and networks, installing software and hardware, and so forth, all requiring human intervention.
· Error handling and recovery. Again, the need to force errors—by powering off a server, for example—means that people must stay engaged during test execution.
· Localization. Only a human tester with appropriate skills can decide whether a translation makes no sense, is culturally offensive, or is otherwise inappropriate. (Currency, date, and time testing can be automated, but the need to rerun these tests for regression is limited.)
· Usability. As with localization, human judgment is needed to check for problems with the facility, simplicity, and elegance of the user interface and workflows.
· Documentation and help. Like usability and localization, checking documentation requires human judgment.
There’s no return on investment in trying to automate these kinds of tests, typically. One of my clients once spent thousands of dollars and staff hours trying to automate configuration and compatibility tests, only to give up months into the
effort.
|