Topic: Cost-Effective Automated Testing

Automated testing is expensive (contrary to what test tool vendors would have you believe). It does not replace the need for manual testing or enable you to "down-size" your testing department. Automated testing is an addition to your testing process. According to Cem Kaner, in his paper entitled "Improving the Maintainability of Automated Test Suites" (www.kaner.com), it can take between 3 to 10 times as long (or longer) to develop, verify, and document an automated test case than to create and execute a manual test case. This is especially true if you elect to use the "record/playback" feature (contained in most test tools) as your primary automated testing methodology. Record/Playback is the least cost-effective method of automating test cases.

Automated testing can be made to be cost-effective, however, if some common sense is applied to the process:

• Choose a test tool that best fits the testing requirements of your organization or company. An "Automated Testing Handbook" is available from the Software Testing Institute (www.ondaweb.com/sti) which covers all of the major considerations involved in choosing the right test tool for your purposes.

• Realize that it doesn’t make sense to automate some tests. Overly complex tests are often more trouble than they are worth to automate. Concentrate on automating the majority of your tests, which are probably fairly straightforward. Leave the overly complex tests for manual testing.

• Only automate tests that are going to be repeated. One-time tests are not worth automating.

• Avoid using "Record/Playback" as a method of automating testing. This method is fraught with problems, and is the most costly (time consuming) of all methods over the long term. The record/playback feature of the test tool is useful for determining how the tool is trying to process or interact with the application under test, and can give you some ideas about how to develop your test scripts, but beyond that, its usefulness ends quickly.

• Adopt a data-driven automated testing methodology. This allows you to develop automated test scripts that are more "generic", requiring only that the input and expected results be updated. There are 2 data-driven methodologies that are useful. I will discuss both of these in detail in this paper.