Topic: Shifting from Manual Testing to Test Automation

During the past five years there has been a noticeable increase in the use of test automation tools particularly in Graphic User Interface (GUI) testing (these tools are sometimes referred to as Record and Playback tools). Four stocks from the American NASDAQ stock exchange indicate the growth of these tools since 1996 (see the graph on page four).

This phenomenal growth is largely attributable to increases in technology and the pressure created on the testing effort by the improved productivity of development tools and languages. It is difficult to convince a project manager that it will take one month to (manually) test a new web site that took developers only two weeks to design and update.

To speed up testing, more and more test teams have turned to test automation tools. Unfortunately test automation tools are often perceived as "silver bullets" to solve all testing issues. The expectations of less time to test, fewer testing resources, and full test automation don't always materialise. Unless a correct implementation approach is followed at the appropriate time, the (expensive) test automation tools can become shelfware and result in disillusioned managements and teams.

In moving to test automation it is imperative that management's perceptions are accurate from the outset, which will then lead to realistic expectations.

• It will initially take longer to develop effective automation scripts than to do manual testing. Evaluating test automation tools in order to make the correct selection can be a significant exercise.

 

• Not all tests should be automated.

 

• Introducing test automation will not solve the absence of an effective testing methodology.

 

• Automated tests do not generally find more defects than manual tests in first-pass testing.

 

Each of these points will now be elaborated:

Automation initially takes longer than manual testing and thereafter can greatly reduce manual testing times. Firstly, time should be allowed to select the correct test automation tool (see article on "Choosing the Right Test Automation Tool" on page seven). This is a significant exercise, as the selected test automation tool will impact on the overall productivity of the automation effort. Secondly, it should be known that the most effective use of test automation tools requires that tool scripts be specified, designed, coded, and then tested and maintained following their own software development life cycle. It follows therefore that the first tests can be particularly slow, as automation test design features which will save overall automation time later, are created.

Effective scripts have been measured to take between two and ten times longer than the manual creation of equivalent tests. This time will impact on budgeting and scheduling and will only generally be regained in the second, third or subsequent required re-runs of any test. The return on investment can take from 3 to 36 months on a fully implemented GUI test automation system.