|
Know your objective
There are many good reasons for doing test automation.
Test automation can save time, make testing easier, and improve the
testing coverage. It can also help keep testers motivated; I can list a
page of other benefits that can be derived from doing functional test
automation. However, it is not likely that your organisation will need
to do all these things at the same time. Different groups typically
have different hopes and ideas of what they want test automation to
offer them. These need to be stated, or else disappointment is likely.
A clear objective to work towards during test automation is highly
important. It provides us with a compass heading towards which planning
and implementation will be directed. Eventually, the results of the
test automation implementation will be compared against the original
objective to establish whether we did reach our goal or not.
Mistakes made...
"Test automation is the answer to our testing problems"
Test automation assists in the testing process, and can therefore
provide value to a test team. Certain benefits can be derived, but we
need to know and understand exactly what they are without having false
expectations or hopes. Automation is not a replacement for an
inadequate test process or an inefficient test team. Address the
testing problems prior to embarking on a costly test automation
implementation. Automation's objective is definitely not to solve all
testing problems in existence.
"Since we have limited time to test, let's use test automation."
Test automation takes 3 to 128 times longer to implement than manual
testing, and therefore it should be properly planned to achieve the
expected benefits. If time is of the essence, consider adding more
resources to the test process (this also has limited benefits, and
might not always be a solution) rather than embarking on the lengthy
process of implementing test automation. Functional test automation is
definitely not a quick solution to a test project in distress. Multiple
other solutions are better suited to address this need.
Test Automation requires a manual test process
"Success in test automation requires careful planning and design
work, and it's not a universal solution. ... automated testing should
not be considered a replacement for hand testing, but rather as an
enhancement." (James Bach [pg. vii] in his foreword: Software Testing
with Visual Test 4.0, Thomas Arnold II, IDG Books Worldwide Inc. Foster
City, CA.: 1996. ISBN 0-7645-8000-0.)
The words of James Bach and many others are quite clear on this
subject. Test automation will not give you a correct testing process or
methodology. It will not enforce a methodology or process. Test
automation merely supports the test process; it will not replace the
manual test process.
Mistakes made ...
"We need to implement testing in our projects. Let's buy a tool and automate the testing."
You can script to your heart's content, if you do not have a proper
manual test process in place the benefits of test automation will never
be seen. It will only assist you in doing the incorrect process a lot
faster. Test automation merely assists and enhances the test process
itself. Ensure that a proper test methodology is implemented before
attempting to add test automation.
"We can just add the test automation to our current test process and start automating our current test scripts."
Test Automation requires test cases to support the automation process.
If my test cases do not incorporate the all-important logon to the SUT
(system under test), the automation script will need a lot of manual
intervention, and would therefore be more manual than automated. The
current test process might have to be changed, or adapted, to be
implemented for the tool suite selected. Review and adapt the test
process to ensure maximum benefit is realised from the tool suite to be
used in your test process.
The earlier the involvement in the Software Development Life Cycle (SDLC) the greater the benefits
We all know how testing has its own phases aligned with the phases
of the SDLC. The earlier testing gets involved in the life cycle of the
project the better. Many articles have been written on this subject,
and it holds true. Since test automation supports the testing process,
the test team can get involved very early in the life cycle of a
project. If the test team uses a test automation tool suite, they will
most likely have a requirements management tool or test management tool
included in the suite. These tools can assist the test team in various
ways from an early stage in the development life cycle, even before a
line of code is developed.
The more mature techniques and methods of test automation are
closely tied to the testing methodology being used. An early start to
the test life cycle, at the start of the development project life
cycle, is vital to achieving the best results, and ensuring maximum
return on investment.
Mistakes made…
"The test automators cannot get involved in the project before the developers release the first build."
A software development project does not start with code development.
Since functional test automation is also a development project, as we
will discuss in Best Practice 9, it follows the same phases as a
software development project and does not start at the coding phase.
Proper planning and design are essential to the success of the
functional test automation implementation, and need to be conducted
hand-in-hand with the test life cycle.
"Testing is only a single phase in the SDLC and therefore testers
and test automation only need to be part of the project for the testing
phase."
This 'prehistoric' misconception in the IT industry has been proven one of the major reasons for bad software quality.
Test Automation is a fulltime effort
When people are allowed to work on test automation on their own
time, or as a back burner project whenever the test schedule allows,
test automation becomes a sideline effort. Test Automation does not get
the time and focus it needs.
Getting into automated testing requires preparing not only
yourself, but also your company and environment. This requires focus
and dedicated resources. {mosgoogle}
Mistakes made…
"We have a test automation tool. Can the test team see if they can use it?"
It is clear that the expectation and an understanding of test
automation is lacking. I can guarantee that the tool implementation
will not be successful, and would result in almost no benefit to the
test team. The test team is usually confronted with manual test work
for development projects that does not meet the code delivery date for
testing. The test automation implementation would probably be of the
capture-record-and-playback form, and therefore the benefit would be
low.
Test Automation will require training for the staff involved with
it. Time is required to design and implement the architecture.
Referring to the development nature of functional test automation
implementation, the team will have to learn a new skill set. Test
automation implementation requires dedication and focus to ensure
success. In this instance, it is better to not implement test
automation than to allocate ad hoc time for experimentation.
Select the correct product or suite of products
Buying the wrong tool is listed as one of the major challenges in
test automation, because - no matter what kind of process, methodology,
or organisation you have - if the tool is not a good technical and
business fit, it will not be used.
We know that a good process and organisation are also essential for
test automation. However, if the tool will not function at a basic
level, the people who should use the tool, and thereby gain the benefit
from the tool, will not use it.
Unfortunately, too few people do adequate research before buying a
test tool. Adequate research includes defining a set of tool
requirements based on what the intended users of the tool need to
accomplish, developing a set of evaluation criteria by which candidate
tools will be judged, and taking the experience of other people who
have used the tools into consideration.
Select a tool that will allow you to implement automated testing in
a way that conforms to your long-term testing strategy and test
methodology.
Mistakes made…
"A vendor demonstrated a tool to us and it seems as if the tool will add value. I think we should buy it."
Take time to define the tool requirements in terms of technology,
process, applications, people skills, and organisation. Then look at
multiple tool vendors and select the right fit for your circumstances.
A good idea would be to do a proof of concept project with the tool
that best fits your criteria.
"We purchased the whole suite of tools from the vendor, but are only using the functional test automation tool."
Test Automation tools are expensive, and to justify the Return On
Investment (ROI) it is normally not effective and efficient to use only
the functional test automation tool in the product suite. Other
benefits can be derived from using the full suite of products; for
example, proper defect management, metric reporting on test status,
etc. Purchase what you need, or use what you have fully.
"The yearly license fees are so expensive we cannot afford to use the tool suite any more."
Too often test automation tools end up as 'shelf-ware' because of the
cost of the licenses compared to the benefit derived. Refer to mistake
two above for one of the reasons why the ROI is not being realised.
Before signing the purchase order, be aware of the impact of annual
licensing fees.
Get executive management commitment
Test automation can yield very substantial results, but requires a
substantial commitment to be successful. Do not start without
commitments in all areas, the most important being executive
management.
'Thomas R. Arnold II, VP at ST Labs, Inc., sums it up fairly well:
"I would like to say up front ... that no test automation tool is a
silver bullet. Automation takes time, effort, and commitment from all
involved, including an understanding from management about the
realities of what automation can and cannot do." ' 1
Management support is needed in designing and deploying test processes
that will support the effective use of test tools, reinforce the role
and use of automated test tools in the organisation, and allow time for
tools to be integrated in the testing process.
Without management support, the entire test automation effort is at
risk. If management does not - clearly and consistently - show their
support for test automation, people will be less inclined to show
interest in using the tools. This is a major concern, especially
considering that overcoming the learning curve of some tools requires
dedication.
Perhaps the greatest challenge seen in management support is
balancing the high expectations of tool benefits against the time,
effort, and discipline it takes to implement the tool. Management may
become impatient about the lack of tool progress and shift their
support to other initiatives.
Mistakes made…
"We thought these tools were going to solve your testing problems."
When making the case to management for acquiring test tools, present
the challenges as well as the benefits. Make sure that management
understand their influence on how others will accept automated test
tools.
"We purchased a tool for you 3 months ago, and yet we cannot see any benefits in the project."
Communicate to management from the start that it takes time and
planning to build a firm foundation of people, processes, and the right
tools. Keep management informed of tool progress, and any issues that
arise
-------------
http://www.quick2sms.com - Send Unlimited FREE SMS to Any Mobile Anywhere in INDIA,
Click Here
|