Topic: Getting started with automated testing..........

                    

Get Familiar with the Tools

    The minimum set of tools you'll need to do effective automated testing consists of Rational TestManager and Rational Robot. If this is your first time, start with these and become thoroughly familiar with them before you think about learning to use other tools. Some people advocate learning all the automated testing tools at once. That's fine if you have weeks to spend training before you need to start testing, but I've found that more often then not, testers are expected to jump in and are given deadlines as soon as they're given the software.

Go through the tutorials that Rational Software provides. While the tutorials aren't the definitive guides as far as training is concerned, they do get you familiar with the software as well as the Rational terminology. Both of these are important. As you'll soon find out, TestManager and Robot are very large and complex tools with feature upon feature. The tutorials will familiarize you with those features you'll be using most.

If you feel you still need more familiarity with the tools after you've completed the tutorials, attend a Rational University class or hire a training consultant to come in and spend some time with you. Having a basic understanding of the tools is essential. Make sure your whole team has had some form of training or some reasonable amount of time playing with the tools before you try to do any real work. (I find that programmers pick up the tools very quickly, while nonprogrammers struggle with some of the programming concepts and need more time.)

Have at Least One Programmer on Your Team

    Rule number one for efficient automated testing is to have at least one real programmer in your testing-automation group. You'll soon find out that automated testing is code development. While it's not Java or C++, you're still building a system of scripts, data files, and libraries. Robot's record-and-playback feature offers quick solutions for the most common tasks and controls, but for an advanced level of testing or for any custom controls, you'll need to be able to write your own code in SQA Basic. That means employing programmers, not manual testers who learn to code as they go.

Set Some Standards

    You'll also find it useful to develop standards for your automated-testing team. This is just as important in testing as it is in conventional software development. Your test system will develop more rapidly and will be easier to maintain if you establish and enforce naming standards, coding standards, environment standards, and procedures for error and defect tracking. Having these standards documented will also allow people new to the project team to come up to speed faster.

• Naming standards for scripts, test logs, directory structures, datapools, and verification points help to keep everyone on the same page. On two of my last three projects, we maintained more than 1,000 scripts and 5,000 datapools for each project. A good naming standard was the only thing that made that possible.

• Coding standards should also be developed and enforced. For most companies this is easy ? you can just steal the standards used by the development staff. If you don't have this advantage, go online and find some. They're out there, and you can find a reasonable set of standards in about 15 minutes. Once you've used them for a while, you can customize them to fit your needs and the needs of your company.

• Environment standards should ensure that the computers you use all have the same operating system, RAM, hard drive space, and installed software configurations. The only difference should be which Rational Suite you have installed on them. I've found that many of my hard-to-find and expensive-to-fix bugs have been due to the fact that a script was developed on a computer that had more resources than the one on which it was executed.

• Procedures for error and defect tracking should describe how to log errors in scripts, submit defects via ClearQuest, code workarounds into scripts, and remove it all after a bug is resolved. I didn't figure this out until a few projects ago. My team had a lot of problems communicating when it came to finding and reporting bugs. One of us would log a bug in ClearQuest and develop a workaround in the script without communicating to the rest of the team what we'd done. Inevitably, someone else would test the bug when it was fixed, mark it resolved, and never remove the workaround in the code. Sometimes things would work themselves out, and sometimes they wouldn't. Almost always this lack of communication caused confusion and rework, and cost the team time. After an audit, we found that 10% of our scripts tested absolutely nothing because they were spotted with workarounds that were never removed.

Document your team's standards, and be sure your team knows the standards and follows them.

Figure Out What You're Testing and Keep It Simple

You know what the application-under-test does, how it looks, and how to use it, but do you know what you want your automated tests to test for? The next step is to figure out and document exactly what you're testing. (Hint: With TestManager and Robot, you'll be testing either simple functionality or performance. You can test other things with other tools, but we're starting with the basics here.)

Figuring out what you're testing and keeping it simple is the most important step as far as political success or failure goes. A common mistake when automating for the first time is biting off more than you can chew and consequently missing deadlines or having to work unrealistic hours in order to meet them. Either of these situations will demotivate your testing team and make them look bad in the eyes of the rest of the development team.

Just like in conventional software development, success in testing depends on developing good requirements ? that is, arriving at reasonable goals for what you plan to test. Start small and keep things simple for your first-time automating. In future implementations you may want to go crazy and automate everything, but by starting small now you'll minimize possible rework later when your technical corridor widens as you add more Rational tools to your arsenal. Also, prepare the team and management for the fact that the team may not meet their deadline. Communicate that manual testers for testing what you're automating should be somewhere in the budget. After your team has had some successes, this can be relaxed.

When deciding what to automate for your first time, start with small milestones.

• If you're testing a GUI or Web application, start with testing simple functionality. This could include verifying that all the correct controls exist on the screen, the proper fields enable/disable when actions are taken, and such.
• If you're automating performance testing, start with just one virtual user and set your goal at a low number (no more than twenty). When you get one virtual user to work, double this and get two to work. Keep doubling this until you get to twenty. Each increment could present a new set of challenges.

Whatever you choose to test, make sure that it doesn't span more than one part of the application-under-test or more than one or two Web pages. Ideally, you should be able to use Robot's record-and-playback feature to perform this basic testing. The record-and-playback script will then become the baseline moving forward.

Robot and TestManager also have a lot of more advanced features such as the ability to add delays and timers, the ability to distribute testing on different machines, and the ability to create graphs for just about everything. Stay away from these as much as possible at first, because they'll just confuse you. Only after your team has had some successes should you explore these useful and often necessary features.

Edited by Amrita - 14Feb2007 at 6:13pm