Topic: Running Tests - Defining Test Sets Running tests

Message

   Running Tests - 
Defining Test Sets 

Running tests is the core of the testing process. As your
application changes, you run the manual and automated tests in your
project in order to locate defects and assess quality. 
You start by creating test sets and choosing which tests to
include in each set. A test set is a group of tests in a TestDirector
project database designed to achieve specific testing goals.
TestDirector enables you to control the execution of tests in a test
set. You can set conditions, and schedule the date and time for
executing your tests. 
Once you have defined test sets, you can begin executing your
tests. When you run a test manually, you execute the test steps you
defined in test planning. You pass or fail each step, depending on
whether the application’s actual results match the expected output.
When you run a test automatically, TestDirector opens the selected
testing tool, runs the test, and exports the test results to
TestDirector. 
In this lesson, you will learn about: 
➤ Defining Test Sets 
➤ Adding Tests to a Test Set 
➤ Scheduling Test Runs 
➤ Running Tests Manually 
➤ Running Tests Automatically 
Defining Test Sets 
You can organize test runs by building test sets. You build
test sets by selecting automated and/or manual tests from the test plan
tree. Note that you can include the same tests in different test sets.
When you run the tests, results are stored separately for the different
test instances. 
To decide which test sets to create, think about the testing
goals you defined at the beginning of the testing process. Following
are examples of general categories of test sets you could create: 

Test Set 

Description 

Sanity 

Tests the entire application at a basic level to check that it is functional and stable. 

Normal 

Tests the system in a more in-depth manner than
the sanity test. A Normal test set can contain both positive and
negative checks. Positive checks test that the application responds to
input as expected. Negative tests attempt to crash an application in
order to demonstrate that the application is not functioning properly. 

Advanced 

Checks the entire application, including its most advanced features. 

Regression 

Verifies that a change to one part of the application did not prevent the rest of the application from functioning. 

Function 

Tests a specific feature or a group of features in the application. 

To define a test set: 
1 Open the TestDirector_Demo project. 
If the TestDirector_Demo project is not already
open, log in to the project. For more information, see "Starting
TestDirector," on page 5. 

2 Display the Test Lab module. 
Click the Test Lab tab. 
3 Add a test set to the Test Sets list. 
Click the New Test Set button or choose Test Sets > New Test Set. The New Test Set dialog box opens. 

In the Test Set Name box, type a name for the test set. For example, type: Mercury Tours Demo.
In the Description box, type a description of the test set. For example, type: This test set includes tests that verify the functionality of the Mercury Tours site.
Click OK. The Mercury Tours Demo test set is added to the Test Sets list in the left window pane. 
Define the test set details. Click the Test Set Properties tab and select the Details link. 

By default, the Status indicates that the test set is Open. 
In the Open Date box, select a date from the calendar. By default, TestDirector displays the current date.
In the Close Date box, select the planned closing date for the test set.
For the purpose of this exercise, skip the following fields: Level, Version, and Project. 
5 Display the test set attachments. 
Click the Attachments link.
You can add an attachment to the test set. An attachment can be a file,
URL, snapshot of your application, an image from the Clipboard, or
system information. For the purpose of this exercise, skip this option.

6 Set rules for the automated tests in the test set in the event of a test failure. 
Click the On Failure link. 

Select
the first check box to set the test set on failure rule. Make sure that
the number of times an automated test should be rerun is set to 1.
The
second check box enables you to include a clean up test from the test
plan tree. For the purpose of this exercise, skip this option. 
The Settings per test link
enables you to change the above on failure rules for any test in the
test set. For the purpose of this exercise, skip this option.
In On final test failure,
you can instruct TestDirector, after the final test failure, to do
nothing, stop the test set, or run the test set again a specified
number of times. Make sure that the Do nothing option is selected. 
7 Instruct TestDirector to send an e-mail to specified users if certain events occur. 
Click the Notifications link. 

Select the first check box to send a notification if any test in the test set fails. 
To specify who should receive the e-mail, type your actual e-mail address in the To box.
In the Message box, type a message for the e-mail. For example, type: This test failed. Please review the test results and submit a defect. 


One Stop Testing Forum : Testing Tools @ OneStopTesting : TestDirector @ OneStopTesting