Topic: Write Selected Test Cases

Now it is time for the main event: actually writing the test case steps and specifying test data. This is a task that you can expect to take ten to forty-five minutes for each test case. That might work out to approximately ten test cases in a typical work day. So, you must be selective to get the most value in return for your limited available time.

Focus on the test cases that seem most in need of additional detail. For example, select system test cases that cover:

• High priority use cases or features
• Software components that are currently available for testing (rather than specifying tests on components that cannot actually be tested yet)
• Features that must work properly before other features can be exercised (e.g., if login does not work, you cannot test anything that requires a logged in user)
• Features that are needed for product demos or screenshots
• Requirements that need to be made more clear

Each test case should be simple enough to clearly succeed or fail, with little or no gray area in between. Ideally, the steps of a test case are a simple sequence: set up the test situation, exercise the system with specific test inputs, verify the correctness of the system outputs. You may use programming constructs such as if-statements or loops, if needed.

Systems that are highly testable tend to have a large number of simple test cases that follow the set-up-exercise-verify pattern. For those test cases, a one-column format can clearly express the needed steps. However, not all test cases are so simple. Sometimes it is impractical to test one requirement at a time. Instead, some system test cases may be longer scenarios that exercise several requirements and verify correctness at each step. For those test cases, a two-column format can prove useful.

In the one-column format, each step is a brief verb phrase that describes the action that the tester should take. For example, "enter username," "enter password," "click 'Login'," "see Welcome page," and "verify that greeting has correct username" are all steps. Verification of expected outputs are written using the verbs "see" and "verify." If multiple inputs are needed, or multiple outputs must be verified, one-column test cases will simply have more steps.

In the two-column format, each test case step has two parts: a test input, and an expected output:

Test Input

The Test Input is a verb phrase describing what the tester should do in that step.

Expected Output

The Expected Output is a noun phrase describing all the output that the tester should observe at that step.

You may notice that the two formats for test cases mirror the two formats for use cases. The difference is that use cases are a form of requirements, whereas test cases deal with more details of the implemented system. Use cases focus mainly on the user's tasks and how the system supports those tasks, while specifying as few implementation details as possible. A major advantage of use cases is that they are simple enough to be read by actual users who can help validate requirements. In contrast, test cases should more technical documents with enough implementation detail to allow any member of the development team to carry out a test exactly the same way.

If you have written use cases, they can be copied and pasted as a good starting point for test cases. When leveraging use cases in this way, make sure to add enough detail to make the test reliably repeatable.

If you only have one test input value for a given test case, then you could write that test data value directly into the step where it is used. However, many test cases will have a set of test data values which must all be used to adequately cover all possible inputs. We encourage you to define and use test input variables. Each variable is defined with a set of its selected values, and then it is used in test case steps just as you would use a variable in a programming language. When carrying out the tests, the tester should repeat each test case with each possible combination of test variable values, or as many as practical.

Carefully selecting test data is as important as defining the steps of the test case. The concepts of boundary conditions and equivalence partitions are key to good test data selection. Try these steps to select test data:

• Determine the set of all input values that can possibly be entered for a given input parameter. For example, the age of a person might be entered as any integer.
• Define the boundary between valid and invalid input values. For example, negative ages are nonsense. You might also check for clearly unreasonable inputs. For example, an age entered as 200 is much more likely to be a typo than a user who is actually two-hundred years old.
• Review the requirements and find boundaries in the valid range that should cause the system to behave in different ways. For example, the system might treat minors differently than adults, so the boundary would be age 18.
• Now you have a set of equivalence partitions: sets of values that the system should treat uniformly. For example, all minors are treated one way, and all adults are treated another way. Double check the requirements to make sure that you have not missed a partition division, e.g., not all adults are old enough to drink alcohol in the U.S.
• Choose one input value somewhere in the middle of each equivalence partition (e.g., -5, 12, and 44), one directly on each boundary (e.g., 0 and 18), and one on each side of each boundary (e.g., 1, 17, and 19). Test data vales that are expected to cause errors (e.g., -5) should be tested in separate robustness test cases.
• In functional correctness test cases, make sure that you have inputs that will force the system to generate each possible type of response to valid input. And, in robustness test cases, make sure to force the system to generate each relevant error message.

Recall that one of the advantages of writing test cases is that it forces you to clearly think through the requirements. Capture your insights by writing notes and questions as you go. If a test case step exposes an unclear requirement, make a note of it in the appropriate part of the system requirements specification.

Edited by moderator - 04May2007 at 11:58pm