Topic: Test Cases

The example I am going to use is an instance of the Brickles game that appeared on many early home computers. The game has been implemented in Java and C++ and we use the example in several of our training courses at Software Architects. I will use the Java version but it really makes little difference, for tests based on the specification, which language is used. There are a few attributes of a method that may be more precisely specified in one language than another and these provide valuable input into testing.

 

This first class under test represents the velocity of a moving object in the game. The velocity of a moving object has two attributes, the speed of the object and the direction in which the object is moving. Since the graphics are represented in a cartesian coordinate system, the velocity is decomposed into components along the x and y dimensions. The value of each component depends upon the magnitude of the speed and the direction. The complete source code for the class can be obtained from http://www.cs.clemson.edu/~johnmc so that you can consider the types of errors that are in the class and whether tests constructed from the class’s specification actually detect them.

 

Note that the pre-conditions for this Java class are carefully written to define when an object comes into existence. This is particularly important in languages such as Smalltalk and Java. In Java much use is made of class methods. For a class method to work, no object needs to have been explicitly created. So, for example, there would probably be a Math.cos() message, in decomposeSpeed, is a message to the Math class rather than a message to an object that must have been initialized by the programmer. The decomposeSpeed method itself has a pre-condition that requires that the programmer has created a Velocity object prior to sending this message.

 

Test Cases

There are several essential features that a test case must possess. First, the test case must define exactly how the pre-conditions will be established before the actual test is conducted. Second, the test case must define a clear sequence of actions, and input data, that constitute the test sequence. Finally, the test case must define an expected result.

 

Our tests must be verifiable. That is, it must be possible to observe the results of the test and determine whether the expected result was achieved. For object-oriented systems this includes being able to examine the state of the object before and after a test is conducted. This last point is a debatable one. A "black-box" testing approach seems to say that only what is externally observable is used to verify the results of a black-box test. But outside of what? A method? An object? I usually want to look at the internal object state after exercising methods because such a small percentage of the overall functionality of an object can be observed via return values. In the PACT approach we typically circumvent information hiding and directly verify directly with the attributes until the accessor methods have been sufficiently well tested to be trustworthy.

 

Expected Results

Having a definitive expected result seems to be an obvious component of a test case, but it is one that is often overlooked and sometimes very difficult to obtain. On projects that involve operations on large databases, the expected results of a specific search often are expensive to determine but still quite necessary. In this environment, the sequencing of tests becomes very important since we probably don’t want to reload a fresh database after each specific test. The expected results in this environment must consider the effects of the previous tests.

 

For the Velocity class, the expected results from several of the operations can be prototyped on a spreadsheet such as shown in Figure 1. One thing to be careful of is the conversion from degrees to radians for the trig functions.

 

Accessor methods

I have saved the accessor methods for last even though they are the second thing I test, after the constructors. In the PACT approach a baseline test suite is created that tests all of the accessor methods. After these tests are run and verified by direct access to the attributes then other tests can be built and verified using these methods to provide access to the object’s attributes. The getSpeedX(), getSpeedY() and getDirection() methods provide access to three attributes of the class. From a black-box perspective the tester is unaware of whether these methods directly access variables or compute the values that are returned.

 

A sufficient set of test cases for these methods can be constructed from the previous analysis. For the baseline test suite, each attribute is retrieved directly from the variable that holds the attribute’s value (if such exists) and compared to the value returned by the accessor method.

Protocol Specification

The protocol specification for an object defines the sequences of messages that will be considered legitimate. These are not flows through the system because the sequence of messages received by an object may well be interspersed with messages to other objects. The state of the object; however, implicitly records these sequences in that the state at any given moment is the result of the sequence of messages received to that point in time.

 

For example, since the Velocity class is part of the Brickles game, we can expect that the reverseX() and reverseY() messages will alternate as the puck bounces first off a horizontal surface and then a vertical one. Testing several sequences of this type will identify any interactions between the two methods.

 

For a more useful example, I want to switch and consider another class, the BricklesGame class. Listing 2 shows the interface for the class and Figure 3 presents a dynamic model for the class. The dynamic model specifies the sequence of messages that are legitimate. Test cases derived from this protocol specification are in the form of graph traversals. We can omit any test case that simply uses a single transition since those will have already been constructed using the other techniques discussed earlier. Each traversal of this state machine will begin with a constructor followed by a sequence of pause/resume messages that transition from InPlay to Paused and back again. Note that the transition labeled OK is in response to the user pressing the OK dialog button.

 

One technique for systematically covering the specification is the n-way switch cover defined by Chow[1] for telecommunication protocol testing. In this technique test cases are developed based on following a transition by N additional transitions beyond the initial one. A one-way transition provides a reasonable level of coverage but does not uncover those faults that are cumulative and will not surface until several repetitions of the same pattern of transitions has been executed. For example, if the switch between paused and running was implemented by an integer counter, a possible error would be to increment the counter on both a pause and a resume. Eventually the integer value would hit MAX_INT and a failure would result of but not until many cumulative iterations of pause() and resume().

 

As always we have the question of how much testing should we do outside the explicit specification. The example here, test case #8, is testing a sequence in which a resume() message is received prior to receiving a pause() message. It is fairly easy to see that the result of such a case should simply leave the game running. This test case would catch an implementation in which the implementer used a boolean and then reversed its value at each call to either pause() or resume(). The GUI might prevent this sequence from happening in the current application, but reusing the component in another application might lead to an error when this behavior is not controlled. This behavior can be tested perhaps by doing a MouseDown (mouse button pressed) event outside the game window and then doing a MouseUp event inside the window. Through callbacks, this is equivalent to doing a resume with no initial pause.