|
The example I am going to use is an instance of the
Brickles game that appeared on many early home computers. The game has
been implemented in Java and C++ and we use the example in several of
our training courses at Software Architects. I will use the Java
version but it really makes little difference, for tests based on the
specification, which language is used. There are a few attributes of a
method that may be more precisely specified in one language than
another and these provide valuable input into testing.
This first class under test represents the velocity of a moving
object in the game. The velocity of a moving object has two attributes,
the speed of the object and the direction in which the object is
moving. Since the graphics are represented in a cartesian coordinate
system, the velocity is decomposed into components along the x and y
dimensions. The value of each component depends upon the magnitude of
the speed and the direction. The complete source code for the class can
be obtained from http://www.cs.clemson.edu/~johnmc so that you can
consider the types of errors that are in the class and whether tests
constructed from the class’s specification actually detect them.
Note that the pre-conditions for this Java class are
carefully written to define when an object comes into existence. This
is particularly important in languages such as Smalltalk and Java. In
Java much use is made of class methods. For a class method to work, no
object needs to have been explicitly created. So, for example, there
would probably be a Math.cos() message, in decomposeSpeed, is a message to the Math class rather than a message to an object that must have been initialized by the programmer. The decomposeSpeed
method itself has a pre-condition that requires that the programmer has
created a Velocity object prior to sending this message.
Test Cases
There are several essential features that a test case
must possess. First, the test case must define exactly how the
pre-conditions will be established before the actual test is conducted.
Second, the test case must define a clear sequence of actions, and
input data, that constitute the test sequence. Finally, the test case
must define an expected result.
Our tests must be verifiable. That is, it must be possible to
observe the results of the test and determine whether the expected
result was achieved. For object-oriented systems this includes being
able to examine the state of the object before and after a test is
conducted. This last point is a debatable one. A "black-box" testing
approach seems to say that only what is externally observable is used
to verify the results of a black-box test. But outside of what? A
method? An object? I usually want to look at the internal object state
after exercising methods because such a small percentage of the overall
functionality of an object can be observed via return values. In the
PACT approach we typically circumvent information hiding and directly
verify directly with the attributes until the accessor methods have
been sufficiently well tested to be trustworthy.
Expected Results
Having a definitive expected result
seems to be an obvious component of a test case, but it is one that is
often overlooked and sometimes very difficult to obtain. On projects
that involve operations on large databases, the expected results of a
specific search often are expensive to determine but still quite
necessary. In this environment, the sequencing of tests becomes very
important since we probably don’t want to reload a fresh database after
each specific test. The expected results in this environment must
consider the effects of the previous tests.
For the Velocity class, the expected results from several of the
operations can be prototyped on a spreadsheet such as shown in Figure
1. One thing to be careful of is the conversion from degrees to radians
for the trig functions.
Accessor methods
I have saved the accessor
methods for last even though they are the second thing I test, after
the constructors. In the PACT approach a baseline test suite is created
that tests all of the accessor methods. After these tests are run and
verified by direct access to the attributes then other tests can be
built and verified using these methods to provide access to the
object’s attributes. The getSpeedX(), getSpeedY() and getDirection()
methods provide access to three attributes of the class. From a
black-box perspective the tester is unaware of whether these methods
directly access variables or compute the values that are returned.
A sufficient set of test cases for these methods can
be constructed from the previous analysis. For the baseline test suite,
each attribute is retrieved directly from the variable that holds the
attribute’s value (if such exists) and compared to the value returned
by the accessor method.
Protocol Specification
The protocol specification for
an object defines the sequences of messages that will be considered
legitimate. These are not flows through the system because the sequence
of messages received by an object may well be interspersed with
messages to other objects. The state of the object; however, implicitly
records these sequences in that the state at any given moment is the
result of the sequence of messages received to that point in time.
For example, since the Velocity class is part of the Brickles game, we can expect that the reverseX() and reverseY()
messages will alternate as the puck bounces first off a horizontal
surface and then a vertical one. Testing several sequences of this type
will identify any interactions between the two methods.
For a more useful example, I want to switch and
consider another class, the BricklesGame class. Listing 2 shows the
interface for the class and Figure 3 presents a dynamic model for the
class. The dynamic model specifies the sequence of messages that are
legitimate. Test cases derived from this protocol specification are in
the form of graph traversals. We can omit any test case that simply
uses a single transition since those will have already been constructed
using the other techniques discussed earlier. Each traversal of this
state machine will begin with a constructor followed by a sequence of
pause/resume messages that transition from InPlay to Paused and back
again. Note that the transition labeled OK is in response to the user
pressing the OK dialog button.
One technique for systematically covering the
specification is the n-way switch cover defined by Chow[1] for
telecommunication protocol testing. In this technique test cases are
developed based on following a transition by N additional transitions
beyond the initial one. A one-way transition provides a reasonable
level of coverage but does not uncover those faults that are cumulative
and will not surface until several repetitions of the same pattern of
transitions has been executed. For example, if the switch between
paused and running was implemented by an integer counter, a possible
error would be to increment the counter on both a pause and a resume.
Eventually the integer value would hit MAX_INT and a failure would
result of but not until many cumulative iterations of pause() and resume().
As always we have the question of how much testing
should we do outside the explicit specification. The example here, test
case #8, is testing a sequence in which a resume() message is received prior to receiving a pause()
message. It is fairly easy to see that the result of such a case should
simply leave the game running. This test case would catch an
implementation in which the implementer used a boolean and then
reversed its value at each call to either pause() or resume().
The GUI might prevent this sequence from happening in the current
application, but reusing the component in another application might
lead to an error when this behavior is not controlled. This behavior
can be tested perhaps by doing a MouseDown (mouse button pressed) event
outside the game window and then doing a MouseUp event inside the
window. Through callbacks, this is equivalent to doing a resume with no
initial pause.
|