Topic: Understanding the Testing Process

Silktest And WinRunner Feature Descriptions

FEATURE DESCRIPTIONS
For the sake of consistency alphabetical ordering was selected to describe SilkTest features first,followed by WinRunner features in each of the following sections.

Startup Initialization and Configuration
• SilkTest derives its initial startup configuration settings from its partner.ini file. Thisthough is not important because SilkTest can be reconfigured at any point in the session byeither changing any setting in the Options menu or loading an Option Set.An Option Set file (*.opt) permits customized configuration settings to be established foreach test project. The project specific Option Set is then be loaded [either interactively, orunder program control] prior to the execution of the project’s testcases.The Options menu or an Option Set can also be used to load an include file (*.inc)containing the project’s GUI Declarations along with any number of other include files containing library functions, methods, and variables sharedby all testcases.
• WinRunner derives its initial startup configuration from a wrun.ini file of settings.During startup the user is normally polled [this can be disabled] for the type of addins theywant to use during the session .The default wrun.ini file is used when starting WinRunner directly, while project specificinitializations can be established by creating desktop shortcuts which reference a projectspecific wrun.ini file. The use of customized wrun.ini files is important because once WinRunner is started with a selected set of addins you must terminate WinRunner and restart it to use a different set of addins.The startup implementation supports the notion of a startup test which can be executedduring WinRunner initialization. This allows project-specific compiled modules [memoryresident libraries] and GUI Maps to be loaded. The functions and variables contained in these modules can then be used by all tests that are run during that WinRunner session.Both tools allow most of the configuration setup established in these files to be over-ridden withruntime code in library functions or the test scripts.

Test Termination
• SilkTest tests terminate on exceptions which are not explicitly trapped in the testcase.
For example if a window fails to appear during the setup phase of testing [i.e. the phase drivingthe application to a verification point], a test would terminate on the first object or windowtimeout exception that is thrown after the errant window fails to appear.
• WinRunner tests run to termination [in unattended Batch mode] unless an explicit action is taken to terminate the test early. Therefore tests which ignore this termination model will continue running for long periods of time after a fatal error is encountered. For example if awindow fails to appear during the setup phase of testing, subsequent context sensitive statements [i.e. clicking on a button, performing a menu pick, etc.] will fail—but this failure occurs after a multi-second object/window “is not present” timeout expires for each missingwindow and object. [When executing tests in non-Batch mode, that is in Debug, Verify, orUpdate modes, WinRunner normally presents an interactive dialog box when implicit errorssuch as missing objects and windows are encountered].
Addins and ExtensionsOut of the box, under Windows, both tools can interrogate and work with objects and windowscreated with the standard Microsoft Foundation Class (MFC) library. Objects and windows created using a non-MFC technology [or non-standard class naming conventions] are treated ascustom objects. But objects and windows created for web applications [i.e. applications which run in a browser],Java applications, Visual Basic applications, and PowerBuilder applications are dealt with in aspecial manner:
• SilkTest enables support for these technologies using optional extensions. Selectedextensions are enabled/disabled in the current Option Set [or the configuration established bythe default partner.ini option settings].
• WinRunner enables support for these technologies using optional addins. Selected addins are enabled/disabled using either the Addin Manager at WinRunner startup, or by editing the appropriate wrun.ini file prior to startup.
Note that (1) some combinations of addins [WinRunner] and extensions [SilkTest] are mutuallyexclusive, (2) some of these addins/extensions may no longer be supported in the newest releasesof the tool, (3) some of these addins/extensions may only support the last one or two releases ofthe technology [for example version 5 and 6 of Visual Basic] and (4) some of these addins andextensions may have to be purchased at an addition cost.

Visual Recorders
SilkTest provides visual recorders and wizards for the following activities:
• Creating a test frame with GUI declarations for a full application and adding/deleting selective objects and windows in and existing GUI declarations frame file.
• Capturing user actions with the application into a test case, using either context sensitive[object relative] or analog [X:Y screen coordinate relative] recording techniques.
• Inspecting identifiers, locations and physical tags of windows and objects.
• Checking window and object bitmaps [or parts thereof].
• Creating a verification statement [validating one or more object properties].WinRunner provides visual recorders and wizards for the following activities:
• Creating an entire GUI Map for a full application and adding/deleting selective objects andwindows in an existing GUI Map. It is also possible to implicitly create GUI Map entries bycapturing user actions [using the recorder described next].
• Capturing user actions with the application into a test case, using either context sensitive[object relative] or analog [X:Y screen coordinate relative] recording techniques.
• Inspecting logical names, locations and physical descriptions of windows and objects.• Checking window and object bitmaps [or parts thereof].
• Creating a GUI checkpoint [validating one or more object properties].
• Creating a database checkpoint [validating information in a database].
• Creating a database query [extracting information from a database].
• Locating at runtime a missing object referenced in a testcase [and then adding that object tothe GUI Map].
• Teaching WinRunner to recognize a virtual object [a bitmap graphic with functionality].
• Creating Data Tables [used to drive a test from data stored in an Excel-like spreadsheet].• Checking text on a non-text object [using a built-in character recognition capability].
• Creating a synchronization point in a testcase.
• Defining an exception handler.Some of these recorders and wizards do not work completely for either tool against allapplications, under all conditions. For example neither tool’s recorder to create a full GUI Map[WinRunner] or test frame [SilkTest] works against large applications, or any web application.Evaluate the recorders and wizards of interest carefully against your applications if these utilitiesare important to your automated testing efforts.

Segue testing methodology is a six-phase testing process:
1. Plan - Determine the testing strategy and define specific test requirements.
2. Capture - Classify the GUI objects in your application and build a framework for running your tests.
3. Create - Create automated, reusable tests. Use recording and/ or programming to build test scripts written in Segue's 4Test language.
4. Run - Select specific tests and execute them against the AUT.
5. Report - Analyze test results and generate defect reports.
6. Track - Track defects in the AUT and perform regression testing.