Topic: Data-driven Testing

Automated tests play back a recorded (or programmed) sequence of user actions that cover a certain area of the tested application.

To get larger coverage, you can perform tests with different input data. Suppose, for example, you test an input form of an application. After you have recorded a script, it contains only values that you entered during the recording. Most likely, data you have specified does not cause errors in the application, but other data may cause them. Data-driven testing allows you to test the form with a different set of input values to be sure that the application works as expected.

• Introduction


• Data Driven Automation Frameworks


• Data Driven Scripts


• TestComplete's Support for Data-Driven Testing

introduction

Data-driven testing means that scripts read data from an external storage site, for example, from a file or database, rather than use values hard-coded in the script.

Such a separation makes tests logically simpler. A script containing several sets of hard-coded values can be rather difficult to modify. For instance, if you need more input data, you will have to modify the script code.

Normally, the storages hold both input and verifying data. A data-driven test includes the following operations performed in a loop:

1. Retrieving input data from storage.
2. Entering data in an application form.
3. Verifying the results.
4. Continuing with the next set of input data.

TestComplete includes a number of features to perform these steps. You can select the functionality that best suits your needs.

Over the past several years there have been numerous articles done on various approaches to test automation. Anyone who has read a fair, unbiased sampling of these knows that we cannot and must not expect pure capture and replay of test scripts to be successful for the life of a product. We will find nothing but frustration there.

Sometimes this manifesto is hard to explain to people who have not yet performed significant test automation with these capture\replay tools. But it usually takes less than a week, often less than a day, to hear the most repeated phrase: "It worked when I recorded it, but now it fails when I play it back!"

Data driven scripts are those application-specific scripts captured or manually coded in the automation tool’s proprietary language and then modified to accommodate variable data.

Variables will be used for key application input fields and program selections allowing the script to drive the application with external data supplied by the calling routine or the shell that invoked the test script.

Since DDT itself is very straightforward, TestComplete’s direct support for the technique is likewise quite straightforward.

If you examine the TestComplete online help, under “data-driven testing”, you’ll find a fairly long help topic with links to external pages on the subject, but also links to five script routines. It’s worth noting that while these methods are listed with the DDT topics, they’re not the only routines you’ll need to perform DDT, but for now we’ll look at those five. They are GetCSVCount, GetCSVItem, CallMethod, SetValue and GetValue. The first two methods are ones that you’ll use quite a bit in DDT, the other three will see less frequent use unless you are getting into some fairly robust DDT scripting. 

GetCSVCount and GetCSVItem work with comma-separated strings in order to provide an easy way to get to the values contained within the strings. This is called “parsing”. A comma-separated value (CSV) string might look something like this: 

You can see that each element of this string is separated by a comma. Each individual element of the string is a called a “field” or a “value”. The GetCSVCount and GetCSVItem methods work with strings like this. If we passed this string in to the GetCSVCount function, we’d get a return value of 10. There are ten discreet values in this CSV string. In addition, if we wanted to the get the customer’s first name, we could use GetCSVItem to do that. 

Here’s an important thing to keep in mind: GetCSVItem is what’s known as “zero-based”. That means that the first item in the CSV string is considered number 0 rather than number 1. Likewise, if we wanted the last item, it would be item number 9, not number 10. Thus, if we wanted the customer’s first name, we’d have to ask for item number 2, not number 3, thus:

So, with these two functions, we’re able to easily retrieve values from CSV files, something that is quite useful if our external DDT data is in CSV text files or spreadsheets. Returning again to our customer/orders example, we might see the following. (This is pseudo-code only.) 

function ReadExternalData(aFileName)
{
  var lsLine
  var lsCustomerName, lsAddr, lsCity, lsRegion, lsPostalCode
  openFile(aFileName)
  while not EOF(aFileName)
  {
    lsLine = readLine(aFileName) 
    if (GetCSVCount(lsLine) == 5 
    {
      lsCustomerName = GetCSVItem(lsLine, 0) 
      lsAddr = GetCSVItem(lsLine, 1)
      lsCity = GetCSVItem(lsLine, 2)
      lsRegion = GetCSVItem(lsLine, 3)
      lsPostalCode = GetCSVItem(lsLine, 4)
     
      EnterCustomerRecord(lsCustomerName, lsAddr, lsCity, lsRegion, lsPostalCode)
    }
  }
  closeFile(aFileName)
}
function EnterCustomerRecord(aCustomerName, aAddr, aCity, aRegion, aPostalCode)
{
  var p, w
  p = Sys.OrdersProcess
  w = p.MainForm
  w.Activate()
  w.ToolButton5.Click(6, 7)
  w = p.OrderFrm
  w.Activate()
  w.CustomerName_Edit.wText = aCustomerName
  w.Street_Edit.wText = aAddr
  w.City_Edit.wText = aCity
  w.State_Edit.wText = aRegion
  w.Zip_Edit.wText = aPostalCode
  w.OKButton.Click()
  validateRecordEntry() 
}

Don’t worry about trying to make this code actually work – it’s rather abstracted and is for demonstration only, but your actual DDT code might be similar. What this small amount of code does is very powerful and serves as a great example of a standard DDT scenario. While it’s also possible to do more complex tasks, which I’ll examine in future papers, this is a good starting place. 

As for the other three methods covered in the help (CallMethod, SetValue and GetValue) I won’t talk about those now – they’re not necessary for this primer on DDT.