Friday, May 31, 2013

Test Automation - Checkpoints

A test must end with a pass or fail status. Humans execute test steps, see the actual results in front of their eyes, compare that with expected results and declare the pass/fail status. Using record/replay/data, test automation tool executes the test steps. But the tool does not know expected results. We need to tell the tool about what is expected and the tool can then declare the results. This is implemented using checkpoint or verification point feature, in every tool. 

Tester feeds data on UI objects such as text box, combo box, radio button etc. When the form is submitted, usually thru a button click to save or update or delete, the application does the operation and posts a message back on the screen. The result is usually a text message such as "Your ticket is successfully booked. Thanks for choosing us.". The tester will then state that the booking test case has passed. If this message does not come, it is treated as failed. Usually the messages will be displayed in specific places of the screen or status bar or the screen may refresh with the data that is entered with an auto-generated id etc. 

As part of automating test cases, every test case will have steps and will have checkpoints. In the checkpoint, we need to feed what is the object to be looked for results, which property of the object reflects the result and what is the expected result. e.g. look in the status bar, look at the text, the text must be "The transaction is completed successfully.". In some cases, it may not be text, instead we will be looking for the price to be updated in the price text box. When you go to a travel portal, usually the number of passengers will be set to 1 and the price will be for 1 person, say $110. When you change the number of passengers to 2, the price must change to $220. So, based on one event, the expected outcome changes in another field. In this case, we must place a checkpoint on the price text box, for its text property with an expected value $220. 

In some cases, only when I check the "I accept the terms" check button, the continue button will be enabled. Till then it will be disabled. In this, we need to place a check point for the continue button, for the enabled property with the expected outcome as true/on. In an email inbox, when I delete a mail, the number of mails shown at the top will be reduced by one. Like this, the screen will show the actual results in specific places. We must find those places and put appropriate checkpoints. Some organizations will use standardization of their UI elements, such as the width and height of all push buttons must be 90 and 30 pixels respectively; in such case we must place checkpoints for the width and height properties of the buttons.

Every test case will have the standard sequence as follows.

  • Test steps to establish pre-requisite state of the application
  • Test steps for the test case
  • Checkpoint-1
  • More test steps for the test case
  • Checkpoint-2
  • Some more test steps for the test case
  • Checkpoint-3
  • Test steps to clean-up

When the checkpoint passes, the tool will declare the results in green color and when it fails, the result will be painted in red color. Ideally for every expected result in my test case, I must have a checkpoint command in the test script.

For free lessons on automation tools, visit us at

Monday, May 6, 2013

Test Automation - Data Driven Test

You need more data for testing. I would rather re-phrase this to state you need more varieties of data rather than more data itself. Each data set must help the tester to drive the application thru a different logic or path, so that we can do better testing. Take the example of booking airline ticket. It is just 1 screen that takes data and users feed these fields to get their ticket booked. Each ticket may be a different combination. A human tester gets bored when repeatedly seeing and operating on the same screen. But this has to be done any way. 

Here comes the data driven test (DDT) feature of the test automation tools. Test steps are same, but data is different, for each transaction I must use a different data set - If this is your case, use data driven test. Every testing tool comes up with a data sheet or data pool. Here you need to provide data in a spreadsheet like file. For booking a ticket you need from place, to place, one way or round trip, date of journey, return date, number of persons etc. Though there are lot more details for ticket booking, let us limit to these data at this point. 

The first thing for DDT is to create a file that has these data. Usually you can use a csv file or xls file. The first line is usually the title for the data. See the example below.

Los Angeles,New York,TWOWAY,10-JUN-2013,14-JUN-2013,2

Los Angeles,Denver,ONEWAY,10-JUN-2013,10-JUN-2013,1
Los Angeles,London,TWOWAY,10-JUN-2013,18-AUG-2013,5

We need to prepare each line of data that represent some equivalent partition set to test our application. 

Once data is created, we need to record a script using the tool, that does one ticket booking, by entering the data in the above fields. While recording we will give some data and that will reflect in the script. e.g.

BookingScreen.FromAirPort.Set "Los Angeles"
BookingScreen.ToAirPort.Set "Denver"
BookingScreen.TripType.Select "TwoWay"
BookingScreen.DateOfJourney.Set "10-May-2013"
BookingScreen.DateOfReturn.Set "12-May-2013"
BookingScreen.NumberOfPersons.Set 1

The syntax given above is generic and not specific to any tool. To ensure that the script uses the data from our file, we may have to modify the script manually or use the data driver wizard of the tool. We need to set the value for each UI field data entry, to point to a column in the data sheet. Also, we must loop thru all rows in the data sheet, so that the script executes for all records and books multiple tickets. It may  look like the following.

DataSheet = "C:\\mydata.csv"

For currRow = 1 to DataSheet.RowCount
   DataSheet.CurrentRow = currRow
   BookingScreen.FromAirPort.Set DataSheet.getColumnvalue("FROM-PLACE")
   BookingScreen.ToAirPort.Set DataSheet.getColumnvalue("TO-PLACE")
   BookingScreen.TripType.Select DataSheet.getColumnvalue("TRIP-TYPE")
   BookingScreen.DateOfJourney.Set DataSheet.getColumnvalue("JOURNEYDT")
   BookingScreen.DateOfReturn.Set DataSheet.getColumnvalue("RETURNDT")

The simple code, will get all records from the csv file and repeat feeding the screen with different data sets from the csv file. This way, a 10 line code can enter 10000 records with little human interference.