Tuesday, July 23, 2013

Test Automation - Exception Handling

Test automation is all about a tester generating test scripts, i.e. code. So, all problems of what a developer would face, a tester would also face. Hence test script must also be fully tested by the tester. But, there are 4 major factors that can affect the test script and those 4 are unpredictable. Hence, the test script must gracefully handle those areas. This is called exception handling or known as recovery scenarios.

Hurdle 1 - Unknown pop-ups. When the script executes, step by step, application reacts to those steps, and there is a specific expectation out of each step. For example, if a user enters a valid account number in account number field and press tab, the account name must be auto populated in name field; when script executes, due to a wrong data fed into account number field, if the application shows a pop-up stating "Hey, the account number does not exist; please check the data", that pop-up is a blocker. Without closing that, nothing can be done and script cannot proceed. This is just one example; similarly in many occasions, due to data issues or application functional issues, OS related issues, unwanted popups will crop up. The tester must ensure all such pop-ups are addressed when script runs. Ideal solution is close the pop-up, and continue or go to next test case.

Hurdle 2 - Objects or pages not found or disabled for input. This happens due to a functional bug in the application. Take an example. When user goes to account balance screen, the account number must be enabled for data entry. Occasionally if that field does not appear or disabled, the script will go ahead try to enter the account number and the type event will fail. It is not possible to check the enabled or displayed property for every field to be ON, before every step. A field not found error may happen, if the display of the field, goes outside the display resolution of the monitor. If this kind of errors happen, it is better to relogin to the app and move to the next test case.

Hurdle 3 - Application crashes. This is usually due to some critical bug in the application. The test script will not find the application itself to execute the next step. In this case, it is ideal to restart the app and start the next test case. If we try to start the same test case, it may crash again and it may go in a loop.

Hurdle 4 - Script error. This is due to a wrong logic by the tester in the test script. This may be due to accessing wrong array locations, divide by zero, trying to open a non-existing file etc. If this kind of errors happen, it is better to relogin to the app and move to the next test case.

Tools provide a variety of mechanisms to handle the exceptions, either thru coding or by configurations or by both. Every exception is a learning. It is very difficult to identify all exceptions and handle those before script runs. It  is an evolving process. So, as and when new unforeseen exceptions do happen, add them to the exception handler library.


For free lessons on automation tools, visit us at http://www.openmentor.net.





Monday, July 15, 2013

Test Automation - Scripting Essentials

Record and replay alone cannot solve test automation problems; also record/replay may not be always possible. The test script may need some kind of decision making at run time; script may need some intelligence in handling situations; the test may have to alter its path based on some values coming on screen at run time. The simple solution for all kinds of such issues is - Scripting. Every tool provides some scripting language such as VBScript, Java, JavaScript, C#, Ruby etc. So, if we mix record/replay and scripting, we increase the power of the tool by thousand times.

Rule 1: Never build application logic in the scripting language. Example. You recorded how to book one-way ticket for 1 person; for 2 people, if app needs the price to be multiplied by 2, do not build that multiplication logic in your script. Because, whenever the app logic changes, script needs to be modified and retested. You may make some bugs in coding in the scripting language, while modifying that logic. Always, see what is the input, manually determine what must be the expected output, feed that output as checkpoint values, go on. Your brain is the best tool, better than any automation tool.

Rule 2: Comment your script well. Maintenance of script is very important. Hence make sure another tester can easily understand your script. 

Rule 3: Unit test your test script. Remember, developers make mistakes in their code. When you program, you will also do mistakes in your code. Being a tester, nothing guarantees that your test scripts will work without testing.

Rule 4: Ensure your test functions, work for a variety of parameters. A single function may feed 1000s of data thru data driven test. Hence test your script with multiple data.

Rule 5: Avoid nested if conditions. At the most you can do 2 levels. This itself will consume more time for you to unit test your test scripts.

Rule 6: Avoid nested loops. Nested loops may not be actually required in 95% of the cases. So be judicial when using nested loops.

Rule 7: Have a traceability matrix for your test scripts. You need to have a spreadsheet that documents the input params, output params, file details, function details, caller details for every test script/function. Else, when it grows to 1000 test scripts, changing one script may affect another, if you are not aware of the dependencies.

Rule 8: Always put all test scripts in a batch and run. This will eliminate the base state problems.
 
Rule 9: Always make another tester (other than the author of the test script) to run the test batch. This will eliminate human related issues and documentation issues.


Rule 10: Always run the test batch from another machine (bot used for building the test scripts). This will eliminate system related issues, hardcoding of drive/folder names etc. and documentation issues.


For free lessons on automation tools, visit us at http://www.openmentor.net.