Test Automation - Object Identification

There are about 30 students in a class; you need to find out whether the person you want to talk to, is there or not; assume you have not seen him. You first go and ask "Is John here? I have a parcel to deliver". So, you use the first name to identify a person. If there are 2 Johns in the same class, then you will ask "John Bosco". So you use last name also as additional qualifier to uniquely identify that person, among 30 other students. As a rare case, if both are John Bosco, what will you do? You may say "John Bosco, son of  David P Bosco". The goal is to deliver the parcel to the right person. Testing tools also work on very similar way to carry out click or type actions, on the exact objects on the application UI.

If one has a very good grasp on object identification concepts, 80% of the automation problems are already solved. An object means an item on the user interface, on which you either click or type or you read it and infer some information; e.g. first name text box, state combo box, save button, etc. Recording feature in every tool, internally does this object identification. Most of the tools use a way of storing the details of objects in a separate file. In HP QTP it is called Object Repository, in IBM Rational Functional Tester it is called Object Map, in Borland SilkTest it is called frames.inc, and so on. 

Every object has a type, aka class. The most common types are text box, check button, radio button, combo box, push button, menu bar, tool bar, label, link, image, grid (rows, columns, cell), scroll bar, status bar, progress bar. These are the typical items we see in every application. But the exact way, every technology calls these classes will be different; for example, native Windows C++ will say Push Button, Java will say JButton, Turbo C++ may say TButton, for the same push button you see on screen. This specific text is called nativeclass.

They also have a name associated with them. This is not mandatory, but preferable. The objects may display a name on screen for the user to read, but it may internally use another name. This is like the nick name we use for people. Tools understand the internal name of the objects, rather than the display name of the objects. Ideally developers of the screens must use same, meaningful names for display as well as internal purposes. 

The best way to identify an object is to use the type/class of the object and its internal name. e.g. button/OK, combobox/States, textbox/ZIP code etc. Sometimes, the same display name may appear 2 times on the UI. Consider a screen that takes your personal and office addresses; both will have street/city/state text boxes. So the display name is same; the humans understand this well; but for the tools to understand two objects with same display name, it is better to use the internal name given by developers. 

When we need additional details to uniquely identify an object, there are other properties such as location (x, y coordinates), index of the object (nth object from top of the screen) are usually used. But position based identification is a risky option; this may fail when screen is updated with new objects or a few objects are removed, that may affect the relative position of this field on the screen. 

To understand the objects and their properties, tools provide object spy or object inspector utilities. Before you start automating, spend a few hours to see the class, name and other properties of every object on every single screen. If you see objects with confusing properties or non-unique properties, negotiate with development team to have consistent naming of objects.

Remember: Proper naming of objects reduces the time to automate significantly. This simple process discipline is required by the dev team to make test automation is easy.

For a live demo of this, please click on this url.

Test Automation Basics

One of the standard issues every test manager faces, is to have a proof of what is tested and what is not. If I have 300- 400 test cases, I am in full control. But if I have 5000 odd test cases how do I know that the 3617^th test case is executed or not ? I trust my testers. Imagine I need to test 200 test cases in 3 different browsers, it multiplies my effort. I can not have 3 testers to do it in three different browsers, nevertheless it must be done. When I need some testing to be done on some critical builds, I do not know, on that day my tester gets illness. He does not turn up to work! My client is waiting to get the status. Oh, what a mess!

4 to 6 weeks from the day of first test execution cycle, my testers get bored of the test cases. Their eyes are not as sharp as before. They feel tired. But they want salary revision! One of my testers claim that he has done 80 test cases since morning I am more than 100% sure that he could not have done that much. How can I be assured of some one did it or did not do it?

The one best answer is automation. Instead of manually executing the test cases, do the testing using a tool. This can solve all the above mentioned problems. Tool never gets tired, tools never get bored, they do not ask salary revisions, they are fast, they do not apply leave and they are consistent!

Before doing any automation test, we must carry out a small proof of concept (POC) or a feasibility study of automation tool on our application. This may take from 4 hours to 8 hours. But this can solve a lot of issues that will be faced  by the team at a later time.

When it comes to test automation, a tester becomes a developer of automated test scripts. This means, the tester generates code using the tool to test the application. There are a variety of tools available in the market. QTP by HP, SilkTest by Borland, Rational Functional Tester by IBM, TestComplete by SmartBear, Selenium, Ranorex etc. are a few to name. Some tools work only on browser based web apps, some work on only rich/thick clients app and some work on both. But all these tools use the UI of the application to run tests. The human tester uses the UI to carry out functional tests and these tools do the same. Instead of human doing a click, the tool clicks on a button, instead of human typing, the tool mimics the key strokes.  

The following are the most common features that almost all tools share.

1. Recording (Capture test steps)
2. Replaying  (Playback test steps)
3. Object Identification (knowing the forms and fields attributes on screen)
4. Data Driven Test (use same steps with different data sets)
5. Check points (Verification points, compare the actual results to expected results)
6. Scripting (use a programming language to add intelligence to test scripts)
7. File and Database handling (if results are stored on disk)
8. Exception handling (recovery path when test script itself fails)

You need to think of test automation if you say Yes to one or more of the following points.

1. The number of test cases for my product is large and I have many regression rounds
2. My application is a product and not just a 4 months project
3. My product needs to be tested on multiple environments for compatibility
4. My product is being used by 1000s of customers and we cannot have a single regression issue
5. I test my product very frequently, almost everyday
6. My team costs me more and more and project bleeds for profitability

In the coming sections, we will discuss more about each of the automation features, in detail.

For free automation courses, visit http://www.openmentor.net.
 

The CRUD Approach

With the knowledge of "what" and "how" to test, let us go one more step forward in testing. Majority of the tests fall under the category of functionality. Functionality is nothing but the way the software responds to its inputs based on the programmed logic. The functionality itself can be broken to smaller pieces to make it simple. Here is a CRUDe way to do it.


Let us take a simple social networking application. You can create a new account, edit your profile, add a friend, delete a friend, add comments, delete comments, reload your photograph, view the comments by others etc. Take an instant messenger like Skype. You can add a contact, rename the contact, edit your text, delete a contact etc. Look at a documenting tool like MS Word. You can create a document, modify its contents, rename it, delete some portions of it etc.


So, what do you see as a common set of activities in all the above? Add something, edit or modify it, read or view it, delete it. In other words, Create Read Update Delete and that is CRUD. There is no exception to this in software. Every software will have to necessarily support these features; so if we chase these, we can test a software in a structural manner. Once you read the specifications of the software, see whether that requirement falls under create or read or update or delete. Once you identify that, you can blindly apply the following test conditions to that particular requirement.


If the requirement falls under Create (also termed as New, Add, Insert) category, you need to define test conditions for:

1. Creating a brand new item one at a time
2. Creating a duplicate item
3. Creating more than one items in a single shot (if applicable)

If the requirement falls under Read (also termed as View, Select, Open) category, you need to define test conditions for:

1. Reading an existing item one at a time
2. Reading more than one existing items at a time
3. Reading a non-existing item
4. Reading in a sorted order (such as sort a grid in ascending or descending order)
5. Read an item based on wild cards (such as search or filter items)

If the requirement falls under Update (also termed as Edit, Modify, Change) category, you need to define test conditions for:

1. Update an existing item one at a time
2. Update more than one existing items 
3. Update a non-existing item 

If the requirement falls under Delete (also termed as Remove, Trash, Erase) category, you need to define test conditions for:

1. Delete an existing item one at a time
2. Delete more than one existing items 
3. Delete a non-existing item 

So, if we apply this CRUD method to every single requirement, we start getting more and more branches for every such operation. 

What is the guarantee that most of the software operations are restricted to only this CRUD concept? If you look at any database, you can see just four important operations - SELECT, INSERT, UPDATE and DELETE. Whether it is Oracle or MySQL or MS SQL or DB2, all use these 4 operations day in and day out. And ultimately every software must deal with a database  to finally store the data. Hope this is quite a convincing argument.

For more in testing, please visit http://www.openmentor.net.

SDLC - Coding, Testing, Implementation, Maintenance Phases

Coding Phase

The LLD document contains all the details for a programmer to convert it into a working program. Designed in the right way, the job of the developer is to convert the pseudo code into a programming language in the specified platform. The developer will be given specific guidelines that tell how to code. This guideline includes the naming convention of procedures, variables, commenting methods etc. As per the guidelines, the developer codes and at the end of the coding, he/she compiles the program unit.

By compiling and correcting the errors, all syntax error and removed. Syntax errors are the easy ones to rectify. The logical errors in the programs are the difficult ones to identify and then rectify. By logical error, we mean, the program behaves in a way, that it is not expected to behave. The role of testers is to identify the logical errors at various stages.

Testing

*This section provides only a shorter version of the testing techniques.

Testing is the process of evaluating a system or application, to check whether the application meets all requirements of the client and to detect the errors.

Generally testing can be classified into static testing and dynamic testing.

Again Dynamic Testing is classified into two types: Structural Testing (or) white box , Functional Testing (or) Black Box testing.

Static Testing

Verification activities fall into the category of Static Testing. Static testing refers to testing something that’s not running. It is examining and reviewing it. i.e., to check whether the work done meets the standards of the organization. Reviews, Inspections and Walk-throughs are static testing methodologies.

Example: The specification is a document and not an executing program. When we read it to find out the issues, it is considered as static testing.

Dynamic Testing

Dynamic Testing involves working with the software, giving input values and checking if the output is as expected. These are the Validation activities. Unit Tests, Integration Tests, System Tests and Acceptance Tests are few of the Dynamic Testing methodologies.

Techniques used are determined by type of testing that must be conducted.

• Functional ("black box") testing.

• Structural (usually called "white box") testing.

Black box testing involves looking at the specifications and does not require examining the code of a program. Tests that examine the observable behavior of software as evidenced by its outputs without referencing to internal functions is black box testing. It is not based on any knowledge of internal design or code and tests are based on requirements and functionality. Nowadays there are automatic code generation tools and code re-use becomes more prevalent, analysis of source code itself becomes less important and functional tests become more important.

Black box testing is easy as it is based on system’s response for a given user’s input. If we know the business functionality of the product, we can do black box testing. On the other hand, white box testing requires programming knowledge to know the internals of the code. Also it is time consuming. So only a developer can become a white box tester.

Unit Testing

Soon after the program is corrected for syntax errors, the program has to be checked for logical errors, at the unit level. Programs may have simple user inputs or outputs thru screens or reports. The inputs must be validated for their format, data type, boundary conditions etc. Also, the elementary functionality of the program must be verified. In most of the cases, the programmers themselves will do this unit testing. Many inputs will be verified in this way and the program unit will be checked for the basic functionality in this phase.

If there are any problems while doing this testing, the programmers will debug the program and the bugs will be fixed and will be tested again for the bug fix. While doing the unit testing, that particular unit need not wait for another unit to be completed. It can be tested independently. Unit testing is also termed as component testing. For example, if we need to test a java class and method, even without any screen, the programmer can test the same. Thus any piece of code, that can be independently executed, can be independently tested as well. This is termed as unit testing.

Integration Testing

When all the individual program units are tested in the unit testing phase and all units are clear of any known bugs, the interfaces between those modules will be tested, to establish that they communicate to each other property via the specified APIs and thus they can be integrated into an application. The integration test may be performed by the independent testers or by the development team members.

System Testing

After all the interfaces are tested between multiple modules, the whole set of software is tested to establish that all modules work together correctly as an application or system or package. This is again performed by independent testes. The testers will have to do the system testing as though they are the end users of the application. Systems testing includes special testing methods like performance testing, interoperability testing, stability testing etc.

Acceptance Testing

After the software product is tested by the SDU in 3 different stages (unit, integration and system), the client will test it, in their place, in a near-real-time or simulated environment. This establishes that the software meets the requirements of the end user. This testing is carried out as though they are using the system in their real office, to mimic the real time scenarios.

Implementation Phase/Support

This Phase will provide users with the documentation and training required to use the system effectively. Data Conversion will only occur once, but user documentation will be required. Deployment of the product will be carried out, on the hardware that is going to be used in production (on live systems). Deployment itself requires careful planning. Once the product is deployed, initial data will be populated, user training will happen.

Release to Production and warranty Period

When the clients to the acceptance testing and finds no problems, then they will accept the software and now they have to start using the software in their real office. It may be a real bank for a banking application. During the acceptance testing also, there may be some bug fixed done to the software. Before the software is put into production (real time environment), the SDU must release the latest versions of the software, which has no known bugs to the client. This may be in a CD or tape or an Internet download. The clients will then install the latest software in their production system and will start using. This is called go-live process.

In the same way every product has a guarantee period software also has a warranty period. Normally it will be a 60-day or a 90-day warranty period for most kinds of software. Depending upon the complexity and size of the software the warranty period may be extended or shortened. During this warranty period, in case of any problems, the SDU has the responsibility to rectify the problem, at no charges. Again, it depends upon the contract, whether to charge for the fix or not and may vary from industry to industry.

Maintenance Phase

After the warranty period, the software enters the maintenance phase. During the maintenance phase, 3 things happen to the software. They are :

• Bug fixing

• Upgrade

• Enhancement

During the maintenance phase, because of some untested scenarios, the software may give errors or logical problems. This has to be fixed. This is same as any other bug fixing. This is similar to any repair works done to a vehicle, by its owner.

In the software field, the software versions are constantly changing and the applications may have to keep themselves upgraded to the newer versions of the software. For example, the anti-virus software must be constantly upgraded to address new viruses, which come very day. If the operating system version charges, it may necessitate to introduce some changes in the programs. So, this kind of upgrade works to the existing software applications, because of operating system or environmental changes or hardware changes may have to be incorporated.

Any software is not all complete and there are enough rooms to add new features to an existing software. For example, today the railways reservation system may not be in the internet; but tomorrow there may be a need to go in the internet way. So, new modules are to be added to the existing system to address this. This is enhancement.

After some time, the software may become obsolete and will reach a point that it cannot be used. At that time, it will be replaced by another software which is superior to that. This is the end of the software. The software will reach its end, when the maintenance cost of the software becomes more than the new software available. Software can become obsolete when the company does not upgrade it to work on new environments such as databases or operating systems. When the software becomes incompatible with the environment, users will move on to another software, dumping the existing one.

For recorded training content, please visit http://www.openmentor.net.

Top 100 Test Scenarios – Inventory Management

When it comes to testing a product that requires a lot of domain knowledge, it usually takes more time to train a new person. To minimize the learning curve, Softsmith Infotech has made an attempt to provide a simple list of business scenarios. We talked to a variety of actual users, who use various inventory management applications and asked them what are the critical scenarios they look at. This list is a compilation of such scenarios from the real users. There may be other people who may give priority to many other test scenarios - no harm, you please add those to this list.

Top 100 Test Scenarios – Inventory Management

1. Configure part number generation settings and generate new part numbers
2. Manage part numbers and serial numbers and check for uniqueness
3. Configure item categories and sub categories and map items
4. Configure bay, bin/bucket settings and allocate items
5. Configure unit of measurement and map to items
6. Configure item reorder levels
7. Configure item expiry dates
8. Configure Bill of Material details and rearrange items
9. Configure alternate BOM details
10. Configure base price and depreciation settings of items
11. Create Goods Receipt Notes with multiple line items
12. Approve or reject GRNs
13. Create Goods Issue Notes with multiple line items
14. Approve or reject Goods Issue Notes
15. Create Goods Inspection notes with multiple line items
16. Approve or reject Goods Inspection Notes
17. Set up lot number allocation for bulk storage items
18. Map item storage UOM to vendor UOM
19. Manage vendor details and map items supplied
20. Create stock adjustment notes with multiple line items
21. Approve or reject stock adjustment notes
22. Create inventory movement within storage areas of same location
23. Create inventory movement across storage areas of different locations
24. Enter Physical Inventory data
25. Compare physical inventory and system data
26. Perform batch updation of stocks based on receipts, issues and movement transactions to sync up system
27. Manage bin cleaning data
28. Configure old part number to new part number mapping details
29. Export inventory batch data to central warehouse
30. Import inventory batch data from central warehouse for synchronization
31. Calculate daily/weekly/monthly closing and opening balances for item quantities
32. Search product availability
33. Map invoice details with GRN details
34. Verify GRN details with PO details
35. Keep issuing items below reorder level and zero stock
36. View email contents on item reorder alerts
37. Reverse a stock adjustment
38. Reverse a goods movement
39. Alter the price details after goods are received or issued
40. Allocate an item to a bin to which the item category is not mapped
41. Alter UOM of items after some time frame
42. View item bin layouts in graphical mode
43. Manage vehicle data that load items to inventory thru goods receipt
44. Manage vehicle data that unload items from inventory thru goods issue
45. Set up special storage conditions for items and bins such as temperature, etc.
46. Manage storage conditions data
47. Review alert contents when storage criteria thresholds are breached
48. Check the contents of auto PO generation based on item threshold level
49. Remove items from bin as scrap or expired
50. Accept rejected items after rework by supplier
51. Assign bar codes to items and lots
52. Record excess items
53. Record shortage items
54. Load product images from time to time
55. Create multiple price list for same item
56. Mark items automatically as dead-stock based on time period
57. Run batch process to re-shuffle items to most used bins
58. Create records of damages inside stores
59. Return items back to stores
60. Approve or reject item returns
61. Create pest control schedule
62. Feed pest control data
63. Enter or detect stores close or open records by employees
64. Enter preventive maintenance records such as fire extinguisher details etc.
65. Enter store safety compliance records such as exhaust fans, drains, ducts etc.
66. Feed standard costing data for items
67. Feed actual costing data for items from orders in real time
68. Define shortest routes within stores or warehouses to move items
69. Enter vehicle details for within the warehouse goods movement
70. Enter trip details of vehicles within the warehouse goods movements
71. Enter zone configurations for loading, unloading
72. Import item or BOM details from csv files or spreadsheets
73. Print goods receipts, issue notes, return notes, adjustment notes
74. Print dispatch labels to different stores
75. Read data from bar code scanners, check duplicates
76. Read item data from RFID sensor interface, check duplicates
77. Read container and vehicle data from RFID sensor interface
78. Archive data based on aging criteria of records
79. Suspend receipt of specific items, look for alerts
80. Suspend issue of specific items, look for alerts
81. Suspend movement of specific items, look for alerts
82. Revoke suspension of receipts of specific items
83. Revoke suspension of issue of specific items
84. Revoke suspension of movement of specific items
85. Configure alerts on supplier data such as rejection, invoice issues etc.
86. Maintain legal certifications for items for receipts or issues
87. Freeze particular item and make it absolete
88. Merge data of two different stores in batch mode
89. Check interfaces to finance, purchase and CRM

90. Generate inventory list based on categories
91. Generate stock register with cost and filters
92. Generate reorder report
93. Generate physical inventory reconciliation report
94. Generate discrepancy reports between invoice and GRN
95. Generate discrepancy reports between GRN and PO
96. Generate goods rejection patterns chart based on reason for rejection
97. Generate consolidated view of many storage areas of items
98. Generate vehicles based report of inventory movements
99. Generate stock ageing report
100. Generate goods damage report

Hope you see some value in this list.

If you want to listen to recorded content on testing field, please visit http://www.openmentor.net

Thanks.