Monday, August 27, 2012

Test environment (test bed) setup - some important points

The environment on which we carry out testing, is the very first challenge for every testing team. My mentor always says, if our hands are not clean before taking food, we are prone to illness. The environment (also known as Test Bed) plays a vital part during test execution stage. Due to delivery deadlines, delay in hardware readiness, etc., people hurry up in setting up test bed and at a later point, it becomes the finger-pointing item in any bug triage meetings. Many to most of the times, the application may display a misleading message and people will start troubleshooting in one direction; but the actual problem will be at the test bed level. 

To preempt such issues, certain rules we must follow,, every time we start testing.

Rule 1. Test bed must not have development studio or IDE or other tools, except the product binaries and configuration files. If we have Eclipse or NetBeans in a java product test bed, that itself will resolve many conflicts and the product may tend to work fine. But in reality, in production environment, no dev tools will be present. Hence we must ensure that test bed resembles the production software environment to the extent possible.

Rule 2. Check the versions of OS along with all service packs, patches and hot-fixes.  Many times, people just check the OS version and start executing tests. The service packs and hot fixes do have a lot of compatibility and security related dependencies.

Rule 3. Check the webserver properties. Usually the configuration xml files will have memory settings, connection settings, database connectivity settings etc. Unless they are correctly configured, those may start showing up at different paths of the application.

Rule 4. Check database properties.  Things such as case sensitivity, connections, connection pooling settings, unicode settings etc will have a definite impact on the application behavior.

Rule 5. Check the free disk space, memory space and log file space. These items may not immediately cause issues in functional testing, but do have an effect on performance of the system. 

Rule 6. Check the system users who will run the application. This is mostly for rich-client applications. But if the application login is obtained from the LDAP settings, for all applications, the security settings of the logged-in user is key for carrying out certain activities at application and system levels.


Rule 7. Check the background processes/services. These services must be brought up, before others access the application. See if these are up and running without any exceptions.  Adequate configurations must be done to restart these if these fail to run during test execution.

Rule 8. Check the connectivity of the system with server and other machines. Try to ping or connect to the machines that need connectivity. Check the proxies and firewalls as well.

Rule 9. Check your antivirus settings. Bad settings may prevent blocking of requests or unwanted quarantine effects on your application data. This may cause connection or even timeout issues.

Rule 10. Check your browser version  and settings. Especially when you have auto upgrade option ON in your browser, you may not be able to track what happened to your browser since the last access, and silently an upgrade might have happened. That latest version may not be in scope of testing at all.

For  free learning content, visit us at http://www.openmentor.net

Tuesday, August 21, 2012

Review before you execute tests

I have been using English as the official language to communicate for the last 20 years. When I write a paragraph, and if I read the same once again, I see some   issues and I correct those. This happens every time. In simple terms, I can find out at least 2-3 issues in a single page that I write. When I see the text written by others, I do see issues that need correction. This is true whether I write my personal diary or letter or short story or test cases. Every document needs proof reading by the author and by at least one other person. 

When we write test cases, we think and start writing; the speed of thinking varies from speed of typing. The ability to choose the right word at the right place need not be consistent always. When we change one particular sentence    in a paragraph, that tone of that may be slightly different from that of others. Hence our human limitations of expressing what we think, in terms of an understandable text by other people, lead to issues in documents. Hence we must have a review done by other people.

Any document in any phase of SDLC, must go thru review. This is also termed as peer review, as your peers will do the review of the document. Reviews can be formal or informal, but must be effective. Usually the author of the document must coordinate to arrange a review. If you write test cases, check with a few other members in the project, fix up a time (30-60 minutes), fix a meeting room for review. Send the document that needs to be reviewed, to the invitees at least 12 hours before. They get time to glance thru it before they come for review.

Usually a formal review will have one senior person as moderator; usually, the project manager or business analyst or test manager will play that role. The author will give a brief introduction on the document to be reviewed. Let us assume we all review a test case document. Then everyone must start reading thru test cases one by one; take one, read it thoroughly, raise any concerns if it is not clear to you. Many people do not talk much during review even if they see issues in the test cases. This is not right and we are not doing any value add if we do not raise concerns. 

Keep these in mind while reviewing.
  1. Is the test case clear? Is it simple to understand?
  2. Are there any grammar or spelling issues?
  3. Is this test case relevant to a requirement? 
  4. Is this test case similar to another test case or is it a duplicate of another?
  5. Are the sentences written in proper format / tense? (e.g. test steps cannot be in past tense)
If any one raises any concerns, author must note down the test case id, the issue raised, the person who raised the same. If it is a valid concern, then the author has the responsibility to correct the same. Correction may happen at a later time. When one test case is cleared, let all reviewers move to next one and review that, raise any concerns. This process repeats for all test cases. At the end, the author will collate all the review findings/comments into a single sheet and circulate to all. This must be used as a reference for correction.

Sometimes, the concerns raised by a reviewer may end up in a heated argument between the author and the reviewer. At that time, the moderator needs to resolve it with logical explanation by author. If this is not resolved, it may trigger ego clashes among people.

After the review, the author must make all those corrections and send the document to all or selected invitees, to ensure that the corrections are made without any further issues. If the number of issues raised during review go beyond certain limits, the moderator may call for another review session as well. Once the corrections are made, the document will go for further approval by project manager or test manager.

There are different review methods adopted by companies. Some of the common approaches are given below.
  1. Walkthru - this is the formal approach we explained above. This is very effective, needs coordination within project
  2. Inspection - this is usually a review conducted by people from other projects.  This is very effective, needs coordination across projects
  3. Desk checking - this is quick review, by just one person, directly coming to your desk and reviewing the document. This is less effective, but needs less coordination
If an organization has a strong reviewing team, there is no doubt that the organization will be strong in delivering products as well. 

For free online training programs, visit us at http://www.openmentor.net





Monday, August 6, 2012

The CRUD Approach

With the knowledge of "what" and "how" to test, let us go one more step forward in testing. Majority of the tests fall under the category of functionality. Functionality is nothing but the way the software responds to its inputs based on the programmed logic. The functionality itself can be broken to smaller pieces to make it simple. Here is a CRUDe way to do it.


Let us take a simple social networking application. You can create a new account, edit your profile, add a friend, delete a friend, add comments, delete comments, reload your photograph, view the comments by others etc. Take an instant messenger like Skype. You can add a contact, rename the contact, edit your text, delete a contact etc. Look at a documenting tool like MS Word. You can create a document, modify its contents, rename it, delete some portions of it etc.


So, what do you see as a common set of activities in all the above? Add something, edit or modify it, read or view it, delete it. In other words, Create Read Update Delete and that is CRUD. There is no exception to this in software. Every software will have to necessarily support these features; so if we chase these, we can test a software in a structural manner. Once you read the specifications of the software, see whether that requirement falls under create or read or update or delete. Once you identify that, you can blindly apply the following test conditions to that particular requirement.


If the requirement falls under Create (also termed as New, Add, Insert) category, you need to define test conditions for:

  1. Creating a brand new item one at a time
  2. Creating a duplicate item
  3. Creating more than one items in a single shot (if applicable)
If the requirement falls under Read (also termed as View, Select, Open) category, you need to define test conditions for:

  1. Reading an existing item one at a time
  2. Reading more than one existing items at a time
  3. Reading a non-existing item
  4. Reading in a sorted order (such as sort a grid in ascending or descending order)
  5. Read an item based on wild cards (such as search or filter items)


If the requirement falls under Update (also termed as Edit, Modify, Change) category, you need to define test conditions for:

  1. Update an existing item one at a time
  2. Update more than one existing items 
  3. Update a non-existing item 
If the requirement falls under Delete (also termed as Remove, Trash, Erase) category, you need to define test conditions for:

  1. Delete an existing item one at a time
  2. Delete more than one existing items 
  3. Delete a non-existing item 
So, if we apply this CRUD method to every single requirement, we start getting more and more branches for every such operation. 

What is the guarantee that most of the software operations are restricted to only this CRUD concept? If you look at any database, you can see just four important operations - SELECT, INSERT, UPDATE and DELETE. Whether it is Oracle or MySQL or MS SQL or DB2, all use these 4 operations day in and day out. And ultimately every software must deal with a database  to finally store the data. Hope this is quite a convincing argument.

For more in testing, please visit http://www.openmentor.net.