Thursday, December 13, 2012

Bug Life Cycle


Like a butterfly going from egg to larva to pupa to a fully-grown butterfly,  the bug also has a life cycle. Each bug should go through the life cycle to reach a logical end. A specific life cycle ensures that the process is standardized. A life cycle is nothing but taking one thru a set of stages, as time passes by. If you take a civil lawsuit, the case goes thru a set of stages such as case filed, case hearing done, sessions court verdict given, applied to high court etc. Treat a bug as a case filed by the testing team and the project manager needs to see all cases are closed before release.

Typical stage or status codes are New, Open, Fixed (or Resolved), Retested, Closed. The codes may slightly vary from company to company, but these are the status codes that are universally accepted.

New: When the bug is posted for the first time, its state will be “NEW”. This means that the bug is not yet approved.

Open: After a tester has posted a bug, the lead approves that the bug is genuine and he/she changes the state as “OPEN”. This also means that the development team has acknowledged the bug. 

Fixed (Resolved): Once the developer fixes the bug, he has to change the status as Fixed and assign the bug to the testing team for retesting. Developer must also provide what kind of resolution he has done to fix the bug. 

Retested: Once the bug is fixed, the tester retests the bug in the subsequent build or in a hot-fix. If the bug is not present in the software, he sets the status to “Retested”.  

Closed: When the bug is successfully retested and the bug is not found any more, usually the test lead or manager closes the bug. This state means that the bug is fixed, retested and approve and no more exists.

Apart from the above regular stage/status codes, there are additional codes used to exactly position the bug's status. 

Reopened: If the bug still exists during retesting even after the bug is fixed by the developer, the tester changes the status to “Reopened”. The bug traverses the life cycle once again.Reopen is on the same line with Open.
 Deferred: This means the bug is expected to be fixed in next releases and not immediately. The reasons for changing the bug to this state have many factors. Some of them are priority of the bug may be low, lack of time for the release or the bug may not have major effect on the software. 
 
Rejected: If the developer feels that the bug is not genuine, he rejects the bug. 

Closed-Duplicate:If the bug is already logged by same tester or another tester in a different context, this can happen. so every tester must do a simple search in the bug tracking system whether a similar bug exists or not.

Closed-Irreproducible: Sometimes, the bug may happen due to a strange environment or data problem. The development team may not be able to exactly reproduce the bug and hence they cannot fix the same. In such a case, the bug may land up in this stage.

Closed-By-Design: When test cases were not exactly matching the specs or design, the understanding of the tester may be wrong. In such case, the bug will be closed in this manner.



Typical disputes on bugs

When someone says that you are wrong will you accept on the first shot or will you try to defend? This is typically the human behavior. More than that, if I come and tell you that your son or daughter damaged my car glass, will you defend or not?  More than defending self, one will defend more when it comes to one’s son or daughter. Same way, when a tester finds an issue on a program, many programmers will defend their program, as if it were their kids !  Quite natural. But in a professional environment this has to be handled carefully. This is usually handled in triage meetings. This section discusses the soft issues that occur due to these disputes.

This is not a bug:
 
The first resistance will be to say that it is not a bug. When this comes, we need to remove our emotions out and look at the requirements and design. As long as the product does not work as per requirements and design, it is a bug. But in some cases, the documents may be wrong or old; in that case, assign the bug to the business analyst or systems analyst to modify the requirements and/or design documents.

Over-testing of one module:

If a tester posts a bug on a module and that is overruled in triage meeting, the tester will defend the bug found. When his/her kid is ignored, when the next build comes for testing, the tester will focus more on that module irrespective of any project priorities and bring out more issues. If they are genuine, it is good. But most of the times, the issues are brought out for the sake of it with a human dislike.

If this is sensed by the Test Lead in advance, he can roll the tester to another module till time heals the attitude of the tester.

The developer pinch:

When the developer is flooded with bugs, he/she may try to talk to the tester and will show the technical superiority. It may go to the extent that the developer telling the tester “what do you know in java ? we create and you do only the testing….”. But, remember, the tester knows what customer wants. So there may be a rift between the developer and tester. If this is identified, we must ensure that the team members do not interact directly except triage meetings. Many companies put these two teams in two different buildings !

Blame it on requirements:

When more bugs are found, and development lead and test lead not getting well, both will start blaming the ambiguity and gaps in requirements. This affects project progress. The position of the business analyst will be very difficult. In this situation, the manager must stress to stick on to what is stated in specifications – nothing more, nothing less. First do, what is stated and then resolve gaps.

Push to Deliver:

When project manager overrules the results published by testing team and approves delivery to customer, the testing team will feel hurt. When the customer sends the product back pointing out the issues, again the same blame will come on the testing team only. To resolve this, we must publish delivery with known issues list to customer. This will give a better feel to testing team.

For further details, please visit http://www.freeqamonitor.com.

Monday, November 19, 2012

Tester's friend - The BUG

Every normal human being gets pleasure while finding mistakes in others, sad, but true. Tester is also a human being and when he/she finds a bug, it further motivates him/her to find more. A tester aims to fail the software in any way possible, and tester succeeds when bugs do appear. I do not worry about the theoretical definition of bug; to me, anything that the software does in a manner that is not intended as per specs or by implied needs, is a bug. Hence bug, error, problem, issue, failure etc. all come under the same category, though you can find zillion definitions for these words if you google.

When a bug appears while you test, do the test again. Ensure it appears again. This will prove that, you as a tester, had not done any mistake in executing the test. Soon after confirming its consistent nature, do not call the developer. Developer has a tendency to reject or dispute. Do not send a mail either. You need to ask a few questions to yourselves before you formally state that it is a bug. 

Try to describe the bug in 1 line. This is like writing the email subject. "User does not get confirmation email after registration", "Application accepts invalid SSN in Add Customer Page". Try to convey the exact problem with minimal words.

Mention the screen name where the bug appears. This will help the developer directly to go to that place, without wasting time. If possible, take a screenshot where the bug appears. A picture means more than 1000 words.

Explain the bug in detail, by telling the steps to reproduce. You get bugs when you execute a test case. Thus, the bug must be appearing against one of the test steps. Copy paste the test case itself and mention the exact problem against the test step. If the developer does the same thing, he/she should get the same bug.

How does this bug affect the product? The answer to this question will help you to set the Severity of the bug. Are you able to proceed further to do other test cases? If not, this bug is termed as a Show Stopper or Blocker. This means, you have no way to test anymore unless development team fixes the same. 

Is there a crash or data loss or a feature is missing in the product? This falls under High or Major category, provided you are able to do other tests. This means, that the product has a serious problem. For example, when we page customer details, Save operation itself does now work and hence the data entered is lost. Technically a developer may say that he can fix this in 2 minutes, but always look from customer's angle. For the customer, data is more important than the product itself. Any loss of data is a major bug. The severity code Major is also termed as High in many companies.


Bug does not stop me from testing, there is no data loss or crash, but the functionality is wrongly implemented. The app saves data, but some part of the results are not accurate. In this case, companies usually treat this as Minor or Medium severity.


The app does the job, but there are spelling mistakes or improper messages to users or imporper field alignments in the UI. Usually companies treat this as Cosmetic or Low severity.

To sum up, if we can write the following, then we are good in reporting bugs to dev team.
  • One line description of bug
  • Details to reproduce the bug
  • Screen name where the bug appears
  • Screenshot of the bug
  • Severity of the bug
By principle, every problem must be reported and tracked. We will see tracking mechanism in the next post. For more details on bugs, please see this bug tracking demo.




Friday, September 14, 2012

Test Execution - How can we do faster?

Development deadlines always slip, but release dates do not change; caught between these 2, the testing time is always compromised. Hence the testing teams need to work faster. If the application has withstood the first one hour of critical tests, it indicates the basic stability of the app. But every test is important from functional point of view. Here are some simple techniques to make it faster.

Testing teams may have a mix of people with different experience levels. It is very uncommon to see a testing team with uniform experience level. The first thing we need to do is to assign navigation and field level validation tests to the junior-most members of the team. The number of tests will be more, but they are relatively simpler to execute. For every screen, let them do the same, and in one or  two rounds they will get familiarized with the app.

Assign the form level tests to mid-level experienced testers. This will involve submitting a variety of data thru forms and trying add, modify, delete, view test cases. 

Assign business flow tests to seniors and leads. They need to wear the cap of the end users and move data across screens to achieve the business flow. These people are the best bet to test the reports. The senior people must start feeling the semantics behind the data that flow across the application and understand the business value of the flows. If this is not done, end user testing will always find finer problems and that may lead to escalation.

Always people feel happy to execute new tests (progression tests, for new features added or for change requests). But by principle, a fix or new feature introduction may create issues at some other place. So we try to cover those areas by doing regression tests. The first set of regression tests must be performed for the form in which  the new features/changes are introduced. The next set of regression tests must be done at the module to which the form belongs. The next set of regression tests must address the path in which the feature or change is introduced. Once these are covered, then only do other regression tests. This will ensure the the impact of the changes are minimal.

For every round of testing, reallocate people. Doing the same test again and again will make them slow. You cannot reallocate a person to a totally unrelated module. Every application has a flow of information from one module to another; e.g. request spare item in spares module, cater the service in service module, raise invoice in accounts module. If tester A has done testing of round 1 for spares module, for round 2, assign tester A to service module and in the 3rd round assign to accounts module. This way, the tester also logically moves to the next step rather than landing on a dark area.

Last but not the least. Ask every tester to stand up and do a simple neck and shoulder exercise for 20-30 seconds every hour. Remember, you need good blood flow towards head. When you are too attentive for hours, doing testing, with eyes locked to the monitor, you will develop neck and shoulder pains. You must be physically good to perform better. So take care of your body well.

For free learning courses, please visit http://www.openmentor.net

Tuesday, September 4, 2012

The first hour of testing

When a person suffers from heart attack or stroke, the first one hour is crucial. If we can treat the person in that first hour, there is about 90% chance that the heart or brain reacts positively to the medicines. This is true in testing also. The first one hour of testing can bring out a great insight about the health of the application being tested. To achieve best results, we have to execute the tests in a structured manner. Here is how.

At this point we assume that the test bed is ready and the application is already installed and ready to be tested. We may have 100s or even 1000s of tests documented and reviewed. Before test execution, ask this question on every test case - "If this test fails, is there any point to continue testing?". Mark all those test cases for which you get an answer "No". That means, those are the vital test cases that must necessarily work or pass. Let us take an example of testing a car. To test a car, we may have to do 1000s of tests; but if ignition does not work, if door does not open, if clutch does not work, if brakes do not work, if accelerator or gas pedal does not work, if steering does not work, is it worth saying that as a car? In your software testing, identify those test cases before test execution.

In the first 5 minutes, navigate to all screens in the application. Simply check the loading of forms and the look and feel. End users do care about screens more and more nowadays.

After this, in every form, if there is a provision to add or create new records, perform those. If no data is added, no data can be viewed; hence do add data. Add with valid data. negative data conditions can be done a little later. Ensure all these tests work well. In every screen try to view all existing data. If you see missing data on screen, we have a problem.

Then, execute critical workflows with valid data. One of the flows in a retail banking scenario may be, create an account, perform a few deposit transactions, perform a few withdrawal transactions, view account balance and transfer some amount from checking to savings account. The flow touches a set of screens with connected data. Usually every application will have top-10 workflows or top-20 workflows. Perform those meticulously. 

Generate a set of key reports and charts that the management people would view on the application on a daily basis. These are the heartbeats of the application to the customer's management; one wrong data in that, the management will lose the confidence in the product.

If we can pack these in the first 1 hour of testing, we can definitely see whether the application is ready to do what it is supposed to do, i.e. it works for positive data. All other invalid data validations, invalid flow tests. non-core features can be tested after the first hour. We call this as BVT (Build Verification Tests). No failure can happen in BVT. Only when all these tests pass, we ask our testing teams to proceed further. You may still ask a question, brakes in car not working does not mean that the horn may not work. I agree. But, if a critical part or path is not working, there is no point in testing less critical part. So let us not waste time and this will ensure that the build or dev teams to ensure they pay proper attention to the paths of the application according to their importance. 

Usually 10-15% of the total tests will fall under BVT category in every application. Attack those first.

For free learning content, visit us at http://www.openmentor.net

Monday, August 27, 2012

Test environment (test bed) setup - some important points

The environment on which we carry out testing, is the very first challenge for every testing team. My mentor always says, if our hands are not clean before taking food, we are prone to illness. The environment (also known as Test Bed) plays a vital part during test execution stage. Due to delivery deadlines, delay in hardware readiness, etc., people hurry up in setting up test bed and at a later point, it becomes the finger-pointing item in any bug triage meetings. Many to most of the times, the application may display a misleading message and people will start troubleshooting in one direction; but the actual problem will be at the test bed level. 

To preempt such issues, certain rules we must follow,, every time we start testing.

Rule 1. Test bed must not have development studio or IDE or other tools, except the product binaries and configuration files. If we have Eclipse or NetBeans in a java product test bed, that itself will resolve many conflicts and the product may tend to work fine. But in reality, in production environment, no dev tools will be present. Hence we must ensure that test bed resembles the production software environment to the extent possible.

Rule 2. Check the versions of OS along with all service packs, patches and hot-fixes.  Many times, people just check the OS version and start executing tests. The service packs and hot fixes do have a lot of compatibility and security related dependencies.

Rule 3. Check the webserver properties. Usually the configuration xml files will have memory settings, connection settings, database connectivity settings etc. Unless they are correctly configured, those may start showing up at different paths of the application.

Rule 4. Check database properties.  Things such as case sensitivity, connections, connection pooling settings, unicode settings etc will have a definite impact on the application behavior.

Rule 5. Check the free disk space, memory space and log file space. These items may not immediately cause issues in functional testing, but do have an effect on performance of the system. 

Rule 6. Check the system users who will run the application. This is mostly for rich-client applications. But if the application login is obtained from the LDAP settings, for all applications, the security settings of the logged-in user is key for carrying out certain activities at application and system levels.


Rule 7. Check the background processes/services. These services must be brought up, before others access the application. See if these are up and running without any exceptions.  Adequate configurations must be done to restart these if these fail to run during test execution.

Rule 8. Check the connectivity of the system with server and other machines. Try to ping or connect to the machines that need connectivity. Check the proxies and firewalls as well.

Rule 9. Check your antivirus settings. Bad settings may prevent blocking of requests or unwanted quarantine effects on your application data. This may cause connection or even timeout issues.

Rule 10. Check your browser version  and settings. Especially when you have auto upgrade option ON in your browser, you may not be able to track what happened to your browser since the last access, and silently an upgrade might have happened. That latest version may not be in scope of testing at all.

For  free learning content, visit us at http://www.openmentor.net

Tuesday, August 21, 2012

Review before you execute tests

I have been using English as the official language to communicate for the last 20 years. When I write a paragraph, and if I read the same once again, I see some   issues and I correct those. This happens every time. In simple terms, I can find out at least 2-3 issues in a single page that I write. When I see the text written by others, I do see issues that need correction. This is true whether I write my personal diary or letter or short story or test cases. Every document needs proof reading by the author and by at least one other person. 

When we write test cases, we think and start writing; the speed of thinking varies from speed of typing. The ability to choose the right word at the right place need not be consistent always. When we change one particular sentence    in a paragraph, that tone of that may be slightly different from that of others. Hence our human limitations of expressing what we think, in terms of an understandable text by other people, lead to issues in documents. Hence we must have a review done by other people.

Any document in any phase of SDLC, must go thru review. This is also termed as peer review, as your peers will do the review of the document. Reviews can be formal or informal, but must be effective. Usually the author of the document must coordinate to arrange a review. If you write test cases, check with a few other members in the project, fix up a time (30-60 minutes), fix a meeting room for review. Send the document that needs to be reviewed, to the invitees at least 12 hours before. They get time to glance thru it before they come for review.

Usually a formal review will have one senior person as moderator; usually, the project manager or business analyst or test manager will play that role. The author will give a brief introduction on the document to be reviewed. Let us assume we all review a test case document. Then everyone must start reading thru test cases one by one; take one, read it thoroughly, raise any concerns if it is not clear to you. Many people do not talk much during review even if they see issues in the test cases. This is not right and we are not doing any value add if we do not raise concerns. 

Keep these in mind while reviewing.
  1. Is the test case clear? Is it simple to understand?
  2. Are there any grammar or spelling issues?
  3. Is this test case relevant to a requirement? 
  4. Is this test case similar to another test case or is it a duplicate of another?
  5. Are the sentences written in proper format / tense? (e.g. test steps cannot be in past tense)
If any one raises any concerns, author must note down the test case id, the issue raised, the person who raised the same. If it is a valid concern, then the author has the responsibility to correct the same. Correction may happen at a later time. When one test case is cleared, let all reviewers move to next one and review that, raise any concerns. This process repeats for all test cases. At the end, the author will collate all the review findings/comments into a single sheet and circulate to all. This must be used as a reference for correction.

Sometimes, the concerns raised by a reviewer may end up in a heated argument between the author and the reviewer. At that time, the moderator needs to resolve it with logical explanation by author. If this is not resolved, it may trigger ego clashes among people.

After the review, the author must make all those corrections and send the document to all or selected invitees, to ensure that the corrections are made without any further issues. If the number of issues raised during review go beyond certain limits, the moderator may call for another review session as well. Once the corrections are made, the document will go for further approval by project manager or test manager.

There are different review methods adopted by companies. Some of the common approaches are given below.
  1. Walkthru - this is the formal approach we explained above. This is very effective, needs coordination within project
  2. Inspection - this is usually a review conducted by people from other projects.  This is very effective, needs coordination across projects
  3. Desk checking - this is quick review, by just one person, directly coming to your desk and reviewing the document. This is less effective, but needs less coordination
If an organization has a strong reviewing team, there is no doubt that the organization will be strong in delivering products as well. 

For free online training programs, visit us at http://www.openmentor.net





Monday, August 6, 2012

The CRUD Approach

With the knowledge of "what" and "how" to test, let us go one more step forward in testing. Majority of the tests fall under the category of functionality. Functionality is nothing but the way the software responds to its inputs based on the programmed logic. The functionality itself can be broken to smaller pieces to make it simple. Here is a CRUDe way to do it.


Let us take a simple social networking application. You can create a new account, edit your profile, add a friend, delete a friend, add comments, delete comments, reload your photograph, view the comments by others etc. Take an instant messenger like Skype. You can add a contact, rename the contact, edit your text, delete a contact etc. Look at a documenting tool like MS Word. You can create a document, modify its contents, rename it, delete some portions of it etc.


So, what do you see as a common set of activities in all the above? Add something, edit or modify it, read or view it, delete it. In other words, Create Read Update Delete and that is CRUD. There is no exception to this in software. Every software will have to necessarily support these features; so if we chase these, we can test a software in a structural manner. Once you read the specifications of the software, see whether that requirement falls under create or read or update or delete. Once you identify that, you can blindly apply the following test conditions to that particular requirement.


If the requirement falls under Create (also termed as New, Add, Insert) category, you need to define test conditions for:

  1. Creating a brand new item one at a time
  2. Creating a duplicate item
  3. Creating more than one items in a single shot (if applicable)
If the requirement falls under Read (also termed as View, Select, Open) category, you need to define test conditions for:

  1. Reading an existing item one at a time
  2. Reading more than one existing items at a time
  3. Reading a non-existing item
  4. Reading in a sorted order (such as sort a grid in ascending or descending order)
  5. Read an item based on wild cards (such as search or filter items)


If the requirement falls under Update (also termed as Edit, Modify, Change) category, you need to define test conditions for:

  1. Update an existing item one at a time
  2. Update more than one existing items 
  3. Update a non-existing item 
If the requirement falls under Delete (also termed as Remove, Trash, Erase) category, you need to define test conditions for:

  1. Delete an existing item one at a time
  2. Delete more than one existing items 
  3. Delete a non-existing item 
So, if we apply this CRUD method to every single requirement, we start getting more and more branches for every such operation. 

What is the guarantee that most of the software operations are restricted to only this CRUD concept? If you look at any database, you can see just four important operations - SELECT, INSERT, UPDATE and DELETE. Whether it is Oracle or MySQL or MS SQL or DB2, all use these 4 operations day in and day out. And ultimately every software must deal with a database  to finally store the data. Hope this is quite a convincing argument.

For more in testing, please visit http://www.openmentor.net.




Monday, May 28, 2012

Equivalence Partitions

When we deal with a lot of data, it seems to be ever increasing to test and exhaust. Let us take a simple courier management application. Typical inputs are - from address, to address, weight of the packet to be couriered, any insurance to be used for the parcel. Based on these, the price is calculated by the system. Imagine you are the tester for this application, what kind of test conditions you will write?

Let us divide the problem into different segments. See the conditions below.
  1. Sending a packet from your home to another house in the same street
  2. Sending a packet from your home to another house in same zip code or area code
  3. Sending a packet from your home to another house in another city, but within the same state
  4. Sending a packet from your home to another house in another city, but outside the state
  5. Sending a packet from your home to another house in another city, but outside the country
Do you see a pattern? We try to get a sample of each business scenario that can happen in real time. One sample is a representative of a lot of similar data. The trick is to identify as many number of good samples, as possible. Another set of samples would be based on the weight of the packet. Usually the charges would be x1 for 10-100 grams,  x2 for 101-250 grams, x3 for 251-500 grams, x4 for 501-1000 grams and so on. If we need to get samples for these different slabs of the packets, what values should I choose? Should I take 4 packets with weights 10, 101, 251, 501 grams each or 100, 250, 500, 1000 grams each? There is no hard an fast rule here - but the boundary values will always be taken care in your BVA test conditions. When it comes to Equivalence Partition, it is advised to take the middle values when it comes to numbers. So if I need to give 4 samples based on weight of the packet, I would choose 50, 175, 375, 750 grams each. These 4 samples clearly represent the 4 slabs of the packet weights.

Let us take a bank that takes deposits. There are many parameters that determine the interests on deposits - but let me take 2 key parameters - Interest Type - Compound or Simple,  and the other one is Deposit Period Tag - Daily, Monthly, Yearly. If we alter these 2 parameters, the interest varies. Look at the following options:
  1. Simple interest, applied on Daily basis
  2. Simple interest, applied on Monthly basis
  3. Simple interest, applied on Yearly basis
  4. Compound interest, applied on Daily basis
  5. Compound interest, applied on Monthly basis
  6. Compound interest, applied on Yearly basis
If we carefully look at the above samples, 2 parameters with 2 and 3 different values, resulted in 6 combination samples. Thus, when the number of parameters and values of the parameters keep increasing, the number of samples you have to choose will also keep increasing.


Exercise: Take a travel booking site. From city, to city, round trip/one way, adults/children count, accommodation required etc. will appear on the screen as input parameters. Arrive at the combinations of these as test samples based on equivalence partition technique.


For recorded sessions on software testing, visit us at www.openmentor.net.



Monday, May 7, 2012

Boundary Value Analysis (BVA)

By pushing one to the limits, you can see a totally different behavior from that person. Stretching a system to extremes will bring out its funny behavior. Usually this is applicable to all fields on the user interface. Always remember, most of the problems to our body happens due to bad input, name it bad food, or bad water or bad air. Same way, a system if it accepts bad inputs, it is in potential danger zone.


Let us take a simple voltage stabilizer that we connect to TV or fridge. Usually they give the range 190-240 volts 50 to 60 hertz input, output will be constant at 220 volts. If this needs to be tested, we need to look at some good and bad extremes.


  1. Connect to 190 volts
  2. Connect to 240 volts
  3. Connect to 189 volts
  4. Connect to 241 volts
In the above, we arrived at 4 conditions - be at the lowest boundary, be at the highest boundary, go below the lowest boundary and go beyond the highest boundary. We just varied one parameter - voltage. 


But there is one more parameter we have - cycles in Hertz. For that, keep a valid voltage and push that cycles to extremes.
  1. Connect to 220 volts 50 Hz
  2. Connect to 220 volts 60 Hz
  3. Connect to 220 volts 49 Hz
  4. Connect to 220 volts 61 Hz
By doing this, we make the system to go thru positive boundaries and negative boundaries. Withing the positive boundaries, the system must work; the system must stop working when it is put under negative boundaries.

Better to apply one parameter and push that to extreme, when we deal with multiple parameters. Else if the system fails, we may not be able to isolate the reason of failure.

In software, imagine you see a field that takes the date of journey in a travel portal. How can we apply the BVA technique in that field? Ask these questions and take a call.

  1. Can I book a ticket for today?
  2. Can I book a ticket for yesterday?
  3. Can I book a ticket for a day 180 days from now? (usually travel sites may allow upto 6 months)
  4. Can I book a ticket for a day 181 days from now?

This list may go as per the restrictions given by that portal. Same way, let us take a US zip code field. How can I apply BVA on a zip code field? A zip code is usually 5 digits such as 95014 or 5 digits hyphen 4 digits such as 95014-2367.


A few home work questions for you to apply BVA.

  1. Write BVA tests for USA social security number
  2. Write BVA tests for the interest rate field in a banking simple interest calculation screen. (interest rate is a percentage value)
When you deal with boundaries, there are 2 kinds of boundaries. Size boundary and value boundary. Value boundaries are the ones we discussed before. Size boundary indicate the number of characters that I can type in a field. You could have seen that many websites registration pages restrict the passwords to be minimum 6 characters and maximum 10 characters. In such cases, we need to test for 5 character password, 6 character password, 10 character password and 11 character password. 

Now is the twist - if a field has both size and value boundaries defined, you end up doing at least 8 tests for that one field alone! Good luck.

For more details, please refer to Open Mentor portal.



Monday, March 12, 2012

Writing Test Cases

After writing test conditions, this is the next logical item one must go thru in a project. A test condition tells "WHAT" to test. A test case tells "HOW" to test. That means, we must provide step by step details in test case. This is like a recipe - you miss one vital step, the execution of test may go wrong.

Simple mantra you must follow when you write test cases - think that you are giving instructions to test the software, to an 8th grade kid. The kid knows English and knows how to operate computer; but does not know anything on how to test your application.

Is it really possible to make a kid to test my application? Yes it is, provided the instructions are clear and simple. Have you ever called a customer support help desk to resolve a problem in your laptop? You might have observed that the help desk person asking a series of simple short questions and solving your problems in minutes. The trick is clear and simple document. Look at this following example.

  1. Open firefox browser.
  2. Navigate to www.openmentor.net
  3. Click on Word Games link.
  4. Click on the link that says "Click Here to start"
  5. Choose 4 from the list box shown.
  6. Click Go.
If we give instructions like this, don't you think that you can make another person to test a software? This is the holy trick of getting work done by vendors and that too relatively junior testers, thus reducing the cost. If we do not do this kind of clear step details, we may require senior testers, who are costly. Every management would like to reduce the cost. If that needs to happen, a good set of test cases must be drafted.

In the industry, a test case is written in a document or in a spreadsheet with the following details.

Test case Id: A unique identifier for this test case, across the project or company.
Test case name: Usually this will be the test condition that you wrote before.
Pre-condition: The pre-requisite that must exist before we do this test.
Test Steps: Simple step by step such as clicks, types, navigation etc to carry out the test.
Test Data: The data yo must choose or type in any forms.
Expected Results: What is the expected outcome when we do these steps with the given data.

Usually people ask me this question - why should we write this much, when I know how to do it? Remember, you or any other person in the project, may be moved to a different project tomorrow. In that case, should other person re-invent the wheel? So better put all the things you remember in testing this product, in a paper. Also, you be the thinker that adds more value; let the test execution be done by other juniors. It also helps in assessing whether we have missed any permutation or combination in testing. If everything is just remembered by you, no one knows how many tests you remember and how many you do not. So better to document every test case.

Writing test cases will take more time. Agreed. But it is usually for one set of test cases; after that, you copy-paste the same text and make modifications in steps or data and expected results. It is not usually all text in all test cases are typed fully by you. Once a pattern of test is identified and typed, rest is all making minor modifications to that text.

In the next posting, we will see examples of test cases and a few techniques like boundary value analysis etc.

Monday, January 16, 2012

What does one need to get job?

Hundreds of job hunters send emails to us asking whether we have any positions open. Every company needs people, all the time. Very rarely, they freeze hiring. But what the applicant does not know or does not care to know, is whether he/she is ready for that position. Everyone feels that they are ready to do the job; but every business leader claims that 80% of the students who pass out in India, are not readily employable. Hence there is a big fight to grab the qualifying ones, by so many companies - the result is the inflated salary levels.

Where and when will it end?

First thing. Whatever is the industry to which one applies for a job, that person must have a good analytical reasoning ability. If one is not able to solve a simple aptitude question, that person is not fit to get an entry. During interviews, we had seen people who do not know how to apply the fundamental principles of simple and compound interests, but applying to a banking position.

Second thing. Be confident. Almost 90% of the people who come for interview at fresh level are submissive. They feel like servants and are afraid of answering in clear loud tone. This attitude must change. Many times we see people pleading ta the end of the interview that they have family commitments and this job is an absolute necessity to them. This will not help.

Third. Communicate clearly in simple way. One need not be a great orator or writer. Simple words, clear sentences will make the interviewer happy. But unfortunately, people try to throw jargons and try to show off their knowledge. But when a practical question is thrown at them, they fail to apply concepts and start blabbering. This wastes a lot of time. Better communication can compensate many other negative points.

Fourth. Unreasonable Expectations. Everyone thinks that they are worth minimum 3 lacs per annum salary. They never care about the job profile or the size of the company. People do not even see the website of the company to which they had applied for a job. For the first 2-3 years, learning is the asset one must look for and not money.

Fifth. 100% job guarantee by training institutes. Do IIT and IIM claim that they can 100% guarantee job to every student? It must be knowledge and skill guarantee and not a job guarantee. Only yourself can guarantee a job for you, based on practice. Any institute that does good training and guidance, will help you; but it is ultimately you, who can make the difference. Lack of self belief is the root cause of this; always people need external factors to help them.

When 50,00,000 students graduate from 22000 arts colleges every year, and 10,00,000 students come out of 3000 engineering colleges every year, only the knowledge and practice can help every fresher. Nothing else.

In worst case scenario, if you do not have a job and someone says work is there but no pay, please take it in a short term at least. That is the intellectual money you can earn; that time will never come back, if you refuse that option also. Sorry, this may sound harsh, but this is reality in India.

Thursday, January 5, 2012

Top 100 Test Scenarios for Online Shopping Apps

I have been seeing a lot of ads for online shopping portals. What do they really do? Sell items right from shoes to iPhones. A few product companies asked us to educate their testers in testing their product. We first educated the testers in the business flow of ecom portals.

What you see here is a slice from the testing checklist.

  1. View the hot promotion plans on home page
  2. View top level categories of products on home page
  3. View the order of sub categories of products under each category
  4. View top products list under each category-sub category
  5. View products as grid
  6. View products as thumbnails
  7. View small and large images of products with short and long descriptions
  8. Search for a product with brand name
  9. Search for a product with a model name or number
  10. Search for a product below certain price
  11. Search for a product within a price range
  12. Search for a product with some keywords in description
  13. Search for a product of certain color
  14. Search for a product in specific store
  15. Search for a product in a specific geography
  16. Search for a product that are referred by many community people
  17. Search for a product for a range of user ratings
  18. Search for a product with multiple combinations of above
  19. Sort product search results based on price
  20. Sort product search results based on store
  21. Sort product search results based on buyer rating
  22. Add a product to cart
  23. Add multiple products to cart from same search results
  24. Add multiple products to cart from different search results
  25. Remove a product from cart
  26. Add the same product to cart after removing it
  27. Add the same product multiple times to cart
  28. Alter quantity of products after adding to cart
  29. Alter quantity to zero in the cart
  30. Provide payment details to buy product
  31. Provide payment details and cancel it
  32. Provide wrong credit card details and try to buy
  33. Purchase product thru corporate purchase plans
  34. Provide shipping address same as user’s registered address
  35. Provide shipping address different from the user’s registered address
  36. Register new user with valid details
  37. Register new user with invalid details
  38. Register same user with same emailed and userid
  39. Provide same credit card details to two different users
  40. View the registration email contents for new users
  41. Change password for a registered user
  42. View change password email content
  43. Do not use account for 60 days and login after that time period
  44. Login as user, add a product, logout and login again
  45. Use correct promotion code for correct product
  46. Use wrong promotion code for a product
  47. Use a valid promotion code to a product but after its expiry date
  48. Use a promotion code more than once for purchase
  49. User a private promotion code of another registered user
  50. Track the purchase order for delivery
  51. Rate a product for its quality
  52. Join a group purchase and confirm purchase
  53. Join a group purchase and cancel after an hour or day
  54. Join a group purchase for more than once
  55. Define alert to notify when product price falls into certain range
  56. Purchase a product whose price is more than your credit card limit
  57. Purchase a product when the product is out of stock
  58. View product details from a different country settings
  59. File a complaint as a buyer
  60. View complaint tracking email content
  61. View complaint status on the buyer portal
  62. View past order history for a user
  63. Modify the user profile after a few purchase transactions
  64. Add products in wish list
  65. Manage different lists as part of user profiles
  66. Comment on a product
  67. Comment on a purchase
  68. Provide feedback on a purchase
  69. Refer a product to another user in the community
  70. Use one time shipping plan for a purchase
  71. subscribe to annual shipping plans and do a purchase
  72. Subscribe to newsletters
  73. Unsubscribe to newsletters

  1. Freeze a user account
  2. Unfreeze a user account
  3. Set transaction limits for users
  4. Generate reports for today’s purchase
  5. Generate reports for purchase for specific products
  6. Generate reports for purchase in specific geographies
  7. Generate reports for shipping status
  8. Generate reports for customer complaints
  9. Generate reports on purchase feedback
  10. Upload stores details who act as suppliers
  11. Upload price details from various stores
  12. Upload products and price details from stores in different formats such as xml, csv, Excel etc.
  13. Upload products and price details from stores with erroneous records
  14. Upload product image contents with static images
  15. Upload product image contents with animated images
  16. Create survey to selected users
  17. Publish a poll to all users
  18. Create product promotion codes and publish to selective users or geography
  19. Manage shipping and handling charges and terms
  20. Manage group purchase settings and products
  21. Extend a promotion offer
  22. Configure prizes for buyers on specific purchase criteria such as 100000th buyer etc.
  23. Configure corporate purchase accounts
  24. Manage out of stock alert settings
  25. Manage different payment gateway settings
  26. Configure SMS alerts to users
  27. View whether purchase details reflect in the CRM thru synchronization

For recorded training programs and online tests, visit us at http://www.openmentor.net