Thursday, December 13, 2012
Monday, November 19, 2012
When a bug appears while you test, do the test again. Ensure it appears again. This will prove that, you as a tester, had not done any mistake in executing the test. Soon after confirming its consistent nature, do not call the developer. Developer has a tendency to reject or dispute. Do not send a mail either. You need to ask a few questions to yourselves before you formally state that it is a bug.
Try to describe the bug in 1 line. This is like writing the email subject. "User does not get confirmation email after registration", "Application accepts invalid SSN in Add Customer Page". Try to convey the exact problem with minimal words.
Mention the screen name where the bug appears. This will help the developer directly to go to that place, without wasting time. If possible, take a screenshot where the bug appears. A picture means more than 1000 words.
Explain the bug in detail, by telling the steps to reproduce. You get bugs when you execute a test case. Thus, the bug must be appearing against one of the test steps. Copy paste the test case itself and mention the exact problem against the test step. If the developer does the same thing, he/she should get the same bug.
How does this bug affect the product? The answer to this question will help you to set the Severity of the bug. Are you able to proceed further to do other test cases? If not, this bug is termed as a Show Stopper or Blocker. This means, you have no way to test anymore unless development team fixes the same.
Is there a crash or data loss or a feature is missing in the product? This falls under High or Major category, provided you are able to do other tests. This means, that the product has a serious problem. For example, when we page customer details, Save operation itself does now work and hence the data entered is lost. Technically a developer may say that he can fix this in 2 minutes, but always look from customer's angle. For the customer, data is more important than the product itself. Any loss of data is a major bug. The severity code Major is also termed as High in many companies.
Bug does not stop me from testing, there is no data loss or crash, but the functionality is wrongly implemented. The app saves data, but some part of the results are not accurate. In this case, companies usually treat this as Minor or Medium severity.
The app does the job, but there are spelling mistakes or improper messages to users or imporper field alignments in the UI. Usually companies treat this as Cosmetic or Low severity.
To sum up, if we can write the following, then we are good in reporting bugs to dev team.
- One line description of bug
- Details to reproduce the bug
- Screen name where the bug appears
- Screenshot of the bug
- Severity of the bug
Friday, September 14, 2012
Tuesday, September 4, 2012
Monday, August 27, 2012
To preempt such issues, certain rules we must follow,, every time we start testing.
Rule 1. Test bed must not have development studio or IDE or other tools, except the product binaries and configuration files. If we have Eclipse or NetBeans in a java product test bed, that itself will resolve many conflicts and the product may tend to work fine. But in reality, in production environment, no dev tools will be present. Hence we must ensure that test bed resembles the production software environment to the extent possible.
Rule 2. Check the versions of OS along with all service packs, patches and hot-fixes. Many times, people just check the OS version and start executing tests. The service packs and hot fixes do have a lot of compatibility and security related dependencies.
Rule 3. Check the webserver properties. Usually the configuration xml files will have memory settings, connection settings, database connectivity settings etc. Unless they are correctly configured, those may start showing up at different paths of the application.
Rule 4. Check database properties. Things such as case sensitivity, connections, connection pooling settings, unicode settings etc will have a definite impact on the application behavior.
Rule 5. Check the free disk space, memory space and log file space. These items may not immediately cause issues in functional testing, but do have an effect on performance of the system.
Rule 6. Check the system users who will run the application. This is mostly for rich-client applications. But if the application login is obtained from the LDAP settings, for all applications, the security settings of the logged-in user is key for carrying out certain activities at application and system levels.
Rule 7. Check the background processes/services. These services must be brought up, before others access the application. See if these are up and running without any exceptions. Adequate configurations must be done to restart these if these fail to run during test execution.
Rule 8. Check the connectivity of the system with server and other machines. Try to ping or connect to the machines that need connectivity. Check the proxies and firewalls as well.
Rule 9. Check your antivirus settings. Bad settings may prevent blocking of requests or unwanted quarantine effects on your application data. This may cause connection or even timeout issues.
Rule 10. Check your browser version and settings. Especially when you have auto upgrade option ON in your browser, you may not be able to track what happened to your browser since the last access, and silently an upgrade might have happened. That latest version may not be in scope of testing at all.
For free learning content, visit us at http://www.openmentor.net
Tuesday, August 21, 2012
- Is the test case clear? Is it simple to understand?
- Are there any grammar or spelling issues?
- Is this test case relevant to a requirement?
- Is this test case similar to another test case or is it a duplicate of another?
- Are the sentences written in proper format / tense? (e.g. test steps cannot be in past tense)
- Walkthru - this is the formal approach we explained above. This is very effective, needs coordination within project
- Inspection - this is usually a review conducted by people from other projects. This is very effective, needs coordination across projects
- Desk checking - this is quick review, by just one person, directly coming to your desk and reviewing the document. This is less effective, but needs less coordination
Monday, August 6, 2012
Let us take a simple social networking application. You can create a new account, edit your profile, add a friend, delete a friend, add comments, delete comments, reload your photograph, view the comments by others etc. Take an instant messenger like Skype. You can add a contact, rename the contact, edit your text, delete a contact etc. Look at a documenting tool like MS Word. You can create a document, modify its contents, rename it, delete some portions of it etc.
So, what do you see as a common set of activities in all the above? Add something, edit or modify it, read or view it, delete it. In other words, Create Read Update Delete and that is CRUD. There is no exception to this in software. Every software will have to necessarily support these features; so if we chase these, we can test a software in a structural manner. Once you read the specifications of the software, see whether that requirement falls under create or read or update or delete. Once you identify that, you can blindly apply the following test conditions to that particular requirement.
If the requirement falls under Create (also termed as New, Add, Insert) category, you need to define test conditions for:
- Creating a brand new item one at a time
- Creating a duplicate item
- Creating more than one items in a single shot (if applicable)
- Reading an existing item one at a time
- Reading more than one existing items at a time
- Reading a non-existing item
- Reading in a sorted order (such as sort a grid in ascending or descending order)
- Read an item based on wild cards (such as search or filter items)
If the requirement falls under Update (also termed as Edit, Modify, Change) category, you need to define test conditions for:
- Update an existing item one at a time
- Update more than one existing items
- Update a non-existing item
- Delete an existing item one at a time
- Delete more than one existing items
- Delete a non-existing item
Monday, May 28, 2012
Let us divide the problem into different segments. See the conditions below.
- Sending a packet from your home to another house in the same street
- Sending a packet from your home to another house in same zip code or area code
- Sending a packet from your home to another house in another city, but within the same state
- Sending a packet from your home to another house in another city, but outside the state
- Sending a packet from your home to another house in another city, but outside the country
Let us take a bank that takes deposits. There are many parameters that determine the interests on deposits - but let me take 2 key parameters - Interest Type - Compound or Simple, and the other one is Deposit Period Tag - Daily, Monthly, Yearly. If we alter these 2 parameters, the interest varies. Look at the following options:
- Simple interest, applied on Daily basis
- Simple interest, applied on Monthly basis
- Simple interest, applied on Yearly basis
- Compound interest, applied on Daily basis
- Compound interest, applied on Monthly basis
- Compound interest, applied on Yearly basis
Exercise: Take a travel booking site. From city, to city, round trip/one way, adults/children count, accommodation required etc. will appear on the screen as input parameters. Arrive at the combinations of these as test samples based on equivalence partition technique.
For recorded sessions on software testing, visit us at www.openmentor.net.
Monday, May 7, 2012
Let us take a simple voltage stabilizer that we connect to TV or fridge. Usually they give the range 190-240 volts 50 to 60 hertz input, output will be constant at 220 volts. If this needs to be tested, we need to look at some good and bad extremes.
- Connect to 190 volts
- Connect to 240 volts
- Connect to 189 volts
- Connect to 241 volts
But there is one more parameter we have - cycles in Hertz. For that, keep a valid voltage and push that cycles to extremes.
- Connect to 220 volts 50 Hz
- Connect to 220 volts 60 Hz
- Connect to 220 volts 49 Hz
- Connect to 220 volts 61 Hz
- Can I book a ticket for today?
- Can I book a ticket for yesterday?
- Can I book a ticket for a day 180 days from now? (usually travel sites may allow upto 6 months)
- Can I book a ticket for a day 181 days from now?
This list may go as per the restrictions given by that portal. Same way, let us take a US zip code field. How can I apply BVA on a zip code field? A zip code is usually 5 digits such as 95014 or 5 digits hyphen 4 digits such as 95014-2367.
A few home work questions for you to apply BVA.
- Write BVA tests for USA social security number
- Write BVA tests for the interest rate field in a banking simple interest calculation screen. (interest rate is a percentage value)
Monday, March 12, 2012
Simple mantra you must follow when you write test cases - think that you are giving instructions to test the software, to an 8th grade kid. The kid knows English and knows how to operate computer; but does not know anything on how to test your application.
Is it really possible to make a kid to test my application? Yes it is, provided the instructions are clear and simple. Have you ever called a customer support help desk to resolve a problem in your laptop? You might have observed that the help desk person asking a series of simple short questions and solving your problems in minutes. The trick is clear and simple document. Look at this following example.
- Open firefox browser.
- Navigate to www.openmentor.net
- Click on Word Games link.
- Click on the link that says "Click Here to start"
- Choose 4 from the list box shown.
- Click Go.
In the industry, a test case is written in a document or in a spreadsheet with the following details.
Test case Id: A unique identifier for this test case, across the project or company.
Test case name: Usually this will be the test condition that you wrote before.
Pre-condition: The pre-requisite that must exist before we do this test.
Test Steps: Simple step by step such as clicks, types, navigation etc to carry out the test.
Test Data: The data yo must choose or type in any forms.
Expected Results: What is the expected outcome when we do these steps with the given data.
Usually people ask me this question - why should we write this much, when I know how to do it? Remember, you or any other person in the project, may be moved to a different project tomorrow. In that case, should other person re-invent the wheel? So better put all the things you remember in testing this product, in a paper. Also, you be the thinker that adds more value; let the test execution be done by other juniors. It also helps in assessing whether we have missed any permutation or combination in testing. If everything is just remembered by you, no one knows how many tests you remember and how many you do not. So better to document every test case.
Writing test cases will take more time. Agreed. But it is usually for one set of test cases; after that, you copy-paste the same text and make modifications in steps or data and expected results. It is not usually all text in all test cases are typed fully by you. Once a pattern of test is identified and typed, rest is all making minor modifications to that text.
In the next posting, we will see examples of test cases and a few techniques like boundary value analysis etc.
Monday, January 16, 2012
Thursday, January 5, 2012
I have been seeing a lot of ads for online shopping portals. What do they really do? Sell items right from shoes to iPhones. A few product companies asked us to educate their testers in testing their product. We first educated the testers in the business flow of ecom portals.
What you see here is a slice from the testing checklist.
- View the hot promotion plans on home page
- View top level categories of products on home page
- View the order of sub categories of products under each category
- View top products list under each category-sub category
- View products as grid
- View products as thumbnails
- View small and large images of products with short and long descriptions
- Search for a product with brand name
- Search for a product with a model name or number
- Search for a product below certain price
- Search for a product within a price range
- Search for a product with some keywords in description
- Search for a product of certain color
- Search for a product in specific store
- Search for a product in a specific geography
- Search for a product that are referred by many community people
- Search for a product for a range of user ratings
- Search for a product with multiple combinations of above
- Sort product search results based on price
- Sort product search results based on store
- Sort product search results based on buyer rating
- Add a product to cart
- Add multiple products to cart from same search results
- Add multiple products to cart from different search results
- Remove a product from cart
- Add the same product to cart after removing it
- Add the same product multiple times to cart
- Alter quantity of products after adding to cart
- Alter quantity to zero in the cart
- Provide payment details to buy product
- Provide payment details and cancel it
- Provide wrong credit card details and try to buy
- Purchase product thru corporate purchase plans
- Provide shipping address same as user’s registered address
- Provide shipping address different from the user’s registered address
- Register new user with valid details
- Register new user with invalid details
- Register same user with same emailed and userid
- Provide same credit card details to two different users
- View the registration email contents for new users
- Change password for a registered user
- View change password email content
- Do not use account for 60 days and login after that time period
- Login as user, add a product, logout and login again
- Use correct promotion code for correct product
- Use wrong promotion code for a product
- Use a valid promotion code to a product but after its expiry date
- Use a promotion code more than once for purchase
- User a private promotion code of another registered user
- Track the purchase order for delivery
- Rate a product for its quality
- Join a group purchase and confirm purchase
- Join a group purchase and cancel after an hour or day
- Join a group purchase for more than once
- Define alert to notify when product price falls into certain range
- Purchase a product whose price is more than your credit card limit
- Purchase a product when the product is out of stock
- View product details from a different country settings
- File a complaint as a buyer
- View complaint tracking email content
- View complaint status on the buyer portal
- View past order history for a user
- Modify the user profile after a few purchase transactions
- Add products in wish list
- Manage different lists as part of user profiles
- Comment on a product
- Comment on a purchase
- Provide feedback on a purchase
- Refer a product to another user in the community
- Use one time shipping plan for a purchase
- subscribe to annual shipping plans and do a purchase
- Subscribe to newsletters
- Unsubscribe to newsletters
- Freeze a user account
- Unfreeze a user account
- Set transaction limits for users
- Generate reports for today’s purchase
- Generate reports for purchase for specific products
- Generate reports for purchase in specific geographies
- Generate reports for shipping status
- Generate reports for customer complaints
- Generate reports on purchase feedback
- Upload stores details who act as suppliers
- Upload price details from various stores
- Upload products and price details from stores in different formats such as xml, csv, Excel etc.
- Upload products and price details from stores with erroneous records
- Upload product image contents with static images
- Upload product image contents with animated images
- Create survey to selected users
- Publish a poll to all users
- Create product promotion codes and publish to selective users or geography
- Manage shipping and handling charges and terms
- Manage group purchase settings and products
- Extend a promotion offer
- Configure prizes for buyers on specific purchase criteria such as 100000th buyer etc.
- Configure corporate purchase accounts
- Manage out of stock alert settings
- Manage different payment gateway settings
- Configure SMS alerts to users
- View whether purchase details reflect in the CRM thru synchronization
For recorded training programs and online tests, visit us at http://www.openmentor.net