Thursday, February 13, 2014

Performance Testing - Analysis - Part 1

Let us assume that we have executed 1000 virtual users on our web application and collected all required performance counters details as well. Now it is time to analyze. A performance tester, must be in a position to go very close to the problem point, so that the development team can fix the performance issues easily. If we simply throw so many details to dev team, it will not help any one. Analysis of performance test results is a separate field by itself. A multi-billion dollar field.

When you see a performance test result report, generated by the tool, first look for any errors. Usually tools will list out http 400 series errors or 500 series errors. The errors may happen randomly or may happen for  longer period of time. Every such error must be explained and fixed - there is no second thought on this. If a page returns http 404 error, dev team must either remove that reference or copy the actual file in proper location. When http 500 series errors do happen, tools provide the response text that came from server side. That must give a clue on what happened. If that is not sufficient, developers need to look at the webserver log files for any exceptions, database logs for any exceptions or app server log or application's custom log files. 

If one sees no issues on server logs, it may most probably be because of a wrong data passed to the pages. Hence tester needs to check what data was passed to that particular vuser, at that specific time - tool will provide that. Never ignore even a single error. One error may cause ripple effect and can lead to multiple failures. 

The next step is to identify the slow transactions. For this, we need to define what is slow. Is it 7 seconds or 9 seconds? Is it 9 seconds for all pages? Tough to say. But there are companies like Compuware that publish industry benchmarks for various industries such as travel, banking, media etc. Check those benchmark response times and compare your page responses. Whichever page that are above the benchmark level, definitely need fine tuning. So, first mark such pages that have response time more than your performance goals or industry benchmark or both.

A page with response time more than your goal of 7 seconds (assumption), may not be always like that. It might have responded in 3 seconds at some point and might have responded in 12 seconds at some point. It is also better to take 90th percentile average response time to compare, instead of simple average. The 90th percentile means the average response time of 90 percent of your transactions. Usually tools will remove the highest 5% and lowest 5% response time and arrive at this - but see respective tool's help file on how they arrive at this 90th percentile average. Our aim is to find the time at which the page responded slow. Some times, the page may be slowing down after 30 minutes from the start of the run. Usually, tool reports will have elapsed time as X axis. From elapsed time, find out at what time windows, the page has slowed down.

Then check whether all pages have slowed down during that time window or only this page that has slowed down. By now, you would have narrowed down to slow responding pages, and slow response time windows.

What to do next? We will see that in our next post.


No comments:

Post a Comment