Know your application better with visual studio 2010 load testing
August 10, 2012 § Leave a comment
Why this article again?
Good question! yeah when you implement a complex system & high volume data performing transaction. The Automation comes to rescue everywhere for testing as well as system. So I sharing it to everyone
Here is article on
Know your application better with visual studio 2010 load testing
We’ve reached a day when every business applications are built with a varieties of complexity and grows with large volume of data. Over the time when data volume grows or user base increase , there is a scope of decreasing the overall performance of the application or even failed to deliver as expected. For this reason as a precaution actual capability of the application must be measured in advance to avoid unexpected situations. Now a days simple application which will serve single user consistently is becoming antique. This is time when applications scalability and performance plays a vital role in the software specification as well as architecture. And thus the performance testing and analysis became handy. Goal of this article to provide a quick and simple approach to configure load test for real scenario and analyze results. Through this article I will share my experience of implementing load testing and analyzing results with visual studio 2010 ultimate.
“load testing with visual studio 2010” goggling this phrase is more then enough for anyone to find lots of fantastic articles about basic load testing setup. So in this article we will not walk in that path, instead I will share some additional info and share some configurations that will require to setup load testing for real life and focus on the analyzing the results. But before moving on I recommend you to learn basics of web test and load test if you are completely new to this kingdom. Visual studio 2010 shipped with very handy testing tool, we are going to utilize this tool to perform load testing. I recommend you to go through how to setup web test first, it will help you understand better.You can suggest any additional info to make this post more helpful.
Load Testing Topology
Load testing involves generating traffic to a system and measuring how it responds. Tools simulate heavy volumes of users taking various realistic scenarios in the system and we will use visual studio 2010 for this purpose. There is two ways to run load tests. Generate load from Visual Studio 2010 this will uses only 1 core/processor with limited to 250 simulated users. Another way is load testing through test rig which can simulate lots more users require much more more configuration in network and hardware layer. To avoid complexity we will not focus on test rig, but some day we will share this experiences too. The following diagram shows how a thorough load test could be set up using visual studio 2010.
While doing load testing these are the considerations
- Understanding the environment of the application being tested
- A test plan defining user scenarios
- Tools to generate load and capture metrics
- Analysis of performance indicators
Visual studio 2010 can simulate approximately 1000 users per processor. we can generate load from Visual Studio 2010 through 2 approach
- Default: uses only 1 core/processor and limited to 250 simulated users
- Test Rig: can use multiple core/processor and can simulate lots more users
It is recommended that performance testing database and application database shouldn’t on the same server otherwise while test controllers sample and save the load test run results in the database on the application server it may impact the test results.
Visual studio load testing tool is consist of versatile concepts implemented and msdn got all well explained, which could be way out of head for a starter, So rather I listed few impotent concepts that are least required for carry out the load test.
- Think Time: the time spent by a user perusing a Web page, including viewing the page and determining the next action. Think time does not include the time that is required for a page to load.
- Ramp time: This property allows the increase number of user from one step to the next to occur gradually rather than immediately
- Sample Rate: Time interval between collecting test data.For longer load tests, increasing the sample rate reduces the amount of data that you collect.
- Run duration: Total time the test is expected to run.
- Warm up: the load test will increase the load gradually during the warm-up period
we will use above concepts are configured in our attached sample project. To simulate real time request/response we can utilize mix that can control the percentage of load that is distributed among tests, browser types, or network types in a load test scenario.There are three types of mix
- Test mix: It can specify for the probability that a particular test will be randomly selected by a virtual user during a load test run.
- Browser mix: It do not simulate the test in a stripped down browser. Test agents includes a set of associated headers for the selected browsers and added to each HTTP request before submit.
- Network mix: It simulate network bandwidth at the application layer. It does not simulate latency.
Configure and Use Sample Project
Sample project consist of 2 projects
- TestingBox: test project consist of web test and performance test
- TestingBox-Demo: we project invoked by the load test
In order to use the sample project within your application, you need to configure theses parameters below according to your needs,
- Think Time: to simulate realistic wait times we set 3 seconds.
- Think Profile: Normal Distribution
- Step Ramp time: the user load would be increased from 10 to 20 users over a 10 second period
- Sample Rate time: system will collect result data after each 10 second
- Run duration: 10 minutes which is calculated based on the step load distribution
- Warm up duration:load test will increase the load gradually within 5 seconds.
Another consideration is database.Visual Studio strictly requires SQL Server for the load test store. If you don’t have a SQL Server database, you cannot save test run data. You can check the databse setting from Test (visual studio menu) > Manage Test Controllers. A dialog will appear which allows set/reset of connection string of test data store.
The primary goal of performance testing is to study application performance under normal load and heavy load for sustained durations or for short durations. In order to know our applications capability, our load tetsing should be answer these questions below
- Does web application function properly under given load?
- Does the page response times satisfy target requirements?
- Which one is the slowest page on web application during tests?
- Which page take longest time to download?
- How many simultaneous users can web application handle?
And in this section we will try to find these answers by analyzing graphs provided by visual studio load testing.
At visual Studio 2010 several built-in graphs are provided. we can use the built-in graphs as is or we can customize them. Additionally, we can create our own graphs.
Load test analyzer graphs display counters. Each graph follow a specific measurement to plot a graph. The y-axis is not labeled with any specific range, but is instead labeled with values from 0-100 that represent a percentage of the total range for each counter. For example, for a counter with a range of 1000, a data point of 60 on the y-axis would correspond to a value of 600 for the counter. The horizontal timeline x-axis for a collapsing graph has an origin of 00:00 and a range that covers the entire test run. Average’s are not plotted in graph.
We can analyze load test results by using the Load Test Analyzer in the following ways:
- Monitor a load test in real time while it is running.
- Analyze a load test results
- Analyze results from previous load test(s).
Now lets try to find our answers from the graphs.
Does web application function properly under given load?
We can create an HTTP summary graph which will contain “Request time out”,”Passed request”,”Failed Requests”. This will provide time-based frequency of errors over the duration of the test. Error rate is an important metric in stress testing. This indicates the maximum number of users that can be served correctly, without errors and hence application capacity.
Ideally error percentage should be zero throughout the test run. If error percent is high, root cause of the error should be analyzed. It could be because of wrong inputs to the load test, network connectivity problem or the server might have crashed.
Does the page response times satisfy target requirements?
Visual studio load testing has a builtin graph called “Page Response Time”. This response Time versus Elapsed Time report indicates the average response time of the transactions over the elapsed time of the load test as shown.
From this graph it is possible to identify peaks in the response time over a period of time. Ideal behavior of response time graph is that response time does not increase with load. The point at which the graph increases sharply indicates beyond this load server cannot serve the request and users will see no response or a very slow response
Which one is the slowest page on web application during tests?
Same “Page Response Time” graph can be utilize for this purpose. This graph represents average response time for the each page in time. Each bar in the graph is the average server response time for each page.From this graph it possible to identify slowest page of the application.
Which page take longest time to download?
Same “Page Response Time” graph can be utilize for this purpose too.Each point in the graph is the average of the page download time during that interval. Page Download time graph of individual pages help you to identify the exact page that has high download time.
How many simultaneous users can web application handle?
Built in”Test Response Time” graph can be used for this purpose. If we analyze Response Time Vs Test response Graph we can find out the capability of the web application.The Response Time graph shows the time the server took for responding as the test load is increased. The peak indicates that the server has reached its capacity to serve data, and is unable to scale further.
Finally, visual studio load test has so much more to offer, this post only focused on those analysis that I thought important to get to know your application better. You can come up with lots of custom reports by utilize the result data if you need to. Hope this will help you.