Types of performance testing and their definitions
Defining Performance Testing:
A performance test is a test of a complete system carried out under reproducible and production-like conditions, where statistically significant response times are measured over a period of time when the system is under a known business emulation load. In simple terms it can be stated as the testing conducted to evaluate the compliance of a system or component with specified performance requirements. It identifies the inefficiencies and bottlenecks in an application under normal operating circumstances.There are different types of performance testing. This classification is mainly based on the way the testing is conducted. The types are:
· Load Testing
· Stress Testing
· Endurance Testing
· Volume Testing
· Availability Testing
· Scalability Testing
Let us see each one of them in detail.Load Testing:
Load testing can be defined as a type of testing where the application is subjected to different type of workload pattern for a given time interval.
Load testing consists of simulating real-life workload conditions for the application under test. It helps you determine various characteristics of the application working under massive workload. You can perform your test several times with a different number of users working concurrently to find out how various parts of the application react to the increasing load.
It can emulate the workload of hundreds or even thousands of users, so that you can predict how an application will work under different user loads and determine the maximum number of concurrent users accessing the site at the same time. Since it simulates a real time user load on the application, it can be called as the more realistic testing and doing this prior to production ensures application will be stable and any performance issues can be addressed in pre-production phase.
Stress Testing:
Stress testing can be defined as structural testing technique that measures whether the application environment is properly configured to handle expected, or potentially unexpected, high transaction volumes. Ideally, stress testing emulates the maximum capacity the application can support before causing a system outage or disruption. It ensures the application which is tested for expected load can take on spikes in the load condition like increase in rate of transactions and study its impact on the system resources and helps tune and configure the system optimally.
A typical example of stress testing would be stressing an application with say 50 users initially and then slowly ramping up the number of users at a constant interval time. As the load on the application increases more system resources like CPU and memory are utilized and this increase is linear. At a given point of time the resource utilization reaches the saturation level and performance starts to degrade. Stress testing helps us to identify this point.
Endurance Testing:
Endurance testing can be defined as a type a testing where the application is subjected to more or less a constant workload for extended duration of hours. The duration can be as minimal as 12 hours and can go to maximum of 2 to 3 days. The main idea behind this type of testing is to check for the behavior of the application when it is subjected to stress for longer duration of time.
A typical example of endurance testing would be like running an application with a constant load for duration of say 24 hours. During execution the important parameters which are monitored are CPU and memory utilizations of the servers. Other parameters like response time and number of transactions achieved per second (TPS) is also monitored. An application can show a memory leak when it is subjected to stress for longer duration of hours. These things can be easily found out using endurance testing.
Volume Testing:
Volume Testing is generally considered as the process of finding weaknesses in a given system with respect to its handling of large amounts of data during short or extended time periods. It is a typical load testing except that a large volume of data is populated on the database to study its impact on the application response time and database overall health.
Volume testing attempts to determine how dependable and robust a given application is rather than focusing on the responsiveness of the application or the throughput of the network that supported it.
For example, an often used window in an application is populated with data by calling a database object which runs a complex SQL query. Supposing the component is tested against tables with only 4-5 records of course it will return within seconds. Everything seems fine. It is then integrated with the window, and system tested. Again everything seems ok. It is only when the application is in User Acceptance (or even gone live) and it is tested against 100,000 records, is it discovered that, the SQL was not properly optimized and the tables not indexed. Thus it should have been tested at the component level. This issue can be found out and fixed on doing volume testing.
Volume testing needs two things. Firstly clear expected outcomes of how the software is to behave for a given level of data. Secondly, data and lots of it.
Availability Testing:
Availability testing can be defined as running an application for a planned period of time, collecting failure events and repair times, and comparing the availability percentage to the original service level agreement. It is primarily concerned with the measuring and minimizing the actual repair time of the given application.
The typical objectives of availability testing are to:
Ø Determine:
1. If the application meets its operational availability requirements. (For example, availability testing can cause failures due to memory leaks or database indexing defects.)
2. How stable the application is.
3. Whether (and how much) downtime is required for maintenance purposes.
Ø Report this information to the development teams so that any associated defects can be fixed.
Ø Provide information that will assist in the optimization of operational availability.
Scalability Testing:
Scalability testing differs from simple load testing in that it focuses on the performance of your Web sites, hardware and software products, and internal applications at all the stages from minimum to maximum load.
One common issue is servers that perform well at lower loads sometimes cannot maintain their level of performance at higher client levels, even when we increase the theoretical capacity of the server. In the case of a Web site, faltering performance levels mean lost customers.
The purpose of scalability testing is to determine whether your application scales for the workload growth. Suppose your company expects a six-fold load increase on your server in the next two months. You may need to increase the server performance and to shorten the request processing time to better serve visitors. If your application is scalable, you can shorten this time by upgrading the server hardware, for example, you can increase the CPU frequency and add more RAM (also, you can increase the request performance by changing the server software, for example, by replacing the text-file data storages with SQL Server databases. To find a better solution, first you can test hardware changes, then software changes and after that compare the results of the tests).
If the scalability tests report that the application is not scalable, this means there is a bottleneck somewhere within the application. Scalability testing can be performed as a series of load tests with different hardware or software configurations keeping other settings of testing environment unchanged.
A good example of scalability testing would be the scalability testing done for Microsoft Exchange 2000 by Microsoft. The final result of this test was Exchange 2000 could process the message traffic for up to 83,000 simulated users, whereas Exchange 5.5 could only handle up to 50,000 simulated users before reaching the capacity. In simple terms it was found out that Exchange 2000 was 66% more scalable than Exchange 5.5 their previous version. 5.5. At all levels of simulated users tested Exchange 2000 produced lower average server response times and consistently good service receiving and delivering e-mail for up to 83,000 of the simulated users.
