Performance testing is part of any healthy IT strategy. It’s a given that any application with high-volume usage will undergo regular performance testing. These tests expose faulty assumptions about how apps handle high volume, ensure infrastructure scaling works as expected and identify load-related defects. Performance testing’s ability to detect flaws occurring under high load can help improve applications regardless of scale.
By their nature, performance tests run differently than other tests that applications leverage. They test basic system functionality while under heavy load. This load is often hundreds or thousands of requests per second to the system under test and continues for enough time to verify the application will handle the load. These rapid and successive testing sequences simulate load on the system and determine system behavior under stress. For performance tests, the word “system” is important: it implies the application working with all or most of the environment it will run in. Meaning, performance tests are a specialized case of integration tests that examines application behavior as it is accessing unmocked external systems*.
So, how do performance tests help us find bugs in all applications regardless of scale? They expose hard to find, replicate, or late occurring bugs by forced repetition. If a bug occurs 0.0001% of the time, then every 10,000 interactions with the system will expose the defect. This ability to find so-called "heisenbugs" is great, but these tests can also help find other rare or slow developing problems. Below are some examples of where performance tests find hidden problems:
1. Memory leaks
Running the application under heavy load creates and frees a lot of memory. If there is a reference that is not being freed, then it will be exposed under performance tests. The specific test to expose this kind of problem reliability is called an endurance test. The idea is to keep load at a high level for an extended period of time, around 24-48 hours. While this makes endurance tests impossible to run on every build / deployment combination, they should still be run periodically to ensure errors that occur after long runs are caught.
2. Threading Problems ("Deadlocks and Races")
While threading problems can occur during regular testing, they are much more common under high load. This is because most systems that are heavily threaded do so to handle high load situations. It is also because many thread problems only occur occasionally and performance tests give more opportunity for the problem to occur.
3. Value size errors
While this is not as common an occurrence as the other problems, sometimes applications and the databases they use undersize a shared variable and have problems with rollover or overflow. These again are issues that are well served by endurance tests.
4. Shared state problems
These are errors where assumptions are made about the behavior of shared state in an application. These problems are found in all three of the other issues.
Given the ability to find difficult problems, I recommend performance testing for every team. This includes applications that generally don’t have a performance concern. It is better to find a defect during initial performance testing than after months of operation as is often the case with these types of defects.
* This is not always strictly true as some systems will require mocking as they are not under the team or company's control.