In my almost ten years working in software testing and test automation, I came upon the testing pyramid term probably more times than I can count.
If you feel like your team is spending too much time waiting for the test runs to finish, it might be possible that you are implementing the test automation pyramid wrong. Keep reading if you want to understand it better
What Is The Testing Pyramid?
The testing pyramid is a model that breaks automated tests into three main categories: unit, integration, and UI (or e2e tests). This allows the development of a more efficient test suite and helps devs and QA specialists achieve higher quality.
These test types are layered as a pyramid, with unit tests at the bottom of the pyramid, integration tests in the middle, and at the top, we have the UI tests.
The idea is that the bottom of the pyramid should have the highest number of tests, and the top should have the lowest number of tests. That’s because the test execution time increases which each level of testing.
Automated Test Levels
To understand the pyramid better, let’s explore each of its levels.
The unit tests, sometimes referred to as component tests, are the base of the test pyramid - this means that they should be the most numerous. They are typically created during the development process by developers or sometimes testers. The main goal of unit testing is to isolate the code to test to verify that it works as intended.
Unit testing is a crucial phase of the software development process because it can help discover early code issues that could be harder to identify in subsequent testing phases. Finding defects early makes them easier to fix before other components are affected.
Because they are very granular, testing only parts (components) of the application, unit tests are usually very fast.
Integration tests represent the second layer of the test pyramid. They usually take a longer time to execute, compared to unit tests, and their frequency should be lower.
In integration testing, we test the interactions between the individual units or software components, or the interaction between internal components and external components - such as databases or APIs.
Integration tests are usually slower and more complex than unit tests since they interact with external sources. You will also need a stable, pre-production environment for these tests, similar to the production environment.
End-to-end UI testing
End-to-end tests are at the top of the pyramid and aim at testing the application as a whole. The tests use environments and data that simulate real-world scenarios.
End-to-end testing is best approached from the perspective of the end user. QA Engineers may consider information such as the various ways a user may interact with the app, what they're doing to accomplish their goals with the app, and what could go wrong. The goal is to create a test based on these behaviors.
According to the testing pyramid, we should have fewer tests on this level - because e2e tests tend to not only take a longer time to execute but are also the most brittle and hard to maintain. They are also executed at later stages of development.
Manual testing and exploratory testing are sometimes included at this level too, because they usually focus on UI testing, and include more complex scenarios that test the system as a whole.
Why Use The Testing Pyramid?
The testing pyramid provides a logical, iterative structure for prioritizing which levels of testing to focus on and when. This is done by running the smaller, simpler tests first and saving the larger, more complex tests for later stages in the testing process.
When you consider the number of tests you will require at each tier of the pyramid, the pyramid structure makes even more sense.
The testing pyramid also provides a way for testers to receive quicker feedback on how the application performs by considering tests that should run in both the early and later stages of software development.
Teams can detect and fix defects more rapidly, which is incredibly cost-effective in the long term if there is a built-in reminder to obtain feedback as soon as the unit test level.
By testing at all these different levels, the testing pyramid provides a good way to increase test coverage.
The Testing Pyramid In Agile
Agile teams benefit most from using the test automation pyramid because:
- Agile methodologies prioritize effectiveness and speed. This is provided by the testing pyramid, which streamlines the testing procedure.
- Work is completed more quickly when a clear progression and logic are added to the testing process.
- Since the pyramid is designed to run the easiest tests first, testers can manage their time more effectively, produce better results, and make life easier for everyone.
- The testing pyramid helps testers prioritize tasks correctly.
If test scripts are developed with a greater emphasis on the user interface (UI), there is a greater chance that core business logic and back-end functions will not be thoroughly tested. This has an impact on product quality and increases the team's workload. Additionally, because UI tests have a big turnaround time, there is less test coverage overall. The test automation pyramid helps avoid such situations.
Cons of Using the Testing Pyramid
It might be wise to consider the following criticism of the testing pyramid:
- The value of the tests is not taken into account: A difficult test that is developed over a longer period (by a factor of 10) might be 100% accurate. If so, why would you require a large number of unit tests?
- It’s a bit outdated: In the ten years since the pyramid was thought of, there have been significant changes in the development and testing process. Modern automated testing helps streamline the test suite as testing now tends to focus more on UI testing.
- What happened to the manual tests? Not all situations call for automation. Manual tests, which many developers consider to be always necessary to some extent, are not usually mentioned in the testing pyramid.
Best Practices when Using The Testing Pyramid
When applying the automation test pyramid, keep in mind the following practices:
- Select the test cases you want to automate, then decide how extensive your automated tests should be.
- According to your use cases and the testing level, select the appropriate tool. For unit testing, tools like JUnit, and TestNG are used. For integration testing, we have tools like FitNesse, Protractor, or Citrus. For UI automation, the most popular are Selenium for web apps, and Appium for mobile apps, but other testing frameworks are used as well.
- Your program will be simpler to maintain and less likely to contain flaws if you write clean test code.
- Set test priorities.
- The foundation for test cases and scenarios should be high-caliber test data.
- Avoid having tests duplicated on multiple levels.
- Include tests in the pipeline for deployment.
- To make sure there are no unforeseen concerns, think about including exploratory tests in your testing strategy.
When applied correctly, the testing pyramid can be a great aid in the testing process and maximize the value of testing tools. By selecting the level at which each test case should be automated, the teams can increase testing efficiency and use their time more wisely.
If you liked this article, I promise you will get even more helpful articles by subscribing to The QA Lead newsletter.