Test Automation Pyramid: How to Automate Test Effectively

Mike Cohn introduced the test automation pyramid in his book Succeeding with Agile: Software Development Using Scrum, which is a concept that defines three layers against which tests can be automated: Unit, Services, and User Interface.

Test Automation Pyramid

If one reason you’re automating tests is to get things done quickly, you’ll want to make sure your automated tests run as quickly as possible. One way to do this is to automate the tests so that they are as close as possible to production.

Unit Level of The Pyramid

Automated Unit testing is a process in which the smallest testable parts of an application, called units, are individually and independently scrutinized for proper operation. Unit tests are small, modular tests that check the logic of individual functions without using other functions, databases, or user interfaces.

Unit tests are fast, and they can pinpoint the exact function where a bug is occurring. Because of this, most of the automated tests should be written at this level. This fits the goal of getting bugs quickly.

To execute Unit Tests, developers write a section of code to test a specific function in the software application. Developers can also isolate this function to test more rigorously, which reveals unnecessary dependencies between the function being tested and other units so that the dependencies can be eliminated. If any good test cases are found later but were not included by the developers themselves, those tests can still be added at this level by anyone with the know-how. Just because the developer didn’t add it to the unit level doesn’t mean it has to be automated at a higher level.

Service Level of The Pyramid

The services level is a little farther away from the code itself and focuses on the code’s functionality but without any user interface. At this level, automated tests can make calls to the product’s APIs to check how well different functions work together.

The unit level should have the most automated tests, but this level should have the second most.

UI Level of The Pyramid

The UI level is the farthest away from the production code. When tests are written at this level, they are harder to write as the UI is not always consistent and takes longer to run as it simulates the user’s behavior. Because of this cost, this level should have as few automated tests as possible.

When thinking about automating a test, figure out what information the test needs to check, and write the test against the level of the pyramid that is as low as possible. The faster the test, the fewer bugs will be found at the upper level, so the cost will be minimized because the cost of fixing a bug increases exponentially as the software moves forward in the SDLC.

In theory, this makes sense, but when the team builds features at the UI level, it’s easy to think that everything must be automated at the UI level. You should definitely test your UI, but you don’t have to do all the combinations of test cases at this level.

Let’s look at a real-world example.

Let’s say your team just added a new Search feature to the system. The main functionality of this search feature is finding the right product that matches the input text in that search box and implementing any filtering, sorting, column selection or page navigation works based on that result.

If you’re automating one Search test, I agree that you should do it at the UI level and do all those checks. But if you need to automate more than one Search test, you should reconsider whether all the cases need to be done at this expensive level. Or you might be able to automate just one case at the user interface level and the rest at a lower level. All you need is to automate one test that checks that UI is working, and the rest you need to do how well the search algorithm works. This can be checked at both the service and unit levels which are less expensive.

Let’s look at another example of automating shopping cart functionality.

The steps of the scenario are:

1. Search the item
2. Find the product through the search results
3. Click on Add to Cart of that desired product
4. Go to the cart by clicking on the cart icon
5. Validate items are on the cart

All of these steps can be automated at the UI level, but if any of these steps fail while running the scripts because of any UI dependencies, then the test won’t pass.

If you’ve already added a search test at the UI level, then you can skip this for this scenario. Instead, looks for some shortcuts in your application to make automation much more effective.

In this case, you can go straight to the product’s URL in your automated test instead of searching for it. That’s how you can get rid of as many steps as possible, which will save you a lot of time and lessen some of the risks of using the UI. Not only that, but you are also removing the need for the Search feature to work in this test. Adding something to the cart has nothing to do with the search.

One more example- Let’s say you need to automate another scenario: Adding more than one product to the shopping cart.

If we think about this situation, the steps could be:

1. Search the item
2. Find the product through the search results
3. Click on Add to Cart of that desired product
4. Search another item
5. Click on Add to Cart of that desired product
6. Go to the cart by clicking on the cart icon
7. Validate items are on the cart

Again, searching for a product and adding it to the cart does not have to be done at the user interface level. By doing them there, you’re adding time, dependencies, duplication (since you already have a test for this at the UI level), and the chance that your tests will break.

You can use some API call that adds the product to the cart, then go straight to the cart URL and start your test. As you already test the cart feature at the UI level, this is much quicker and less likely to fail because of something unrelated.

In conclusion, we need to plan when and where to automate, which will be less expensive and take less time. If you plan the tests correctly and select which cases need to be automated at which level of the pyramid, it will save you a lot of time and resources.