How to test microservices?

Example driven practical test pyramid for testing microservices

Developing software products with microservice architecture is a de facto standard for over a decade now. This approach definitely helps and there are hundreds of articles, books, videos and courses about how it is beneficial. To quote Wikipedia, it is “an arrangement of application as a collection of loosely coupled services”.

Though it solves a lot of problems in traditional software development and brings in gazillions of benefits, have you ever wondered if there is anything that becomes difficult with the adoption of microservices? It is testing and that’s what this article is about.

As opposed to microservices, in a monolith, all the code is bundled and deployed as a single application. So, testing your product means testing one application (most of the times). Depending on how the monolith is designed, it is relatively easy to test one application than testing bunch of applications in a microservices architecture. (Having said that, even monolith could be built in such a way that testing can become very painful. That’s why don’t start building microservices if you can’t build monolith right.)

In this post I’ll share my experience of how my team started thinking about testing microservices so that all the user actions a.k.a user journeys are tested. This post is structured in following parts —

1. What is the example product under test?
2. How the product looks in microservice architecture?
3. Revisiting the purpose of testing modern applications.
4. Testing each microservice in isolation.
5. User journeys tests with all microservices together a.k.a end to end tests
6. How CI/CD looks?

1. What is the example product under test?

As an example for this post, I am picking up an order management portal. Users of this application will be ordering products for their online shop from third party vendors and will be able to manage the inventory.

Let’s define the scope of this application from product requirement’s perspective and without going too much into technical details.

We can break the requirements into 2 types — (a) Functional and (b) Non-Functional.

1.1 Functional Requirements

1. When users log into the application, they see products which they eventually want to order from the vendors.
2. Users can select those products and create orders for vendors.
3. Users can edit the orders and change the product quantities they want to order, change the delivery address, change the potential delivery date etc.
4. In a “Past Orders” section, they should be able to get a read only view of the orders they have placed in the past and vendor’s reference number for each of the orders.

1.2 Non-functional Requirements

1. Multiple users should be able to use the system without loosing their orders.
2. The system should be able to track user actions so that the resulting data can be used by the analytics team.
3. If the vendor integration has problems, the system should automatically retry sending the orders so that user has seamless experience.

2. How the product looks in microservice architecture?

Now that we have a set of functional and non-functional requirements, let’s start designing the application in microservice architecture.

Please note that this post is not about how to break your application into microservices but about testing the microservices. Your architecture may look different than what is shown in the diagram below.

Microservices under test

Order Management Portal is everything you can see inside the box. The application is broken into several components which can be independently developed, deployed and scaled. Let’s talk about each of the components shown in the diagram above. Each of these applications is running as a docker container and accessible as a web service. For the sake of example, let’s say the front end service is developed using ReactJs and all the backend services are developed using Spring-Boot and Java.

1. Front End Service — UI application where user can log in and perform all the activities such as selecting a product, creating an order, editing an order and sending an order.
2. Product Service — The main responsibility of this service is to maintain a collection of products being used by the online shop at any point of time. Front End Service is directly using this service to fetch the product details whenever a user is trying to search for any product and wants to order them. It exposes a bunch of REST API for CRUD operations on Products.
3. Order Management Service — This component is responsible for maintaining the orders created by users. Any change made by the users on their orders are stored in the database maintained by this service.
4. Analytics Service — This service is responsible for gathering user actions such as which products are being ordered more, what is the typical volume of orders during a specific period of time etc. Based on the analysis, it tries to find out the less selling products and tries to remove them from Product Service.
5. Vendor Integration Service — This service acts as a facade and it is responsible for all the communication with the actual vender systems. It maintains all the configurations related to vendors and based on the raised orders, identifies vendor system to call and places the order with those vendors.
6. Event Stream — When a user decides to place an order for a set of products, it is raised as an event and sent to event stream. Now, Vendor Integration Service and Analytics Service have event consumers which consume these events. Event stream is particularly useful so that no orders are lost and can be retried if there are failures.
7. Vendors — These are third party systems which the order management portal has no control over. Vendor Integration Service communicates with these vendors based on the API contract with the vendors. On successful order creation on vendor systems, let’s say they return HTTP 201 i.e Order Created.

3. Revisiting the purpose of testing modern applications

We test the software roughly for following purposes —

3.1 To verify that the software is doing what it is expected to do. We write the tests based on the requirements and assert things to match the expectations.

3.2 Since software development is an iterative process, we test the software to be sure that the recent changes have not broken anything. By having test suites to verify the typical application flow, we make sure that the application keeps behaving the way it is expected even after introducing any new changes to the codebase.

4. Testing each microservice in isolation

A typical web service application is generally divided into following layers.

Typical Web Service Application

1. Controller — This layer of the application defines all the API the application is wants to expose. e.g GET /products, PUT /products/{productId} etc.
2. Service — Service layer is where all the business logic resides and it is directly used by the layer above i.e. Controller layer or by other services in the same layer.
3. Repositories — This layer is responsible for all the interactions with the database. It can use any ORM frameworks like Hibernate, Spring Data or MongoDB ORM frameworks depending on which database you are using for that service.
4. Database — It is self explanatory. This is a storage layer and can be any Relational or NoSql database depending on the use case and purpose of the service.

4.1 Test Pyramid

There are different versions of test pyramid. It is interpreted and customized as one needs. The end goal however is to make sure every piece of code and all the business logic is tested. In my opinion, the test pyramid should be able to depict your testing strategy in terms of both functional and non-functional requirements (check section 1.1 & 1.2)

Below is my version of practical test pyramid for the microservices based applications.

Practical Test Pyramid

4.1.1. Unit tests

This is the most crucial part of any application. For every microservice, unit tests mean you write tests for each and every layer i.e. Controllers, Services, Repositories in the backend services and similar layers in the front end services. This can include the integrations tests for repositories with embedded databases, services layer tests with mocked repositories and controller layer tests with mocked services. There are multiple libraries/frameworks used for mocking the dependencies. To name a few —

1. Mockito — used to mock dependencies
2. WireMock — used to mock api calls

One of the important parts of unit testing is to make sure that you track the code coverage. There are many tools which you can use this purpose SonarQube being the most widely used one. The main purpose of tracking the code coverage is to find out if you have any untested code and improve your test suites to cover those. It is easy to get carried away with the test coverage percentage. It is important to know that the test coverage percentage does not really reflect the quality of your code, it just shows which lines/conditions/methods are covered by your tests and not how well written your tests are. There is a really nice article about this which you can go through to expand more on this topic.

4.1.2. API tests

In the microservices based applications, generally the services communicate with each other over REST API. It is very important to test if the APIs are working as expected. You can test the api at unit level using MockMvc provided by spring boot. But with this, you still need to mock the service layer. How about testing API more in a black box manner?

The unit tests can cover the the things such as API returning HTTP 400 for various bad requests scenarios, HTTP 200 for happy scenarios etc. However, It is equally important to test the API behavior in terms of business requirements. These scenarios can be described and tested in a typical given-when-then style. Let’s see some scenarios for couple of services we mentioned above —

Product Service

1. Given product service database has some active products, When GET /products API is called, Then it should return all these active products.

2. Given product service database has some active products, When PUT /products API is called to make a product inactive (by analytics service), Then the GET /products API should not return the inactive product.

Order Management Service

1. Given user has some products to order, When POST /orders API is called, Then it should create an order and return HTTP 202 (that is the order is accepted and it will be eventually created).

2. Given user has placed some orders, When GET /orders API is called, Then the it returns all the past orders for that user.

This layer of testing adds more of black box tests and tests API from common business scenarios in that particular microservice.

I highly recommend using Rest-Assured for such API tests.

4.1.3. User Journey tests

Unit tests or API tests are more of testing just one microservice in isolation and most of the engineers do this correctly. When it comes to testing all the services together, it becomes tricky. Because, now we need to test the coordination and communication between multiple microservices. As the practical test pyramid above shows, these tests are less in number and should just test the minimal user journeys from frontend. More on this in section 5 below

4.1.4. Manual tests

Even after you have a fleet of automated tests for your product, sometimes it’s not enough. You always need to test things manually for some corner cases. As the pyramid shows, try to keep them to minimum and on exploratory basis. If you see yourself doing the manual testing for a scenarios more often, it’s time to push it to the layers below i.e. automate it in user journey tests.

5. User journeys tests with all microservices together a.k.a end to end tests

In the steps above, we tested all the microservices in isolation with unit tests, api tests and have a sense of confidence that all our services work as expected. But, now it’s time to test the user journeys end to end i.e. with all the services together.

In the context of our example, ideal user journey tests would be following —

1. When user logs in, he/she sees a list of products to be ordered. That means, we need to spin up product-service and it should have a set of products already in its database.
2. After selecting the products and required quantities, when user places the order, it will land in order-management-service and eventually go to the vendor via event stream.
3. After placing the orders to the vendors, user should be able to see the past orders in the “Past orders” section on the frontend.

These 3 steps cover almost everything that a user will experience on front end.

Now, as vendor services are third party API, we can not really call the actual third party API from these tests because the test should not depend on the services which you are not responsible for. If you are using third party systems, the tests can become fragile and unreliable. You would not want to see your tests failing because the actual vendor system is down or returning HTTP 400 instead of HTTP 200 because of some internal issue in their system.

A colleague of mine asked me an interesting question — how will you test the connectivity between vendor integration service and actual vendors? Yes, we definitely need to test the connectivity between our system and the third party system but not via automated tests which run after every git push. It is network connectivity and need to be done only once in the beginning. If the third party system is down, the calls will anyway fail and your logs will be flooded with such errors/exceptions. You need to have good monitoring for such cases. Since the chances of loosing network connectivity between your service and the third party systems is close to 0%, you don’t need to waste CPU cycles by checking it via automated tests every single time some code changes are pushed.

There are multiple ways to implement these tests. The most common and easy way is to use docker-compose. You can easily create an isolated environment with all your services running as docker containers just with a yaml file.

However, there are few things you need to consider —

1. All your microservices need to have docker images and the user-journey-tests has a way to pull these docker images for testing purpose.
2. As you can not use actual third party systems for tests, you need to create stubs for them. Stubs should be lightweight services exposing the third party API. Stubby4j is one of the popular libraries used to generate such stubs. I am a big fan of using express.js for such purposes because you can literally expose REST API with 5 to 6 lines of code. You can check my previous article on performance testing the api to see how easy it is to create an api server.
3. Databases need to run as docker containers too. That way you can easily link them with other containers.
4. Once you have all the services mentioned in the docker-compose.yaml file, it is important to manage the coordination while booting up the services. You can use wait-for-it.sh to do that.
5. Now coming back to writing actual test code for these types of tests. As these tests are driven by the user flow, it’s no surprise that we need something that works on Frontend/DOM. You can use most widely used framework called Selenium or javascript library like Puppeteer.

6. How does CI/CD look?

This is the most important section which answers the question of when should you run these tests. When it comes to Unit tests and API tests, they have to be run for every change in the particular service. User journey tests however need to be run after any change in any of the services which are involved in the user journey.

For example, if you make any change in the GET /products API in the product service, you will definitely run Unit tests and API tests as part of the product service’s build. However, after both these steps pass, you need to run the User journey tests by pulling the latest images of other services like order management service, front end, analytics service and vendor integration service and run the user journey tests.

Again, user journey tests are expensive because running tests with selenium and performing actions on the DOM takes time. The more time the tests take the more delayed the feedback is. That is the reason, these tests need to be focussed and need to be kept minimal.

So, in short this is how you need to run these tests —

1. Unit tests — run on every git push
2. API tests — run on every git push after unit tests
3. User journey tests — run on every git push after unit tests and API tests. The service in which you made changes can have a the docker image created from the pull request, for all the other services you can pull the latest available docker images from the docker repository.

This particular structure worked out very well for my team and may differ from the way you are testing your services. Please add your comments if there is anything missing or can be improved.

Happy testing! 🙂