The Microservices pattern has been the hype among developers for quite some time now, all because of its robustness and simplicity. It has enabled developers to write applications quickly and in a more integrated way. While a lot of developers are creating state-of-the-art architectures using microservices, it also has antipatterns that cause more harm than good. Understanding various microservice patterns is very important to avoid basic design mistakes that developers make. In this post, let’s discuss database patterns in the context of microservices paradigm and how they can be implemented. We will look at what these patterns are and how they have been used in the e-commerce space.
Database per Service Pattern
Usually, a monolith application has a single big data store that stores the data for all the needs of the application. In microservices, we divide the application into independent services. The database per service pattern suggests having independent, scalable and isolated databases for each microservice instead of having a common datastore. Each service should independently access its own DB, services cannot access each other’s DBs directly.
Database per service patternThis pattern ensures loose coupling between services, any issues, or changes in one service would not affect any other service or its database. Also, it is not necessary to use a single type of DB for all services. Each service can choose its database according to its requirements.
Example Scenario
Let us take examples of some of the most common services that any e-commerce platform requires. The typical Orders flow is handled by multiple services. Two services that the order management system interacts with are item service and inventory service. Item service needs data related to items, their cost, sellers selling it, and inventory service needs data related to how much inventory is left for the items. Both data storage requirements are related, but if we store them in a single database, it will become huge, complex, difficult to manage and the queries (especially joins) will take a lot of time to execute.
To solve this problem, we can use the database per service pattern. An items database will store items data and another inventory database will store inventory data. Items service will connect to items db and inventory service will connect to inventory db. This way, if there is any issue in items service, inventory service will still keep running.
In database per service pattern, each service has its own databaseHow to run queries that require data from more than one service?
One fundamental problem to solve here is what happens if reading data from both items and inventory databases is required. For example, if I need the item name along with inventory numbers for all sellers selling that item? In this case, usually we would perform a join if there was a single database and two tables in it.
Services don’t access each other’s database directly (Data access using API composition)The applications can perform the join instead of the database performing the join. If the items service requires data from inventory service, it can request. The inventory service will have APIs to provide this data. For example, for inventory for a specific item, the item name can be retrieved from the items db, and then using the item id, an API request can be sent to inventory service to get the inventory for that item id. This is known as API composition. One of the ways of achieving this is by using the Saga pattern, as explained further below.
Advantages Over Single Shared Database
You get the following benefits by using the database per service pattern for designing your microservices.
- Faster development — There is no development time coupling between services; each service can be developed independently. For example, if a developer working on items service wants to update the schema of its db, they can do so. The schema of inventory service would not be affected, thus development on inventory service can continue without interference.
- No runtime interference — Any queries on one database will not affect other services. If a query on orders service is taking time to run, queries on inventory service can still execute. Or if inventory service puts a lock on any data item in its database, data in the other databases is not locked.
- Freedom to choose different databases — A single database may not be suited for all microservices. If one service requires a relational database and another service requires a NoSQL database, they are free to choose. In the case of shared database, a single type of db has to be used by all the services, but database per service pattern gives the independence for services to use the database type they require for their use case.
- Fault Tolerance — If orders service database fails, inventory service will still be operational. Database per service pattern gives more fault tolerance than shared database pattern.
- Loose coupling — Changes in one database will not affect the other databases. In case of a shared database, suppose if the orders data has to be moved from one type of db to another type, the whole database needs to be moved. In the case of database per service, only the orders data can be moved independently.
When not to use Database per Service
While it is immensely helpful to use this pattern in large sets of microservices, it is not advisable in the following scenarios:
- In a transaction-heavy system, it might become difficult to use multiple databases. A single RDBMS system is best suited for such use cases. For example, implementation of a banking application where multiple services can do transactions, such as customer service and account manager service should be done using a shared database design to maintain reliability of the transactions.
- Queries that require complex joining of multiple tables might be difficult to implement in this pattern. For example, services that require complex searching use cases where data needs to be derived using joins.
The Saga Pattern
The word “saga” means a long story with many events. That is what we are also trying to achieve. We are trying to read data that is residing in a database that is not directly readable from the service we want to read it in. If our service is the start of the story and the database is the end, we will be performing a series of local transactions in the intermediate services that will act as the various events of our story.
When a request is received at our receiving service, it will perform operation at its local database and send an event to the next service. This will go on until the whole saga is executed. To attain atomicity, each local transaction has a rollback transaction that will revert the database in case its local transaction fails.
So, if we take our ecommerce example, if a user requests to purchase an item, the item service will read the item details from its local database and send these details via an event to the inventory service. The inventory service will then check the inventory details for the given items in its local database and update the inventory if available. It then sends a response back to the items service and the item service completes the transaction.
In case the transaction at inventory service fails, it will send a compensatory response to items service indicating a failed status for the transaction. If the inventory database is already updated, it is the responsibility of this compensatory transaction to revert it to its original state.
Implementation
There are two ways of implementing the saga pattern in a microservices based system, choreography, and orchestration.
Choreography is simply implementing sagas using events. Each service sends an event directly to the next service it wants to trigger and receives a response in return. Here, there is no central service dedicated to managing the transactions. Therefore, choreography is the decentralized way of implementing sagas. The main advantage it provides is that there is less coupling between the services. Scalability is easier and there is no single point of failure. The drawback is that it is harder to monitor, maintain and test as there is no central system.
Choreography based sagaUsing choreography, our item service will invoke an event for the inventory service to get the data it needs, the inventory service will receive the event, perform its local transaction, and send the response back to the items service. In case any local transaction fails, a compensating transaction will be invoked. For example, if inventory service updates the inventory and sends the response back to the item service, but the item service fails to send data to the user, it will send a compensating transaction to inventory service for reverting its database to its original state.
Orchestration involves a centralised service called the orchestrator that receives each request and contains the business logic to decide which service to call next based on the request. The advantages this brings is that it is easier to monitor the transaction through the orchestrator, it is easier to maintain and test the orchestrator as responsibilities are more segregated between the services. The drawbacks are that the orchestrator can become a single failure point for the transactions and must be individually scaled and replicated, increasing coupling between the services.
Orchestration based sagaSummary
The database per service pattern allows services to have their own databases instead of a single shared database. This makes the services loosely coupled, and they can be individually developed, deployed, tested and scaled. Transactions that require data from more than one service are implemented using API composition. One pattern that can be used to implement this is the Saga pattern. It defines concepts like choreography and orchestration which are different ways to implement transactions that span services.
Designing Microservices Using Database Per Service and Saga Patterns was originally published in Walmart Global Tech Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.