Eventual Consistency with Integration Events using RabbitMq
A common consistency paradigm in distributed systems, including microservices architecture,
is eventual consistency. Changes made to a system are not instantly propagated to all of the nodes in the system
according to this model. Instead, the modifications are eventually propagated, which means that eventually,
all nodes will show the modifications.
In a microservices architecture, each microservice manages its own data store, which can lead to inconsistencies
between different microservices' views of the same data. To address this issue, microservices often use
eventual consistency to ensure that all microservices eventually reflect the same data.
In real-world applications, eventual consistency is achieved using methods like message passing,
and versioning. For instance, a microservice might broadcast a message to all other microservices informing them
of an update to its data store. Then, each microservice can modify its own data repository to incorporate
the modification.
All microservices may not immediately receive the notification and update their data stores.
Inconsistencies might exist during this period, but they will eventually get resolved as all microservices
update their data stores.
In SuuCat, Eventual Consistency has been implemented using RabbitMq and MassTransit.
When a microservice needs to update its data store, it publishes an event to a message broker.
Then, other microservices can subscribe to the event and update their data stores accordingly.
Think of it as a topic message sent by Azure Service Bus, but here RabbitMq is used.
In the real world, you will most likely be using Azure Service Bus or SQS-EventBridge in AWS.
In the future, I plan to do a version of this with Azure Service Bus.
Health checking can be installed using the following Docker files. More information about the installation is here: Getting Started
https://github.com/ebubekirdinc/SuuCat/blob/master/docker-compose.yml
Now that we have the project working, let's see how we use Eventual Consistency in SuuCat. As you can see in the
following code, the SignUp method is an HTTP POST method that creates a new user in the Identity Microservice.
When a new user is created, an event named UserCreatedEvent is published using the message broker: RabbitMQ.
The UserCreatedEvent contains information about the newly created user, including the user ID, email, and username.
The UserCreatedEventConsumer class in Account Microservice is a consumer that listens for UserCreatedEvent messages
and updates the local database in response. The Consume() method of this consumer checks if the user already exists
in the local database. If the user doesn't exist, the consumer creates a new user entity in the local database
using the information contained in the UserCreatedEvent.
The new user is then saved to the database.
This implementation of Eventual Consistency ensures that all microservices eventually have consistent user data.
The SignUp method creates a new user in one microservice and publishes an event to notify other microservices of the new user.
The UserCreatedEventConsumer listens for this event and updates the local database accordingly.
However, it may take some time for the event to propagate to all microservices and for the database to be updated.
As mentioned earlier, during this time, inconsistencies may exist, but they will eventually be resolved as all microservices
eventually update their local databases.