Repository navigation
MicroService Implementation & Structure
On the first page of this series, we talked about the main components of a service and discussed how services can integrate with and talk to each other. On the second page, we drilled into the service contract and focused on the behavior of a service as a message-processing unit in the service landscape. On this page, we will look at the internal structure of a MicroService, which is composed of several different layers. We will explain which layers are present in a MicroService, how they connect and which role each layer plays when processing different types of messages.
The primary architecture of a MicroService is the Onion Architecture, which is a specific kind of layered architecture. In layered architectures, a system or component in sliced up into a stack of different layers, where each layer has a very specific responsibility and provides very specific features to its upper layer(s). The most basic functions are provided by the lowest layer, and as you move up the stack, each layer provides functions and capabilities of a higher order than its lower layers, thereby using and depending upon the functions and features of the layer(s) below it.
To illustrate how the main service components map to the layers of a MicroService, recall from the first part of this series that a MicroService can basically be divided up into front-end and back-end components:

If we remodel this diagram a bit, we can actually argue that a MicroService uses a simple client-server architecture, where the client is represented by the front-end components and the server is represented by the back-end components:

Here, the clients-package maps to the so-called Presentation Layer, which is the layer where all user interface components go. The server-package, however, does not map to a single layer, but can be divided up into three different layers, which are the following (shown here from highest to lowest):
- The Endpoint Layer, which is responsible for communication and integration with clients and other services, thereby exposing all features of the back-end. In other words, this is the layer that will receive all messages from clients (i.e. the Presentation Layer), other services and/or service buss and is also responsible for sending/publishing messages to other services. The components that will send and receive these messages are called endpoints, hence the name of this layer.
- The Data Access Layer, which provides access to the service's private data, typically stored in one or more databases, files and other types of persistent storage.
- The Business Logic Layer, which contains all (business) logic to perform mutations to the service's private data and is responsible for publishing the appropriate (business) events as a result.
Furthermore, the Business Logic Layer can itself be divided up into two separate layers:
- The Application Layer, which is responsible for coordinating the required mutations upon receiving specific Commands and Events.
- The Domain Layer, which contains the data model or rich domain model to which the mutations are performed.
Having identified all the layers of a MicroService, we can now model the architecture of a MicroService as follows:

Note that the contract of a service does not represent nor is any part of specific layer, because it has no processing logic.
Although layered architectures are traditionally visualized as a stack, which is sort of reflected in the image above, the architecture of a MicroService is typically visualized as a hexagon, where the lowest layer represents the core of the service and the outer layers surround it as rings of an onion, hence the name Onion Architecture:

In the following few paragraphs, we are going to explain the roles and responsibility of each layer a bit further by describing how different types of messages are processed by a MicroService.
When a service processes a Command, that Commands is effectively received by the Endpoint Layer. The component that has sent the Command is called the sender and can be a client, which typically sends Commands synchronously through, for example, a http-request, or another service, which typically sends its Commands asynchronously, as we will get back to later.
As discussed earlier, the Endpoint Layer first authorizes (the sender of) that Command - does it have sufficient right to execute this Command? When the authorization succeeds it then validates the Command - does it contain all required data and do all values satisfy the declared constraints? The Endpoint Layer performs these checks so that the Command can be discarded early if one of those checks fail. When both checks pass, the Endpoint Layer typically opens up a transaction so that all mutations are performed atomically (and can be rolled back in case of an error). Before the Command is processed any further, however, the Endpoint Layer is also allowed to get some data from other services. This is optional, but for some Commands the service needs information from other sources, and since the Endpoint Layer is the layer connected to the network, it is this layer's responsibility to obtain that information.
It then passes the Command to the Data Access Layer. This layer will retrieve all the data that is required for the operation, typically in the form of entities (i.e. parts of the domain model) and passes that data to the Business Logic Layer for further processing.
The Business Logic Layer takes this data and will perform all necessary changes/mutations according to the specifications of the Command. As it makes these changes, it will also produce the Events that reflect those changes. Note that the Business Logic Layer does not actually publish these Events - it just produces them. This is because the transaction may still fail and these Events may only be published after the transaction has completed successfully. The Business Logic Layer is also allowed to produce Commands that are to be sent to other services. Similar to the Events, though, these Commands are only scheduled to be sent, because the transaction must be completed first.
When the Business Logic Layer has done its job, control is shifted back up to the Data Access Layer. At this stage, the Data Access Layer will save all changes to the data store of the service. After the changes have been persisted, the Endpoint Layer will commit the transaction and send and/or publish all Commands and Events produced by the Business Logic Layer. It finally completes the operation by returning an OK-response to the sender of the Command so that it knows the operation has been completed successfully. The whole process is visualized in the diagram below.

A general overview of processing a Command.
Note that when a service outputs a batch of Commands that are to be sent to other services, it is recommended that these Commands are sent asynchronously, e.g. by placing them into a (persistent) queue. This way, the run-time independence between services is maintained. Moreover, on the previous page we already explained that you should only send Commands from one service to another if it is very likely that this Command will be processed successfully or that a failure to process this Command does not cause problems in the business process. Both premises imply that the sending service does not have to await the outcome of the operation and is enabled to send to asynchronously using a queue. You'll read more on this on the next page.
When a Command is executed successfully, we talk about the happy flow. At this point, however, we know that there are several places in the process where a functional error can occur. For starters, the authorization or validation steps may fail. In that case, the Endpoint Layer stops processing the Command immediately, logs the error, and returns an appropriate error to the sender. When this happens, we refer to it as an alternate flow.
The second place a Command can fail is in the Data Access Layer when it attempts to retrieve all required data from the private data store: if the Command references entities that don't exist, the read operation fails, which is a functional error, because apparently the Command contains an incorrect reference.
Suppose, for example, that a PlaceOrderCommand is being processed and that the Command has a property UserId that should refer to the User who is placing that order. The Data Access Layer will then attempt to retrieve that User by this UserId, but it will fail if no User with that UserId is found in the data store. Consequently, the Data Access Layer will throw an exception reporting the problem and abort the operation.
The third opportunity a Command may fail is when it is being processed by the Business Logic Layer. In this phase, a functional error will occur when a business rule is violated. Again, the error will be reported by throwing an exception explaining which business rule was violated. An example of this is if a specific User attempts to place an order while he is no longer allowed to place new Orders because his account has been blocked for some reason.
The last opportunity where a functional error can occur is when the Data Access Layer attempts to save all changes back to the data store and then discovers that another operation has mutated the same data while the operation was in progress. We call this a concurrency conflict or concurrency error and it prevents the Data Access Layer from saving the changes, because this may corrupt the integrity of the data (e.g. changes of the previous operation may be overwritten). As a result, the Data Access Layer will throw an appropriate exception to report the error and abort the operation.
To detect concurrency conflicts, the domain model data typically assigns specific timestamps or version numbers to entities which are updated with every change. When retrieving these entities, the original values of these timestamps or version numbers are kept in memory, so that when saving the data back to the data store, the Data Access Layer can compare the original value of those timestamps or version numbers with the current value in the data store. If they are different, another process has updated the same data, implying a concurrency conflict has occurred.
No matter when or where an error occurs in the process, the Endpoint Layer will eventually catch it and handle it by rolling back the transaction, logging the error and by returning an appropriate error-response to the sender of the Command. As such, the sender of the Command is informed about the problem and can take action accordingly (e.g. by showing the error to the user or by simply redelivering the Command).
The diagram below shows the process of executing a Command again, but now labeled with all possible points of failure:

A general overview of all places where a functional error can occur while processing a Command.
Referring back to the previous page, we already pointed out that processing an Event is very similar to processing a Command, but that there are a few subtle differences, especially related to the opportunity of functional errors to occur.
The first difference is that Events are never send directly from a client or other service but always delivered (asynchronously) by a service bus. Secondly, the Endpoint Layer doesn't have to authorize or validate Events, because they come from a trusted source and because they don't represent a request that can be denied for such reasons. This means that the happy flow of handling an Event can be visualized as follows:

Furthermore, when an Event arrives at the Data Access Layer, and that layer attempts to read data from the data store, it should be very unlikely that the Event contains references to entities that do not exist. If they do, it typically means that the service that produced the Event contains a bug (e.g. doesn't set the reference correctly) or that there was a problem with serialization and deserialization of the Event which caused the reference to be lost or corrupted. In the exceptional case that an entity got deleted after the Event* that refers to it was published, then the service handling the Event should account for this and handle the situation gracefully, i.e. by ignoring the Event or process it without the referenced data. In other words: in no situation should the Data Access Layer throw an exception here.
*It is, as a matter of fact, considered bad practice to ever hard delete business data. There are two main reasons for this. First of all, other data in the system (or even outside of it) could still contain references to it, which become corrupted if the data is removed. Secondly, even old data that will never be mutated again (e.g. closed accounts) still hold business value for historic reasons.
Similarly, Events could never violate a business rule. Remember that Events are plain facts that a service chooses to act upon. If an Event ever causes a business rule to be violated, it means there is a bug in your code and the error should actually be treated as a technical error.
Concurrency conflicts are a different story, however. Whether the service is handling a Command or Event, there is always that chance that another operation is accidentally mutating the same data and the Data Access Layer finds out when attempting to save those changes back to the data store. On the previous page we already mentioned that this is the only situation in which a functional error can occur when handling an Event. Incidentally, though, it is also the only type of functional error that can be remedied by simply redelivering the message to the service and trying to process it again. Hence, to the service bus delivering the message it doesn't really matter whether or not the service fails to process an Event because a functional error (concurrency conflict) or technical error occurred: it can always just attempt to redeliver the Event (following a specific retry policy) until the Event is either handled successfully or the maximum number of retry attempts is reached and the Event is dead-lettered:

The only type of function error that can occur when handling an Event are concurrency errors.
The last thing to look at are Queries. Since Queries represent requests like Commands do, their Request-message is authorized and validated by the Endpoint Layer. In addition, both Commands and Queries can be sent from a client or another service. However, whereas Commands can be either sent synchronously (by a client) or asynchronously (by another service), Queries are always sent synchronously, because by definition, the sender awaits the response (i.e. the requested data).
The biggest difference between Commands and Queries is, of course, that Queries just retrieve data and don't mutate business data. For this reason, the Business Logic Layer is not involved when processing a Query. As in all other cases, the Endpoint Layer is still allowed to assist by retrieving data from other services, but the Data Access Layer is meant to contain the core logic of a Query by simply reading the required data from the service's private data store. In this step, the Request-message is typically used to filter and sort the data as required. When all data has been gathered, the Data Access Layer builds the Response, hands it back to the Endpoint Layer, which then returns this Response to the sender:

Queries just read and return data from the service's private data store; as such, they don't touch the Business Layer.
Note that the Endpoint Layer may still open up a transaction in which the Query is executed, even though no changes are made in the data store. The reason for doing this is that a Query may make multiple round-trips to the data store(s) to gather and aggregate all data, and by executing the whole Query inside a single transaction (with an appropriate isolation level), the integrity is the related data can be guaranteed. That being said, it is typically preferred to optimize Queries as much as possible and to retrieve all data in a single round-trip to the data store (e.g. by using a single stored procedure, view, table or file per Query).
Since Queries represent requests, there a various cases where functional errors can occur. Needless to say, authorization and validation of the Request-message can fail, resulting in an immediate error to be returned to the sender. Authorization errors are, however, a bit less common compared to Commands, because instead of returning errors, Queries can also be implemented such that they only return data that a particular user is authorized to obtain, effectively filtering the data rather than blocking the entire request.
Secondly, the Request-message can, similar to Commands, contain illegal references to data. For example, if a service processes a GetUserProfileRequest that contains a UserId of a particular user and no user with that UserId exists, the Data Access Layer may throw an appropriate error (analogous to a 404 Not Found of the http-protocol). Once the Response-message has been created, there are no more opportunities for functional errors to occur, which means the alternate flows of Queries can be visualized as follows:

Functional errors can occur when executing a Query, but there are fewer places where they can occur compared to Commands.