Skip to content

Commit

Permalink
change most diagrams from pictures to mermaid
Browse files Browse the repository at this point in the history
this way they are in the document in an editable way, instead of as
binary, locked files.
  • Loading branch information
TomasEkeli committed Jun 22, 2023
1 parent 13ac74a commit 19f4b67
Show file tree
Hide file tree
Showing 8 changed files with 123 additions and 18 deletions.
16 changes: 13 additions & 3 deletions Source/content/en/docs/concepts/event_handlers_and_filters.md
Expand Up @@ -25,7 +25,10 @@ Each processor processes events within a single [scope]({{< ref "event_store#sco

The filter is a processor that creates a new stream of events from the [event log]({{< ref "event_store#event-log" >}}). It is identified by a `FilterId` and it can create either a partitioned or unpartitioned stream. The processing in the filter itself is however not partitioned since it can only operate on the event log stream which is an unpartitioned stream.

![Filter](/images/concepts/filter.png)
```mermaid
flowchart LR
EL[(Event Log)] --> StreamProcessor --> F[EventProcessor<br/>Filter code] --> S[(Stream)]
```

The filter is a powerful tool because it can create an entirely customized stream of events. It is up to the developer on how to filter the events, during filtering both the content and the metadata of the event is available for the filter to consider. If the filter creates a partitioned stream it also needs to include which partition the event belongs to.

Expand All @@ -39,8 +42,15 @@ Since there are [two types of streams]({{< ref "streams.md#public-vs-private-str

The event handler is a combination of a filter and an event processor. It is identified by an `EventHandlerId` which will be both the id of both the filter and the event processor.

![Event Handler](/images/concepts/eventhandler.png)

```mermaid
flowchart LR
subgraph implicit filter
direction LR
EL[(Event Log)] --> FSP[StreamProcessor] --> F[Filter based on<br/>EventType] --> S[(Stream)]
end
S --> SP[StreamProcessor]
SP --> EP["EventProcessor<br/>Handle() function"]
```
The event handler's filter is filtering events based on the [`EventType`]({{< ref "events.md#event-type" >}}) that the event handler handles.

Event handlers can be either partitioned or unpartitioned. Partitioned event handlers uses, by default, the [`EventSourceId`]({{< ref "event_sourcing#event-source-id" >}}) of each event as the partition id. The filter follows the same rules [for streams]({{< ref "streams#rules" >}}) as other filters.
Expand Down
17 changes: 15 additions & 2 deletions Source/content/en/docs/concepts/event_horizon.md
Expand Up @@ -6,7 +6,20 @@ weight: 15

At the heart of the Dolittle runtime sits the concept of Event Horizon. Event horizon is the mechanism for a microservice to give [Consent]({{< ref "#consent" >}}) for another microservice to [Subscribe]({{< ref "#subscription" >}}) to its [Public Stream]({{< ref "streams#public-vs-private-streams" >}}) and receive [Public Events]({{< ref "events#public-vs-private" >}}).

![Anatomy of an Event Horizon subscription](/images/concepts/eventhorizon.png)
```mermaid
flowchart BT
subgraph Producer
ProdEventLog[(Event Log)] -->|Public events| PublicFilter[Public Filter]
PublicFilter -->|matches go into| PublicStream[(Public Stream)]
Consent(((Consent))) -->|gives access to| PublicStream
end
subgraph Consumer
direction LR
Subscription(((Subscription)))
Subscription -->|stores| ConEventLog[(Scoped Event Log)]
end
Subscription -->|asks for events| Consent
```

## Producer

Expand Down Expand Up @@ -39,7 +52,7 @@ Subscription {
TenantId Guid
PublicStreamId Guid
PartitionId string
// the consumers scoped event log
// the consumers scoped event log
ScopeId Guid
}
```
Expand Down
15 changes: 14 additions & 1 deletion Source/content/en/docs/concepts/event_sourcing.md
Expand Up @@ -11,7 +11,20 @@ Event Sourcing is an approach that derives the current state of an application f

Here's an overview of Event Sourcing:

![Basic anatomy of event sourcing](/images/concepts/eventsourcing.png)
```mermaid
flowchart TB
P[Presentation] --> DP[/DishPrepared/]
subgraph write
DP --> RA[/RecipeAdded/] --> DATM[/DishAddedToMenu/]
DATM --> ES[(Event Store)]
end
ES --> Ext([External Systems])
subgraph read
ES --> Consumer -->|Generates the read cache| RC[(Read Cache)]
end
RC -->|Query for read data| P
```

## Problem
A traditional model of dealing with data in applications is [CRUD](https://en.wikipedia.org/wiki/Create,_read,_update_and_delete) (create, read, update, delete). A typical example is to read data from the database, modify it, and update the current state of the data. Simple enough, but it has some limitations:
Expand Down
15 changes: 13 additions & 2 deletions Source/content/en/docs/concepts/events.md
Expand Up @@ -12,7 +12,7 @@ An event is a change (fact) within our system. The event itself contains all the

More usually, it is a simple Data Transfer Object (DTO) that contains state and properties that describe the change. It does not contain any calculations or behavior.

## “that has happened”
## “that has happened”
As the event has happened, it cannot be changed, rejected, or deleted. This forms the basis of [Event Sourcing]({{< ref "event_sourcing" >}}) If you wish to change the action or the state change that the event encapsulates, then it is necessary to initiate an action that results in another event that nullifies the impact of the first event.

This is common in accounting, for example:
Expand Down Expand Up @@ -90,7 +90,18 @@ For the Runtime, the event is just a JSON-string. It doesn't know about the even

This diagram shows us a simplified view of committing a single event with the type of `DishPrepared`. The Runtime receives the event, and sends it back to us to be handled. Without the event type, the SDK wouldn't know how to deserialize the JSON message coming from the Runtime.

![Flow of committing an event type](/images/concepts/eventtype.png)
```mermaid
sequenceDiagram
participant SDK
participant Runtime
participant Event Store
SDK->>Runtime: Commit(DishPrepared)
Runtime->>Event Store: Serialize the event into<br/>JSON and save it
Runtime->>SDK: Commit successful
Runtime->>Runtime: Process the event in<br/>handlers and filters
Runtime->>SDK: Send the JSON of the event<br/>to the event-handler
SDK->>SDK: Deserialize according to EventTypeId<br/>found in JSON and call on the handler
```

Event types are also important when wanting to deserialize events coming from other microservices. As the other microservice could be written in a completely different programming language, event types provide a level of abstraction for deserializing the events.

Expand Down
30 changes: 27 additions & 3 deletions Source/content/en/docs/concepts/overview.md
Expand Up @@ -33,11 +33,20 @@ Dolittle applications are built from microservices that communicate with each ot
## Microservice
A _microservice_ consists of one or many heads talking to one Runtime. Each microservice is autonomous and has its own resources and [event store]({{< ref "event_store" >}}).

The core idea is that a microservice is an independently scalable unit of deployment that can be reused in other parts of the software however you like. You could compose it back in one application running inside a single process, or you could spread it across a cluster. It really is a deployment choice once the software is giving you this freedom.
The core idea is that a microservice is an independently scalable unit of deployment that can be reused in other parts of the software however you like. You could compose it back in one application running inside a single process, or you could spread it across a cluster. It really is a deployment choice once the software is giving you this freedom.

This diagram shows the anatomy of a microservice with one head.

![Example anatomy of a Dolittle microservice](/images/concepts/anatomy.png)
```mermaid
flowchart LR
Frontend --> Backend
subgraph Head
Backend --> SDK
end
SDK --> Runtime
Runtime --> ES[(Event Store)]
Runtime --> RC[(Read Cache)]
```

{{< alert title="Read Cache" color="info" >}}
The _Read Cache_ in these pictures is not part of Dolittle. Different [projections]({{< ref "event_sourcing#projections" >}}) call for different solutions depending on the sort of load and data to be stored.
Expand All @@ -49,7 +58,22 @@ Since computing is the most expensive resource, the Dolittle Runtime and SDK's h

This diagram shows a microservice with 2 tenants, each of them with their own resources.

![Example of multi-tenant microservice](/images/concepts/multitenant.png)
```mermaid
flowchart LR
Frontend --> Backend
subgraph Head
Backend --> SDK
end
SDK --> Runtime
Runtime --> ES1[("Tenant 1
Event Store")]
Runtime --> RC1[("Tenant 1
Read Cache")]
Runtime --> ES2[("Tenant 2
Event Store")]
Runtime --> RC2[("Tenant 2
Read Cache")]
```

## What Dolittle isn't
Dolittle is not a traditional backend library nor an event driven message bus like [Kafka](https://kafka.apache.org/). Dolittle uses [Event Sourcing]({{< ref "event_sourcing" >}}), which means that the state of the system is built from an append-only [Event Store]({{< ref "event_store" >}}) that has all the events ever produced by the application.
Expand Down
28 changes: 27 additions & 1 deletion Source/content/en/docs/concepts/projections.md
Expand Up @@ -10,7 +10,33 @@ Read models defines the data views that you are interested in presenting, while

Example of a projection:

![Diagram of projections](/images/concepts/projections_v2.png)
```mermaid
flowchart LR
subgraph Business moments
direction LR
CR["Customer Registered<br/>Id: 123<br/>Name: John Doe"]
DAO["Debit Account Opened<br/>Id: 456<br/>Balance: 0"]
DP["Debit Performed<br/>Account: 456<br/>Amount: $20"]
WP["Withdrawal Performed<br/>Account: 56<br/>Amount: $10"]
end
subgraph Operations
CR --> O1["Customer = Id<br/>Name = Name"]
DAO --> O2["Id = Id<br/>Type = Debit"]
DP --> O3["Id = Id<br/>Amount += Amount"]
WP --> O4["Id = Id<br/>Amount -= Amount"]
end
subgraph Read Model
O1 --> RM
O2 --> RM
O3 --> RM
O4 --> RM["Account Details
Id: 456
Type: Debit
Customer: 123
Name: John Doe
Balance: $10"]
end
```

## Read model

Expand Down
10 changes: 9 additions & 1 deletion Source/content/en/docs/concepts/tenants.md
Expand Up @@ -16,7 +16,15 @@ In a multi-tenant application, the same instance of the software is used to serv

Multi-tenancy allows for easier scaling, sharing of infrastructure resources, and easier maintenance and updates to the software.

![Simple explanation of multi tenancy](/images/concepts/multitenant-explanation.png)
```mermaid
flowchart TB
T1((Tenant A)) --> A[Application]
T2((Tenant B)) --> A
T3((Tenant C)) --> A
A --> DB1[(Tenant A)]
A --> DB2[(Tenant B)]
A --> DB3[(Tenant C)]
```

## Multi-tenancy in Dolittle

Expand Down
10 changes: 5 additions & 5 deletions Source/content/en/docs/tutorials/event-horizon.md
Expand Up @@ -67,7 +67,7 @@ A public filter is defined as a method that returns a partitioned filter result,
- a boolean that says whether the event should be included in the public stream
- a partition id which is the [partition]({{< ref "docs/concepts/streams#partitions" >}}) that the event should belong to in the public stream.

Only public events get filtered through the public filters.
Only public events get filtered through the public filters.

{{< tabs name="public-filter-tab" >}}
{{% tab name="C#" %}}
Expand Down Expand Up @@ -215,7 +215,7 @@ using Microsoft.Extensions.Hosting;
Host.CreateDefaultBuilder()
.UseDolittle(_ => _
.WithEventHorizons(_ => _
.ForTenant(TenantId.Development, subscriptions =>
.ForTenant(TenantId.Development, subscriptions =>
subscriptions
.FromProducerMicroservice("f39b1f61-d360-4675-b859-53c05c87c0e6")
.FromProducerTenant(TenantId.Development)
Expand Down Expand Up @@ -275,7 +275,7 @@ import { DishPrepared } from './DishPrepared';
Now we have a consumer microservice that:
- Connects to another Runtime running on port `50055`
- Subscribes to the producer's public stream with the id of `2c087657-b318-40b1-ae92-a400de44e507` (same as the producer's public filter)
- Puts those events into a [Scope]({{< ref "docs/concepts/event_store#scope" >}}) with id of `808ddde4-c937-4f5c-9dc2-140580f6919e`
- Puts those events into a [Scope]({{< ref "docs/concepts/event_store#scope" >}}) with id of `808ddde4-c937-4f5c-9dc2-140580f6919e`
- Handles them incoming events in a [scoped event handler]({{< ref "docs/concepts/event_handlers_and_filters#scope" >}}) with an id of `6c3d358f-3ecc-4c92-a91e-5fc34cacf27e`

There's a lot of stuff going on the code so let's break it down:
Expand Down Expand Up @@ -306,7 +306,7 @@ We'll see this reflected in the `docker-compose.yml` file [later]({{< ref "#setu
```csharp
// Program.cs
.WithEventHorizons(_ => _
.ForTenant(TenantId.Development, subscriptions =>
.ForTenant(TenantId.Development, subscriptions =>
subscriptions
.FromProducerMicroservice("f39b1f61-d360-4675-b859-53c05c87c0e6")
.FromProducerTenant(TenantId.Development)
Expand Down Expand Up @@ -520,7 +520,7 @@ services:
- 27017:27017
logging:
driver: none

consumer-runtime:
image: dolittle/runtime:latest
volumes:
Expand Down

0 comments on commit 19f4b67

Please sign in to comment.