Event-sourcing and Event-storing

[CodeWithKashif]

1. Here in WorkshopManagementEventStore database you have saved the events as complete serialize json object in one column(EventData)
   Instead of that how about saving data in normalized form with proper multiple columns for each field/property?
   How and when should we chose saving events as serialized object over classical normalized data.
   Is it an standard practice to save events as a json object?

2. I do understand about Event sourcing and Events Storing however i am not quite confident on that.
   Can you please share your brief thoughts in that context that how both("Event sourcing" and "Events Storing") are inter-related and how are different too.

[EdwinVW]

Question 1
The idea is that we want to be able to store any event without the need to change the writemodel schema. If we would deconstruct every event into separate columns, every time an event is added with a new field for which we have no column in the table, we need to update the writemodel schema with the new column and migrate the data. This is not what you want! So therefore I've used SQL Server for storing the events and used a single text column for storing the data of each event.

The format you use for storing the event-data in is a design choice. The serialization format is up to you (XML, binary, ProtoBuf, ...). For me, JSON works best because I can easily read the events in the database without any processing using a simple query. This is also nice for demos (but that is not applicable in a production scenario obviously). If saving space or performance would be an issue for you, ProtoBuf would be a better alternative for instance.

An alternative for storing the events could be a document database that can store JSON documents. The advantage of this alternative is that a document database often allows you to query the JSON data and builds indexes on the JSON properties. But because I use the database as an append-only event-stream, I don't need these capabilities. I always apply a strict distinction between storing state (in the event-store) and querying (on the viewmodel(s)). So SQL Server is fine for me and I've used that successfully in production scenarios.

Finally, there are also products out there that are specialized in storing event-streams and replaying from these streams (e.g. Event Store). The cool thing is they often also allow you to create projections that are executed every time an event is added to the stream. I've played around with products like that, but never used one in production (because I've never needed the capabilities).

Question 2
Event-sourcing and event-storing go hand in hand imo. Event-sourcing is the name of the pattern where the state of an object is always restored by replaying all the events in the event-store for that object. In order to do that, you need to persist the events somewhere (hence, event-storing).

Does that make sense?

[CodeWithKashif]

Thanks a lot for such a detailed response, very grateful to you!

Regarding Question 1
I am very much aligned with your thought now.
So this means for any production scenario - if we are using sql server for event storing then typically two/three tables tables would be enough to store most of the events in the system?

[EdwinVW]

I always use 2 tables per aggregate. An aggregate is a DDD term and represents a cluster of related domain objects that must be kept consistent as a whole. The aggregate handles this consistency. In Pitstop, the WorkshopPlanning is the planning for a single day and all the MaintenanceJobs planned on that day. Every aggregate is identified by a unique Id. For the WorkshopPlanning this is the date of the planning.

For string the state of the WorkshopPlanning aggregate, I use 2 tables. The first table contains a row for every WorkshopPlanning. It has the unique Id and the Version as columns. The second table contains the events for that aggregate (with a foreign key to the first table). With every update, I pass in the expected version (the version that was read from the database before the update was started) and the new version of the aggregate (+1 for every event that was created as a result of handling a command in the domain). This is de SQL Command for the update:

update WorkshopPlanning
set [CurrentVersion] = @NewVersion
where [Id] = @Id
and [CurrentVersion] = @CurrentVersion;

If this update does not update any rows (ROWCOUNT == 0), then the expected version is not the current version in the database. This way I implement optimistic concurrency control (to prevent multiple users from updating the same aggregate version). If the update does update a row (ROWCOUNT == 1), then I insert the event(s) into the events table:

insert WorkshopPlanningEvent ([Id], [Version], [Timestamp], [MessageType], [EventData]) 
values (@Id, @NewVersion, @Timestamp, @MessageType,@EventData);

I do this all in a single transaction to make sure every update is atomic.

You can see how I do this in code in the SaveWorkshopPlanningAsync method in the SqlServerWorkshopPlanningRepository.

[CodeWithKashif]

Yes sir @EdwinVW
I had seen that code and do understand that logic, that is really great!

[EdwinVW]

Yes, my answer was quite extensive.
The short answer to your question is: I use 2 tables per aggregate :)

[CodeWithKashif]

Regarding Question 2
(please correct me if i am wrong or something is missing)

Event sourcing helps in storing all the changes (events) to the system, rather than just its current/latest state, means we can persist different version(change history) for for any domain object.
So we can see the audit trail and or back track(roll back) to an early stage.

Questions:
(a)
So in future do you have any plan to use SAGA to demonstrate the roll back among multiple micro-services?

(b)
Kindly also explain the realtime usage of IsReplaying property

    public class WorkshopPlanning
    {
        /// <summary>
        /// Indication whether the aggregate is replaying events (true) or not (false).
        /// </summary>
        private bool IsReplaying { get; set; } = false;

and of course thanks again for your wonderful help and support - I am highly obliged and grateful to you!

[CodeWithKashif]

Sir @EdwinVW

Since it is closed now, so i have created a new issue for above comment #43