Persistence "model adapter" for separating models and upcasting

[ktoso]

Add a "man in the middle" between PersistentActor and the Journal which can be used to.

No API is yet decided, first we'll need to draft a few example impls.

Use cases this adapter should cover (possibly all, or we split it up, not sure yet):

    "domain model to data model" case
                                       +---------+
+---------+     domain => BSON         | journal |
|persist()+---------------------------->         |
+---------+                            |         |
                                       |         |
           <---------------------------+         |
              BSON => domain           |         |
                                       |         |
                                       +---------+

• give users more control over "domain model to db model mapping", see also Make Serializer more friendly with persistence #17434
• this allows for Domain => BSON Store journal messages / snapshots in BSON ironfish/akka-persistence-mongo#88
• allows for "pass through" serialization as in "if my Journal understands BSON, I can give it BSON!"

    "upcasting events" case

+                                  +---------+
<----------------------------------+ journal |
|     model-v1 => [0..n] model-v2  |         |
|                                  |         |
|                                  +---------+
+                                      

• provide a place where event upcasting on recovery could be performed
  • figure out if/how/which we support: bytes => logic => upcasted event and/or old-model => upcasted-model
  • some inspiration to be gained from [http://www.axonframework.org/docs/2.4/single.html#event-upcasting](Axon upcasters)
  • upcasting may require 1 "v1 event" to emit more than 1 "v2 event", example is when migrating from ClientDetailsChanged to AddressChanged + PersonalDataChanged
• if not needed, it should be an identity pass through by default
  • if needed by the Journal, the user must configure it in code - as it's a function and could be combined based on some logic with more upcasters (e.g. when I have "domain-module-a" and "domain-module-b" and both provide adapters for their classes)
  • if needed, and not configured the journal should throw exceptions eagerly

                domain-event                          
                    =>                 +---------+
+---------+     tagged(domain-event)   | journal |
|persist()+---------------------------->         |
+---------+                            |         |
                                       |         |
                                       |         |
                                       |         |
                                       |         |
                                       +---------+

• additional: since tagging is also journal specific, some journals need it and some not, this seems to be a good place to encourage to use for tagging as well, as it's bound to the journal, and gets the event so it can extract tags

[olger]

Could explain a bit what in 'identity pass through by default' the identity is ?

[olger]

Is the basic idea to use BSON for the payload serialisation ? It seems that at this moment that protobuf is part of the serialisation. Should that be removed ?

[ktoso]

BSON is just an example, inspired by the issue linked above (the mongodb journal), it could be any format that the journal plugin understands. It could also be that the adapter just passes the object through, and the journal plugin performs the serializaiton inside.

[ktoso]

Closing this ticket, we decided that serializers are powerful enough to express all migrations we need.

Will be replaced by:

• 1:n handling in serializers by special return type
• come back of the Tagger Provide metadata extractor / tagger infrastructure for Persistence #17617

[ahjohannessen]

@ktoso I like the ideas you have lined up above. What do you mean by come back of Tagger ?

[ktoso]

By Tagger I meant an additional facility which can be asked to extract metadata about a given event. More info on it in #17617.

However after a few drafts of adapters and taggers we're back to adapters - they're more powerful and can do the generic "I want to separate data from domain completely" in a nice way, PR coming up right now.

[ahjohannessen]

@ktoso Cool stuff, I think this will make a difference 👍 Secondary indexes by event type / category is probably the most wanted feature in akka persistence and I think this will make it easier for journals like the eventstore to utilize their built-in features, e.g. projections by event type, without workarounds and custom hacks :)