Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Manifest-less jackson serializers #28113

Closed
jroper opened this issue Nov 4, 2019 · 3 comments
Closed

Manifest-less jackson serializers #28113

jroper opened this issue Nov 4, 2019 · 3 comments

Comments

@jroper
Copy link
Contributor

@jroper jroper commented Nov 4, 2019

Fully qualified class names can take up a lot of space, especially when attached as the manifest to every serialized event, message and distributed data element. Jackson has an in built feature that can assist here, that is its polymorphism support. Using a combination of @JsonTypeInfo and @JsonSubTypes, users can declare logical names for their messages, eliminating the need for fully qualified class names. If Akka were to allow the configuration of multiple Jackson serializers, each one for a single polymorphic supertype, then it could avoid persisting the classname in the manifest, if configured to do so. For example, it might look like this to configure it:

akka.serialization.jackson {
  serializers {
    user-events {
      disable-classname-in-manifest = on
      id = 12345
    }
  }
}

akka.actor {
  serializers {
    user-events = "akka.serialization.jackson.JacksonSerializer"
  }
  serialization-bindings {
    "com.example.organizationunit.usermanagement.persistence.UserEntity$Event" = user-events
  }
}

So, for all user entity events, the serialized id would be 12345, and there would be no manifest, it would be up to the user to ensure that Jackson handles the polymorphism appropriately. This mechanism could also be used to allow per root type configuration of the Jackson serializer, eg, it would let you specify migrations that only apply to user entity events, or particular Jackson object mapper features or modules.

@patriknw

This comment has been minimized.

Copy link
Member

@patriknw patriknw commented Nov 4, 2019

Good point.

One thing to be aware of is that the manifest in remote messages is optimized to an Int with a so called manifest compression mechanism. Same is used for recipient and sender ActorRefs.

It's already possible to configure several JacksonSerializer with different ids.

akka.actor {
  serializers {
    jackson-json = "akka.serialization.jackson.JacksonJsonSerializer"
    jackson-cbor = "akka.serialization.jackson.JacksonCborSerializer"
    jackson-json2 = "akka.serialization.jackson.JacksonJsonSerializer"
  }
  serialization-identifiers {
    jackson-json = 31
    jackson-cbor = 32
    jackson-json2 = 12345
  }
  serialization-bindings {
    "com.example.Jsonable" = jackson-json
    "com.example.Jsonable2" = jackson-json2
  }
}

So, if I understand this correctly the missing piece would be the disable-classname-in-manifest and instead use the class name from serialization-bindings when deserializing?

When deserializing there are two cases depending on if migrations are defined or not.

No migrations:

objectMapper.readValue(decompressedBytes, clazz)

With migrations:

val jsonTree = objectMapper.readTree(decompressedBytes)
val newJsonTree = transformer.transform(fromVersion, jsonTree)
objectMapper.treeToValue(newJsonTree, clazz)

Would this work with the the Jsonable2 as the clazz if @JsonTypeInfo and @JsonSubTypes have been defined, or is there something more needed for readValue or treeToValue?

@jroper

This comment has been minimized.

Copy link
Contributor Author

@jroper jroper commented Nov 6, 2019

Ah that's good to know, I did read the code but not carefully enough to see that it supported multiple serializers.

Yes, the class name from serialization-bindings would be used. Two serialization bindings to the one serializer with classnames disabled would be a fatal error on startup, there would be no way for Akka to resolve that. Migrations presumably would be supported by putting the version in the manifest with no classname.

An interesting case to consider might be the classname being disabled, but a manifest that contains a classname is present for the serializer. One option might be, if there is a classname in the manifest, but classnames are disabled, then a migration must be defined for that classname, and that migration would be expected to add the necessary type info into the JSON so that it can be deserialized as normal. Alternatively, we could just use the classname as is (with migrations applied) - users would then have the option of creating a migration for it that introduced the type information and returned the supertype from transformClassname.

I'm pretty sure the migrations will work fine, they'll just have to be defined for the supertype, which works because the information about which sub type it is can be read from the JSON tree, and so the migrations will know what they're dealing with. It would actually allow some neat stuff, like deleting sub types, a migration could then transform payloads of that sub type into another subtype, or even introducing sub types which were previously distinguished by a value rather than a type.

Also good to know about the manifest compression, I never knew about that until now.

@patriknw

This comment has been minimized.

Copy link
Member

@patriknw patriknw commented Nov 7, 2019

sounds like we could work out the details of this, sounds like a good improvement

@patriknw patriknw added this to Backlog in Akka 2.6.x Nov 18, 2019
jroper added a commit to jroper/akka that referenced this issue Dec 3, 2019
Implements akka#28113

This adds the ability to make Jackson serializers not output the class
name in the manifest in cases where type information is stored in a more
concise format in the JSON itself.
jroper added a commit to jroper/akka that referenced this issue Dec 3, 2019
Implements akka#28113

This adds the ability to make Jackson serializers not output the class
name in the manifest in cases where type information is stored in a more
concise format in the JSON itself.
patriknw added a commit that referenced this issue Dec 5, 2019
* This adds the ability to make Jackson serializers not output the class
  name in the manifest in cases where type information is stored in a more
  concise format in the JSON itself.

* documentation about rolling updates

* Ensure programatic bindings are search in Jackson serializer
@patriknw patriknw added this to the 2.6.1 milestone Dec 5, 2019
@patriknw patriknw closed this Dec 5, 2019
Akka 2.6.x automation moved this from Backlog to Done Dec 5, 2019
navaro1 added a commit to navaro1/akka that referenced this issue Dec 17, 2019
* This adds the ability to make Jackson serializers not output the class
  name in the manifest in cases where type information is stored in a more
  concise format in the JSON itself.

* documentation about rolling updates

* Ensure programatic bindings are search in Jackson serializer
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Akka 2.6.x
  
Done
2 participants
You can’t perform that action at this time.