Configure grain state serializer per storage provider

[benjaminpetit]

Right now, for the Azure providers at least, it's not possible to use a custom serializer: you only have the choice between Newtonsoft.Json or the Orleans SerializationManager.

Users can configure the SerializationManager to use a custom serializer for certains types, but this method has two main limitations:

• The custom serializer will be used for wire-transmission
• The output is still padded by some binary content from SerializationManager.

This PR enable configuration of custom serializer for grain state per storage provider.

The base interface for these serializer is IGrainStorageSerializer. In this PR I included 3 implementations:

• JsonGrainStorageSerializer: an implementation that uses Newtonsoft.Json
• OrleansGrainStorageSerializer: an implementation that uses the Orleans SerializationManager
• GrainStorageSerializer: an implementation that use a primary Serializer, but capable of deserialize multiple formats.

The idea of this interface is that it's easy to compose. The Serialize method will return a tag, that will be used to identify the type of the serializer when calling Deserialize.

It should be easy to add custom implementation that read from a config, or from class Attributes for example.

Here is the new way to configure the Azure table storage provider:

siloBuilder.AddGrainStorage(name, configure =>
{
    configure.UseOrleansSerializer(); // Configure the serializer here
    configure.UseAzureTable(options => options.ConnectionString = "xxx");
});

For now, old configuration method are still available, and the GrainStorageSerializer will be used by default. I think for 4.0 we should remove this and ask the user to configure explicitely a serializer.

The configuration may seems more complicated, but maybe we can add some configuration helper methods like that:

siloBuilder.UseAzureTable<JsonGrainStorageSerializer>(options => options.ConnectionString = "xxx")

for serialization implementation that requires none or few configuration steps.

[veikkoeeva]

When is this expeced to merge? I think the ADO.NET ones should be brought on par (Azure and otherwise) and maybe at least I should use a bit of time to understand what ought to be changed.

The following is a short background for questions in the end.

It looks to me this affects at least how a (de)serializer is configured. It is not clear to me how this affects how during running the application the IGrainStorage implementation is affected.

It looks to me, I am not sure, this affects the configuration and what and how configuration is provided to the IGrainStorage implementation, but it is then provider implementation specific how it actually works.

The reason I think this is that currently in ADO.NET allows one to choose dynamically from the provided (de)serializers with several parameters such as grain type, persisted data type and even payload if needed. The interface is at https://github.com/dotnet/orleans/blob/master/src/AdoNet/Orleans.Persistence.AdoNet/Storage/Provider/IStorageSerializationPicker.cs#L81.

In practice this is fixed to JSON, XML or binary during configuration as passing this selector implementation from options to the ADO.NET IGrainStorage implementation is very difficult. Hence bother you a bit @benjaminpetit that do you have a minute to spear and and tell if is now possible to configure such a custom (de)serializer selector for the provider and pass it via options to IGrainStorage implementation? :)

In effect I am asking if it is possible now and should we coordinate on the interface to achieve the following:

1. Be able to provide a selector function that can be used when Orleans is running. Requires looking at parameters and what granularity the allow.
2. Configure (de)serializers separately as a set from which the selectors in 1. choose from.
3. Design how would look a convenience function that selects something like "NewtonsoftJson for all" by default.

I would like to remove references to specific sets of (de)serializer libraries and this selector function load whatever the user likes to use. So basically in the options would look like having a selector function and a matching (de)serializer implementation. By default a convenience method could be used that implements JSON, XML and binary as is done currently. Maybe the convenience functions could look like in "Azure implementation".

I appreciate decoupling the selector function for (de)serializer may not be the goal on these loaders, but I am also thinking that if it very "easily doable", maybe it's possible to coordinate a bit here with interfaces.

[benjaminpetit]

When is this expeced to merge?

Not in a rush, but it should be in for 4.0

I think the ADO.NET ones should be brought on par (Azure and otherwise) and maybe at least I should use a bit of time to understand what ought to be changed.

Yes! That's the plan.

The reason I think this is that currently in ADO.NET allows one to choose dynamically from the provided (de)serializers with several parameters such as grain type, persisted data type and even payload if needed.

Ideally, it should be doable with a custom implementation of IGrainStorageSerializer.

From IStorageSerializationPicker, serviceId, storageProviderInstanceName could be injected from the implementation constructor. But the grainType is missing, and I can see that it could be very useful, so we could add that to the IGrainStorageSerializer` interface.

Ideally, I think IGrainStorageSerializer and IGrainStorage configuration should be independant. During the IGrainStorage configuration you should point to a IGrainStorageSerializer to use, and that's it. It's up to the IGrainStorage to use the payload and tags from/to the serializer.

Let's sync on gitter or Teams to address your concern :)