Add System.Text.Json IGrainStorageSerializer

[willg1983]

This PR aims to add support for System.Text.Json for Grain state persistence.

System.Text.Json is optimized for performance and low-allocation, leading to potential performance gains when reading and writing grain state in Orleans. This does require all grain state to be serializable with System.Text.Json, which may require some developer involvement as it is not as forgiving or flexible as Newtonsoft.

This PR adds System.Text.Json support for Orleans types commonly serialized in grain state using the same structure as Newtonsoft, therefore allowing migration to STJ without breaking compatibility with existing persisted grain state.

Support has been added for:

• ActivationId
• EventSequenceToken
• EventSequenceTokenV2
• GrainId
• AsyncStreamReference
• GrainReference
• GuidId
• IpAddress
• IpEndPoint
• MembershipVersion
• PubSubSubscription
• QualifiedStreamId
• SiloAddress
• StreamId
• UniqueKey

Each with unit tests to verify the instance serialized with Newtonsoft can be deserialized by the System.Text.Json converter, and the instance serialized with System.Text.Json can be deserialized by Newtonsoft.

This has meant maintaining backward compatibility even in cases where the existing Newtonsoft output was less than optimal.

A good example of this is StreamId, which Newtonsoft serializes as:

{{"$id":"1","$type":"Orleans.Runtime.StreamId, Orleans.Streaming","fk":{"$type":"System.Byte[], System.Private.CoreLib","$value":"bmFtZXNwYWNla2V5"},"ki":9,"fh":225343565}}

whereas we could write it in a much more friendly and idiomatic format as

  "namespace/key"

The more idiomatic approach is used when serializing the type as the key in a dictionary, and the backward compatible approach used in all other cases.

Replacing Newtonsoft with System.Text.Json as the default grain storage serialization is hidden behind an experimental flag in this PR. To enable it call:

#pragma warning disable ORLEANSEXP006 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
                builder.UseSystemTextJsonGrainStorageSerializer();
#pragma warning restore ORLEANSEXP006 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.

Using System.Text.Json on individual persistence providers by setting their Serializer property if they have one is also possible.

STJ provides support for out-of-order metadata properties such as $id and $ref required for if we wanted toe enable cyclic referencing whilst maintaining backward compatibility with Newtonsoft, but only in .NET 9.0+ packages, which are not referenced by Orleans 9.0 currently.

StreamId.TryFormat method output was changed as it was not readable by the corresponding StreamId.Parse() method - this appeared to be unintentional but may not have been.

IUtf8SpanFormatter was also implemented on StreamId in this PR.

Remaining work in this PR:

Must

• Test wholesale adoption of STJ and run all Orleans tests (might need some help with this)
• Verify the change to StreamId.TryFormat does not cause regressions

Should

• Ensure consistent low-allocation approach in all STJ.JsonConverter, including property name parsing
• Look at providing the capability to move to a more idiomatic JSON representation of Grainid, StreamId etc by sacrificing the ability to revert to Newtonsoft, perhaps as a config option?
• Implement IUtf8SpanFormatter on other persisted types where appropriate.
• Look for opportunities to use JsonConverter attribute directly on the Orleans types rather than registering a converter in configuraiton.
• Ensure converters are public
• Consolidate the two STJ GrainId JsonConverters into one.

Could

• Add benchmarks to quantify potential performance improvement

Won't

• Replace Newtonsoft in membership providers, reminders, and other places. This work can follow in separate PRs.

Microsoft Reviewers: Open in CodeFlow