Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for MemoryPack #197

Open
De-Crypted opened this issue Feb 21, 2024 · 5 comments
Open

Support for MemoryPack #197

De-Crypted opened this issue Feb 21, 2024 · 5 comments
Assignees
Labels
enhancement New feature or request question Further information is requested

Comments

@De-Crypted
Copy link

Is your feature request related to a problem? Please describe.
I was trying to implement custom marshaller for MemoryPack (https://github.com/Cysharp/MemoryPack) but I quickly learned it wouldn't work. The internally used Message class could not be serialized and the reason for this is that MemoryPack does not support [DataContract] attribute nor do they plan on supporting it (Cysharp/MemoryPack#188).

What MemoryPack requires is to make the class partial and annotate it with [MemoryPackable] attribute.

Describe the solution you'd like
Could the Message class be exposed as partial class so that it could be annotated with [MemoryPackable], I think this should allow Source Generator to do it's things (not 100% sure).

Or maybe there is some other way to solve this?

Anyways, thank you for this library. I'm enjoying it very much.

@max-ieremenko
Copy link
Owner

Cysharp/MemoryPack#188
... Unlike protobuf-net and MessagePack for C#, you first need to make it partial ...

Making Messages classes partial won't help: for your project Message(<,>) is external-type. see Serialize external types.

Or maybe there is some other way to solve this?

An example of the MemoryPackMarshaller draft:

public sealed class MemoryPackJsonMarshallerFactory : IMarshallerFactory
{
    public static readonly IMarshallerFactory Default = new MemoryPackJsonMarshallerFactory();

    static MemoryPackJsonMarshallerFactory()
    {
        MemoryPackFormatterProvider.RegisterGenericType(typeof(Message<>), typeof(MessageMemoryPackFormatter<>));
        // TODO: implement MemoryPackFormatterProvider.RegisterGenericType(typeof(Message<,>), typeof(MessageMemoryPackFormatter<,>));
    }

    public Marshaller<T> CreateMarshaller<T>() => new(Serialize, Deserialize<T>);

    private static void Serialize<T>(T value, SerializationContext context)
    {
        var bufferWriter = context.GetBufferWriter();
        MemoryPackSerializer.Serialize(bufferWriter, value);
        context.Complete();
    }

    private static T Deserialize<T>(DeserializationContext context)
    {
        return MemoryPackSerializer.Deserialize<T>(context.PayloadAsReadOnlySequence())!;
    }
}

// TODO implement internal readonly partial struct SerializableMessage<T1, T2>
[MemoryPackable]
internal readonly partial struct SerializableMessage<T>
{
    internal readonly Message<T> Message;

    [MemoryPackInclude]
    public T Value => Message.Value;

    [MemoryPackConstructor]
    public SerializableMessage(T value)
    {
        Message = new Message<T>(value);
    }

    public SerializableMessage(Message<T> message)
    {
        Message = message;
    }
}

// TODO implement internal sealed class MessageMemoryPackFormatter<T1, T2>
internal sealed class MessageMemoryPackFormatter<T> : MemoryPackFormatter<Message<T>>
{
    public override void Serialize<TBufferWriter>(ref MemoryPackWriter<TBufferWriter> writer, scoped ref Message<T>? value)
    {
        ArgumentNullException.ThrowIfNull(value);

        writer.WritePackable(new SerializableMessage<T>(value));
    }

    public override void Deserialize(ref MemoryPackReader reader, scoped ref Message<T>? value)
    {
        if (reader.PeekIsNull())
        {
            throw new NotSupportedException();
        }

        var wrapped = reader.ReadPackable<SerializableMessage<T>>();
        value = wrapped.Message;
    }
}

Test

MemoryPackFormatterProvider.RegisterGenericType(typeof(Message<>), typeof(MessageMemoryPackFormatter<>));

var input = new Message<string>("foo");

var payload = MemoryPackSerializer.Serialize(input);
var actual = MemoryPackSerializer.Deserialize<Message<string>>(payload);

actual.ShouldNotBeNull();
actual.Value.ShouldBe("foo");

@De-Crypted
Copy link
Author

This does look like good solution. If you manage to craft it I can give it a good test run.

@max-ieremenko
Copy link
Owner

This does look like good solution. If you manage to craft it I can give it a good test run.

I invested my time in helping you, have you tried my code example to solve your issue?

@De-Crypted
Copy link
Author

Yes I got to try it today and it works great. I also tested making wrappers for Message (empty message?) and Message<T1, T2> which worked as expected.

Do you want PR for this?

@max-ieremenko
Copy link
Owner

The approach will work for limited functionality: operation with a maximum of 3 input parameters.

Task DoSomething(p1, p2, p3) // will
Task DoSomething(p1, p2, p3, p4) // will not

You are welcome to create PR Examples/MemoryPackMarshaller, it would be helpful for others, something like Examples/CustomMarshaller.

@max-ieremenko max-ieremenko added question Further information is requested enhancement New feature or request labels Feb 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants