Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Help with custom record struct serialized in .Net6 doesn't deserializing just after upgrade .Net7 #107

Closed
thiagof0701 opened this issue Dec 29, 2022 · 5 comments

Comments

@thiagof0701
Copy link

thiagof0701 commented Dec 29, 2022

Hi,

First at all, I am your fan! I use your libs about 4 years and you rock!!!

About 2 months ago I moved all our binary database (+200k objects) from MessagePack to MemoryPack+BrotliCompressor. My projects were targeting .Net6.

Everything was working as good as expected. Here is a sample that's working on production: https://nextdial.nyc3.digitaloceanspaces.com/20220701.db

Yesterday I updated the VS Studio, installed the .Net7 and upgraded the projects from .Net6 to .Net7. After that, I run a small local test and it worked as expected. So, I export publish it to a clean test VPS with Ubuntu 22.04.1 + .Net7, run again and got a "deserialization failure" (it deserialize all correct except the custom struct, letting it with a default value).

That is the model I have:

    [MemoryPackable]
    public partial record struct DateTimeParam
    {
        [MemoryPackConstructor]
        public DateTimeParam(DateTimeOffset dateTime, long timestamp, bool isItInSeconds)
        {
            this.DateTime = dateTime;
            this.Timestamp = timestamp;
            this.IsItInSeconds = isItInSeconds;
        }

        public DateTimeParam(long timestamp, bool sec = false)
        {
            this.DateTime = sec ? DateTimeOffset.FromUnixTimeSeconds(timestamp) : DateTimeOffset.FromUnixTimeMilliseconds(timestamp);
            this.Timestamp = timestamp;
            this.IsItInSeconds = sec;
        }

        public DateTimeParam(DateTimeOffset dt, bool sec = false)
        {
            this.IsItInSeconds = sec;
            this.Timestamp = sec ? dt.ToUnixTimeSeconds() : dt.ToUnixTimeMilliseconds();
            this.DateTime = dt;
        }

        public DateTimeOffset DateTime { get; }
        public long Timestamp { get; }
        public bool IsItInSeconds { get; }
    }

    [MemoryPackable]
    public partial record class PlaySession
    {
        public DateTimeParam BeginDt { get; set; }
        public DateTimeParam? EndingDt { get; set; }
        public ushort? Duration { get; set; }
        public double Lat { get; set; }
        public double Lng { get; set; }
        public string Country { get; set; }
        public string City { get; set; }
        public string Region { get; set; }
        public string UserAgent { get; set; }
        public string? Referer { get; set; }
        public string? PlayerName { get; set; }
        public string? DeviceName { get; set; }
        public string ListenerId { get; set; }
        public string Ip { get; set; }
        public string Mount { get; set; }
        public string Server { get; set; }
        public string IcecastId { get; set; }
        public string? TrackId { get; set; }
    }

    [MemoryPackable(GenerateType.Collection)]
    public partial class PlaySessionCollection : PooledList<PlaySession>
    {
        public PlaySessionCollection() { }
        public PlaySessionCollection(int capacity) : base(capacity) { }
        public PlaySessionCollection(ReadOnlySpan<PlaySession> list) : base(list) { }
        public PlaySessionCollection(IEnumerable<PlaySession> list) : base(list) { }
    }

The "PooledList" is this package

What could I try to do to solve this issue?

Best,
Thiago

@neuecc
Copy link
Member

neuecc commented Dec 30, 2022

please show error details(message and stacktrace)

@thiagof0701
Copy link
Author

No exception are throw, just a default value after deserialization only for BeginDt/EndingDt.

Maybe is some type of bug in compilation.

Is there anything should looking for to clean the build process?

@neuecc
Copy link
Member

neuecc commented Dec 31, 2022

thanks, I've understood, DateTimeParam has different memory layout between .NET 6 and .NET 7.
I think it is a compiler or runtime change, but I did not expect it at all.
I will look for the cause, please wait a moment.

@neuecc
Copy link
Member

neuecc commented Dec 31, 2022

Although it is disucussioned in this Issue
dotnet/runtime#44579
.NET 6, it was forced to be promoted to LayoutKind.Auto, but since .NET 7, it seems to be Sequential.

as the workaround, .NET 6 and .NET 7 to the same memory layout by specifying LayoutKind.Auto.

[MemoryPackable]
[StructLayout(LayoutKind.Auto)]
public partial record struct DateTimeParam
{
}

Also, in the case of netstandard 2.1 code-generator, I will add this new analyzing feature,
if the unmanaged struct field contains an auto struct (such as DateTime or DateTimeOffset), code-generator displays the error that LayoutKind.Auto must be specified.

@thiagof0701
Copy link
Author

It worked like a charm!

Tks for the reply and congrats for the best work you've been doing for so many years.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants