System.Text.Json serialization order

[bugproof]

AB#1166328
Serializer puts base properties at the end. In JSON net you could create a contract resolver that let you change serialization order.

class Entity
{
    public int Id { get; set; }
}

class Player : Entity
{
    public string Name { get; set; }
}

class Program
{
    static void Main(string[] args)
    {
        Console.WriteLine(System.Text.Json.JsonSerializer.Serialize(new Player {Id = 1, Name = "noname"}));
    }
}

the output will be:

{"Name":"noname","Id":1}

instead of

{"Id":1,"Name":"noname"} which is more readable

[ahsonkhan]

In JSON net you could create a contract resolver that let you change serialization order.

Do you have a scenario in which you need to control the serialization order where the current behavior/order is blocking you? I am asking to understand the help motivate the feature since JSON is generally unordered (i.e. folks shouldn't be relying on the ordering of the properties within the payload).

{"Id":1,"Name":"noname"} which is more readable

Is that the primary benefit? Having the payload written in a specific order so it is easier to read? Adding support for such capabilities would be a bit of work, and this alone doesn't seem worth the trade off (at least when prioritizing features for 5.0).

The contract resolver feature in Newtonsoft.Json has some other capabilities that are currently not supported. How are you controlling the serialization order today (and is that the only capability you need)? Can you please share a code snippet of your resolver.

If ordering is super critical for you, a workaround would be to create and register a JsonConverter<Player>.

public class PlayerConverter : JsonConverter<Player>
{
    public override Player Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
    {
        throw new NotImplementedException();
    }

    public override void Write(Utf8JsonWriter writer, Player value, JsonSerializerOptions options)
    {
        writer.WriteStartObject();
        writer.WriteNumber(nameof(value.Id), value.Id);
        writer.WriteString(nameof(value.Name), value.Name);
        writer.WriteEndObject();
    }
}

And here's how you can register and use it:

var options = new JsonSerializerOptions();
options.Converters.Add(new PlayerConverter());

Console.WriteLine(JsonSerializer.Serialize(new Player { Name = "noname", Id = 1 }, options));

[bugproof]

order is very helpful when testing the API with tools like insomnia, postman etc. you instantly see what's the most important at the top, and naturally ID comes first and it's later easier to navigate in code because you know that base properties come first... JSON is meant to be human-readable otherwise I wouldn't use it. The ordering would be only helpful for the developer and many existing APIs put Id as the first JSON property. Why is it hard to do?

code for resolver:

public class BaseFirstContractResolver : DefaultContractResolver
{
    public BaseFirstContractResolver()
    {
        NamingStrategy = new CamelCaseNamingStrategy();
    }

    protected override IList<JsonProperty> CreateProperties(Type type, MemberSerialization memberSerialization)
    {
        return base.CreateProperties(type, memberSerialization)
            .OrderBy(p => BaseTypesAndSelf(p.DeclaringType).Count()).ToList();

        IEnumerable<Type> BaseTypesAndSelf(Type t)
        {
            while (t != null)
            {
                yield return t;
                t = t.BaseType;
            }
        }
    }
}

no other benefits than readability, I think it would be more natural for base class properties to be serialized in the first place.

[stevendarby]

Have an unusual use case where we have one set of services in .NET for GETs and another set of services for write operations on another platform. We are doing ETag / If-Match checking by hashing JSON responses, so responses have to match across both platforms. It would be nice to be able to control this from the .NET side.

Also, if order really isn't important, could you consider putting the base properties first by default? This just seems to make more logical sense.

[Drawaes]

Hashing for cache invalidation/etag is also one I have come across. As it was a service to service scenario we moved to protobuf in the end to get the control over the order and bytes but that is not always possible for everyone

[layomia]

From @Jogge in #1085:

I'm missing an attribute for setting the order of the element in the JSON output.

Example:

I want the @SEGMENT to be the first element in the serialized JSON output of MyClass:

public abstract class MyBase
{
    [JsonPropertyName("@SEGMENT")]
    public virtual string Segment { get; set; } = "1";

    public int ID { get; set; }
}

public class MyClass : MyBase
{
    public string Name { get; set; }
}

The result when serializing MyClass is:

{
    "MyClass": {
    	"Name": "Foo",
    	"@SEGMENT": "1",
    	"ID": 42
    }
}

I would like the serialized JSON output to be:

{
    "MyClass": {
    	"@SEGMENT": "1",
    	"Name": "Foo",
    	"ID": 42
    }
}

or

{
    "MyClass": {
    	"@SEGMENT": "1",
    	"ID": 42,
    	"Name": "Foo"
    }
}

Json.NET has an attribute for this: https://www.newtonsoft.com/json/help/html/JsonPropertyOrder.htm

Related question: .NET core 3: Order of serialization for JsonPropertyName (System.Text.Json.Serialization)

[aktxyz]

+1 for this ... very helpful for DX to be able to have key things at the top

[neelabo]

(Reprinted from issue #33854)

Currently, the serialization order is not guaranteed.
I use Json to save application settings, but there is a problem if the order is not guaranteed.

• Diff cannot be obtained, making it difficult to find differences in settings.
• Not suitable for version control.
• The deserialization order is changed. -> The application initialization order is changed.

To fix the order, we propose the following enhancements:

• Extend so that alphabetical sort flag or sort function (IComparer<PropertyInfo>) can be set in JsonSerializerOptions. This option only affects the output.

[benmccallum]

Adding another use case for this feature. The GraphQL spec recommends here that the errors prop is rendered at the top, and indeed it's bitten me when debugging things to have it after data and not notice the errors as graphql is kind of unique in that it'll return data it resolved successfully and add errors for parts of the tree it couldn't resolve.

Open issue to implement this sorting is here. We'll be able to knock of the NewtonsoftJson serializer, but not the S.T.Json one until this is possible. and I've just realised for S.T.Json we are using a custom converter to write the response, so can just shift the property write ordering in there without need for this feature for S.T.Json. Still, there's another example use case I guess.

[joelwreed]

Also Algorand's canonical encoding is a MessagePack with sorted json properties. Seems more straightforward to do this with NewtonsoftJson at the moment.

[bjorhn]

This would be a very useful feature to have. I've been using System.Text.Json for handling "huge" amounts of metadata that get checked into source control for about 8 months now, and until today the order has always been the same. I had assumed it was guaranteed all along.

[gtbuchanan]

To add another use case for controlling serialization order, Safe Exam Browser requires generating a sorted JSON object for their Config Key algorithm since it involves comparing hashes.

[bugproof]

Btw. this is the code responsible for the behaviour:

runtime/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonClassInfo.cs

Lines 111 to 143 in 650f28f

	// We start from the most derived type.
	for (Type? currentType = type; currentType != null; currentType = currentType.BaseType)
	{
	const BindingFlags bindingFlags =
	BindingFlags.Instance |
	BindingFlags.Public |
	BindingFlags.NonPublic |
	BindingFlags.DeclaredOnly;
	foreach (PropertyInfo propertyInfo in currentType.GetProperties(bindingFlags))
	{
	// Ignore indexers and virtual properties that have overrides that were [JsonIgnore]d.
	if (propertyInfo.GetIndexParameters().Length > 0 || PropertyIsOverridenAndIgnored(propertyInfo, ignoredMembers))
	{
	continue;
	}
	// For now we only support public properties (i.e. setter and/or getter is public).
	if (propertyInfo.GetMethod?.IsPublic == true ||
	propertyInfo.SetMethod?.IsPublic == true)
	{
	CacheMember(currentType, propertyInfo.PropertyType, propertyInfo, typeNumberHandling, cache, ref ignoredMembers);
	}
	else
	{
	if (JsonPropertyInfo.GetAttribute<JsonIncludeAttribute>(propertyInfo) != null)
	{
	ThrowHelper.ThrowInvalidOperationException_JsonIncludeOnNonPublicInvalid(propertyInfo, currentType);
	}
	// Non-public properties should not be included for (de)serialization.
	}
	}

Maybe Stack<T> could be used to push base types and then just pop and get properties in the appropriate order.

var typeStack = new Stack<Type>();
for (Type? currentType = type; currentType != null; currentType = currentType.BaseType)
{
    typeStack.Push(currentType);
}

Whatever will result in the best performance... maybe sorting by MetadataToken ? https://stackoverflow.com/questions/8067493/if-getfields-doesnt-guarantee-order-how-does-layoutkind-sequential-work

Should this be a default behaviour? Should this be configurable? I can confirm that sorting by MetadataToken works fine.

[zcsizmadia]

@layomia
Here is a proposed implementation/prototype of JsonProperty ordering

It introduces 2 new attributes: [JsonPropertyOrder] and [JsonPropertyOrderByName]

JsonPropertyOrder
Property or field Attribute to specify the order of the given property (compatible with the Newtonsoft JSON JsonProperty.Order behaviour).

JsonPropertyOrderByName
Class Attribute. If it is specified, the properties will be sorted by name (using an optional StringComparison arg)

Sorting logic
Calling sorting
Sorting function

Examples and expected serialization

        public struct PropertyOrderTestStruct1
        {
            [JsonPropertyOrder(1)]
            public int A { get; set; }
            public int B { get; set; }
            [JsonPropertyOrder(-2)]
            public int C { get; set; }

            public static readonly string e_json = @"{""C"":0,""B"":0,""A"":0}";
        }

        public class PropertyOrderTestClass1
        {
            [JsonPropertyOrder(-1)]
            public int A { get; set; } = 1;
            [JsonPropertyOrder(-1)]
            public int B { get; set; } = 2;
            [JsonPropertyOrder(-1)]
            public int C { get; set; } = 3;

            public static readonly string e_json = @"{""A"":1,""B"":2,""C"":3}";
        }

        public class PropertyOrderTestClass2
        {
            [JsonPropertyOrder(2)]
            public int a { get; set; } = 1;
            [JsonPropertyOrder(1)]
            public int b { get; set; } = 2;
            [JsonPropertyOrder(0)]
            public int c { get; set; } = 3;

            public static readonly string e_json = @"{""c"":3,""b"":2,""a"":1}";
        }

        public class PropertyOrderTestClass3
        {
            public int A { get; set; } = 1;
            public int B { get; set; } = 2;
            [JsonPropertyOrder(-2)]
            public int C { get; set; } = 3;

            public static readonly string e_json = @"{""C"":3,""A"":1,""B"":2}";
        }

        public class PropertyOrderTestClass4
        {
            [JsonPropertyOrder(1)]
            public int A { get; set; } = 1;
            [JsonPropertyOrder(2)]
            public int B { get; set; } = 2;
            [JsonPropertyOrder(-1)]
            public int C { get; set; } = 3;

            public static readonly string e_json = @"{""C"":3,""A"":1,""B"":2}";
        }

        public class PropertyOrderTestClass5
        {
            [JsonPropertyOrder(2)]
            public int A { get; set; } = 1;
            [JsonPropertyOrder(-1)]
            public int B { get; set; } = 2;
            [JsonPropertyOrder(1)]
            public int C { get; set; } = 3;

            public static readonly string e_json = @"{""B"":2,""C"":3,""A"":1}";
        }

        public class PropertyOrderTestClass6
        {
            [JsonPropertyOrder(0)]
            public int A { get; set; } = 1;
            [JsonPropertyOrder(0)]
            public int B { get; set; } = 2;
            [JsonPropertyOrder(0)]
            public int C { get; set; } = 3;

            public static readonly string e_json = @"{""A"":1,""B"":2,""C"":3}";
        }

        public class PropertyOrderTestClass7
        {
            [JsonPropertyOrder(1)]
            public int A { get; set; } = 1;
            public int B { get; set; } = 2;
            [JsonPropertyOrder(-2)]
            public int C { get; set; } = 3;

            public static readonly string e_json = @"{""C"":3,""B"":2,""A"":1}";
        }

        [JsonPropertyOrderByName]
        public class PropertyOrderTestClass8
        {
            public int C { get; set; } = 3;
            public int B { get; set; } = 2;
            public int A { get; set; } = 1;

            public static readonly string e_json = @"{""A"":1,""B"":2,""C"":3}";
        }

        [JsonPropertyOrderByName()]
        public class PropertyOrderTestClass9
        {
            public int cC { get; set; } = 3;
            public int CC { get; set; } = 3;
            public int bB { get; set; } = 2;
            public int BB { get; set; } = 2;
            public int aA { get; set; } = 1;
            public int AA { get; set; } = 1;

            public static readonly string e_json = @"{""AA"":1,""BB"":2,""CC"":3,""aA"":1,""bB"":2,""cC"":3}";
        }

        [JsonPropertyOrderByName(StringComparison.OrdinalIgnoreCase)]
        public class PropertyOrderTestClass10
        {
            public int cC { get; set; } = 3;
            public int CC { get; set; } = 3;
            public int bB { get; set; } = 2;
            public int BB { get; set; } = 2;
            public int aA { get; set; } = 1;
            public int AA { get; set; } = 1;

            public static readonly string e_json = @"{""aA"":1,""AA"":1,""bB"":2,""BB"":2,""cC"":3,""CC"":3}";
        }

        [JsonPropertyOrderByName]
        public class PropertyOrderTestClass11
        {
            [JsonPropertyName("C")]
            public int a { get; set; } = 3;
            [JsonPropertyName("B")]
            public int b { get; set; } = 2;
            [JsonPropertyName("A")]
            public int c { get; set; } = 1;

            public static readonly string e_json = @"{""A"":1,""B"":2,""C"":3}";
        }

        [JsonPropertyOrderByName]
        public class PropertyOrderTestClass12
        {
            [JsonPropertyName("C")]
            [JsonPropertyOrder(-2)]
            public int a { get; set; } = 3;
            [JsonPropertyName("B")]
            public int b { get; set; } = 2;
            [JsonPropertyName("A")]
            public int c { get; set; } = 1;

            public static readonly string e_json = @"{""C"":3,""A"":1,""B"":2}";
        }

[bjorhn]

Looks promising!

Would it make sense to also have a simple JsonSerializerOptions setting that does the same thing as JsonPropertyOrderByName, for the whole structure being serialized, or am I the only one interested in such an option?

[zcsizmadia]

Would it make sense to also have a simple JsonSerializerOptions setting that does the same thing as JsonPropertyOrderByName, for the whole structure being serialized, or am I the only one interested in such an option?

The implementation should be fairly trivial. It is more of a design/vision question.

[silvairsoares]

In JSON net you could create a contract resolver that let you change serialization order.

Do you have a scenario in which you need to control the serialization order where the current behavior/order is blocking you? I am asking to understand the help motivate the feature since JSON is generally unordered (i.e. folks shouldn't be relying on the ordering of the properties within the payload).

{"Id":1,"Name":"noname"} which is more readable

Is that the primary benefit? Having the payload written in a specific order so it is easier to read? Adding support for such capabilities would be a bit of work, and this alone doesn't seem worth the trade off (at least when prioritizing features for 5.0).

The contract resolver feature in Newtonsoft.Json has some other capabilities that are currently not supported. How are you controlling the serialization order today (and is that the only capability you need)? Can you please share a code snippet of your resolver.

If ordering is super critical for you, a workaround would be to create and register a JsonConverter<Player>.

public class PlayerConverter : JsonConverter<Player>
{
    public override Player Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
    {
        throw new NotImplementedException();
    }

    public override void Write(Utf8JsonWriter writer, Player value, JsonSerializerOptions options)
    {
        writer.WriteStartObject();
        writer.WriteNumber(nameof(value.Id), value.Id);
        writer.WriteString(nameof(value.Name), value.Name);
        writer.WriteEndObject();
    }
}

And here's how you can register and use it:

var options = new JsonSerializerOptions();
options.Converters.Add(new PlayerConverter());

Console.WriteLine(JsonSerializer.Serialize(new Player { Name = "noname", Id = 1 }, options));

@ahsonkhan, I even agree that people don't care about the order of JSON properties. But, what about when you use the JSON received in the request, to convert it into XML, which in turn, must have a specific ordering?
Note: This is a real use case, and it kept me using Newtonsoft ([JsonProperty (Order = xx)])

[alastair-todd]

Another use-case: iterate through a response array on the client-side using the object's properties to create an html table (and the headings). Useful when a lot of stuff (including the client component) is generic, to have this as a known convention.

[StingyJack]

I use Json to save application settings, but there is a problem if the order is not guaranteed.

This can be fixed by using XML for config files. Which for most things .net should preserve ordering (preserving formatting may require a little extra work ). Also you get the ability to comment and explain what the settings are for and what eligible values are, which is a win for dev and support teams.

I even agree that people don't care about the order of JSON properties

I care, and we all should care. Not adhering to an order makes supporting the system more difficult and time consuming. There is an order being chosen, and that choice needs to be reliable.

what about when you use the JSON received in the request, to convert it into XML, which in turn, must have a specific ordering?

@silvairsoares - I agree, and this idea extends much farther than just the json->xml. Any machine generated data that may need to be inspected (compared, etc.) or should have a guaranteed and well known ordering. This could be config objects turned into config files, or the result of code generation from an API, .*proj/.sln files, data payloads exchanged between systems, and so on.

[rabidkitten]

To the commenters asking "why" this feature is necessary, beyond perhaps readability, perhaps what you should be asking is why features that used to work, features that people expected to be there - are no longer there? Sure, one can just deal with the now lack of functionality, but why should anyone have to? Why is it that upon every release code that used to work, tests that used to pass, UI's that used to depend on casing or date formats or custom formatters, all stop because a developer didn't think it was important or didn't feel they had time to do it...

[gtbuchanan]

@rabidkitten Are you sure System.Text.Json had this functionality in the past? You may be thinking of Newtonsoft.Json, which is a completely different library, not managed by Microsoft, and has been around for about 15 years. As for unmet expectations, that's why this issue exists. AFAIK, Microsoft didn't port System.Text.Json from any other source so your claims of this feature being intentionally excluded are somewhat unfounded. Nonetheless, you can always still use Newtonsoft.Json until this feature is available (see this and this tutorial).

[StingyJack]

@gtbuchanan - the disconnect most of us have is in how System.Text.Json was and is still being presented as the successor and replacement for the newtonsoft library.

This and related blogs/GH posts was the first place many of us heard about the STJ and how fast it was supposed to be, what its potential future was, and how to try it out now.

There is also an official docs page for migration. Even though that has disclaimers at the beginning it uses some fancy words to obfuscate large and unexpected gaps in functionality.

JSON.net has been replaced in all of the recent .NET platform updates by System.Text.Json.

The author of the Newtonsoft library started working for Microsoft a few months before the announcement of the STJ package. I'm pretty sure I read somewhere that he played a large part in the creation of STJ.

So while technically you may be correct, you are also not "right" (no offense intended, I am in this same situation often and its lack of logical sense makes me cray-cray) because all of the presentation and communications from Microsoft give a clear impression that STJ is a continuation or replacement of JSON.net, leading us to expect it to do things that we could do before, but faster or with more features.

---

EDIT to avoid glomming up the topic (sure would be nice to have threaded conversations here). I'm not frustrated, I'm disappointed that it cant serialize correctly. Sure its fast, but its losing my data so its this makes it unusable.

EDIT2: someone at microsoft should start stressing the importance of reliable ordering of data structures, be it JSON or the elements in a proj file. Everyone who had merge pain with the non-SDK style proj format sticking elements into random but often overlapping positions in the file would have had that pain mostly eliminated had the project fie been ordered in a common and consistent way. AFAICT there is still no ordering for these files, so that problem didnt get fixed, it just got a lowered occurrence rate.

[gtbuchanan]

@StingyJack I understand your frustration. To be fair, successors do not generally have 100% feature parity right off the bat (hence why they're successors and not just a new version) and the official docs page looks like it makes a decent effort to list the differences. The fact of the matter is Newtonsoft.Json is still under active development and it is possible to swap out System.Text.Json when it does not fit your needs.

leading us to expect it to do things that we could do before, but faster or with more features

I read all the same blog posts and, aside from the speed claim, I saw nobody stating feature parity or more features. Any assumptions anyone makes on that matter are their own. To quote Immo Landwerth:

Furthermore, customers who need more performance can also choose to use the new JSON APIs, at the expense of the rich feature set that Json.NET offers.

I commented to provide an alternative to those who seem to think there isn't one. I have no interest in continuing this conversation about Microsoft's decisions as I'm purely interested in this feature and don't want to see it locked.

[PoP2005]

I am looking for this as well. In addition to much easier 'human' readability (which is also why the pretty formatting was created I suppose), it would allow for a very basic and efficient equality comparison between two Json serialized output strings (two identical Json objects would give two identical string outputs). I suspect (correct me if I'm wrong) that a simple string comparison would be faster than having to deserialize two objects and then compare all their properties ?

[MPBinSRV]

One more voice added to the hope of its future inclusion.

[thomas-lm]

Another use case :
I try to render Object respecting OData V4 Standard. The official documentation (see here) say that @odata.context "MUST be the first property of any JSON response"
There is no possibility to get it in first position when use with Inheritance.

[stevetalkscode]

Another use case I have is for controlling the ordering properties in models within a Swagger.json. Since Swashbuckle has moved over to STJ, the properties appear in alphabetical order. I would like to be able to control the order of properties using an attribute on the model

[Turnerj]

We're in the same boat with Schema.NET. It affects our tests currently (though that can be overcome) however any users consuming our library may also be dependent on ordering so we wouldn't want to lose that.

I'm trying to work out the stage this issue is in - are we deciding whether this will be implemented, how this will be implemented, or more just waiting on someone to implement it? Happy to help out with implementing the change though it does look like @zcsizmadia basically has done all the work.

[BreyerW]

Another use case is more efficient patching. With guaranteed order patch size is minimal since it involves finding diff in json and with unordered json patch size is undefined instead of smallest

[steveharter]

Closing; basic property ordering is addressed by a new JsonPropertyOrderAttribute.

[bugproof]

@steveharter Is there any issue to track "allowing write access to the JsonPropertyInfo.Order property" as explained here #54748 ?

[steveharter]

Is there any issue to track "allowing write access to the JsonPropertyInfo.Order property" as explained here #54748 ?

For V6, the metadata infrastructure was not opened up, so modifying the Order at runtime is not possible. The issue to expose metadata is tracked by #36785