Provide a way to order serialization

[pdvrieze]

I'm busy writing a serialization provider that serializes to XML. There are some complexities with manually passing information to the right place (TaggedOutput helps a lot, but is far from perfect). I have one problem that will create inefficiency. Attributes should be written before child elements (but that depends on the order of serialization (and declaration)). While I can use a buffer to work around this problem (and specifying properties in the right order in the class can help as well), this is not efficient - it creates overheads in both memory and cpu). Instead it would be very useful if there were a way to somehow manipulate the order in which properties are written by the system.

Alternatively if I can determine that the type has a default generated serializer I could do the serialization programmatically.

[sandwwraith]

If I understand you right, you want to manipulate order in which generated serializer's save calls to writeElementXXX ? And usecase is following:

data class MyData(
    val a: Int, // attribute
    val b: OtherData, //child
    val c: Int, // attribute, better to write before b
)

if I can determine that the type has a default generated serializer

Is checking that class have @Serializable annotation or nested class $$serializer doesn't work for you? Or you want solution without reflection?

[pdvrieze]

@sandwwraith That is correct. I want to he able to specify the order. It would also really be great if serialDesc would contain the kind of the child (so I don't need to guess). Currently the writing code will do a bit of a guess and when it thinks it has an inverted order it will use a buffer to delay writing all non-attribute values.

I'll probably update it to buffer save commands rather than serialized xml, but the need to do buffering is still annoying. It needs to be multiplatform safe so reflection at runtime is not an option.

[MariusVolkhart]

I am a library consumer, but for me it is also necessary to be able to specify the order in which properties are serialized to JSON. In my case I need them sorted alphabetically.

For example,

data class Shape(val sides: Int, val name: String)

should always serialize to {"name":"value","sides":4}

Just to be clear, I am not saying this behavior should be the default, but rather that I need some way to hook into the ordering.

[sandwwraith]

@MariusVolkhart You can try to serialize your class to JSONObject first, then sort it, then convert to string.

[MariusVolkhart]

@sandwwraith 👍 that works for now. Thanks!

[nanodeath]

For others who stumble upon this, here's a scrappy example based on what @sandwwraith suggested:

val jsonObject = json.toJson(MyDataClass.serializer(), MyDataClass(foo)).jsonObject
val ordering = listOf("field1", "field2", "field3")
val map = jsonObject.content.entries
    .sortedBy { (key, _) -> ordering.indexOf(key).takeIf { it >= 0 } ?: throw IllegalArgumentException("Unexpected key $key") }
    .map { it.key to it.value }
    .toMap()
val jsonString = json.stringify(serializer(), map)

It's inefficient but it works.

[nanodeath]

As for why I'd like this feature too, I'm writing a JSON-RPC serializer. While the spec doesn't specify it, the ideal order for keys in the request is jsonrpc, then id, then the rest. This is because the response needs to include the id key even if there's a parse error. If the request is horribly malformed and looks like {"jsonrpc":"2.0","id":1,, then the response can say "hey request 1 was malformed!". But if the request looks like {"jsonrpc":"2.0","method":"sum",, then the response can only say "hey I got a malformed request" with no further information.

[xavi-]

I have a similar requirement, though for me the specific order doesn't matter. It just needs to be consistent across executions and environments.

[sandwwraith]

@xavi- In general, serialization order is the same as program one (i.e. properties in JSON should have the same ordering as in your class declaration in source code)

[xavi-]

That's great news. I didn't realize property order for classes is guaranteed to be stable. Unfortunately, I also need a stable order from Map's as well. For what it's worth, here's a bit code that recursively orders the properties of a JsonObject:

private fun normalize(elem: JsonElement): JsonElement {
    return when (elem) {
        is JsonObject -> JsonObject(
                elem.content.map { it.key to normalize(it.value) }.sortedBy { it.first }.toMap())
        is JsonArray -> JsonArray(elem.content.map { normalize(it) })
        else -> elem
    }
}

[ScottPierce]

@sandwwraith Can you confirm that serialization order for json is guaranteed to be the order of property definition in the class?

[sandwwraith]

@ScottPierce It's not specified anywhere, but we try to maintain this order when possible. So you can think of this as 'implementation-defined'

[sandwwraith]

No particular plans for the issue now. For JSON ordering, better use JsonTransofrmingSerializer

[ctadlock]

Its disapointing that this feature would not be considered. How is it any different than pretty printing? Neither have any functional difference, both are for UX. Making every developer implement a hackish workaround is a sign that the development team is not listening to real user needs. This should just be a property on JsonConfiguration, at least for alpha sorting.

For a comparison; .NET has had this for years:
https://learn.microsoft.com/en-us/dotnet/api/system.text.json.serialization.jsonpropertyorderattribute.order?view=net-7.0

[Ayfri]

Is there any update for this issue?

[sandwwraith]

@Ayfri Not for now.

[pdvrieze]

@Ayfri The original request was to allow the format to somehow influence the order in which a serializer would serialize its children (the order in which serializeElement(...) etc are called), it is mainly relevant for implementers of formats (in XML attributes must be written before children) - it avoids having to buffer writes - it is format specific thus has to be "dynamic".

This is separate from a (format independent) way to use annotations to specify an order that is different from the order in which they are written in the code (which is the normal way) - that approach can be static but could be challenging (how do you deal with inheritance?). It could however still be overridden by the format (again attributes in XML must be before children).

[Ayfri]

Is there any issue for defining the order of the serialized properties then ?

[pdvrieze]

@Ayfri It is defined by the (generated) serializer and should be the order of declaration of the type (I'm not sure how it deals with inhertance though).
Another "hack" would be to have a wrapper serializer that reorders its children (and potentially auto-wraps the child serializers as well).