Allow for overriding JSON serialization/deserialization related Code Generation

[The-Funk]

Is your feature request related to a problem? Please describe the problem.

The request adapter pattern used by Kiota is pretty slick and it stores request and response parameters as object properties (which I rather like), however some request or response parameter names are valid as JSON or a query param but invalid as an object property name. In particular I am generating Java code and have found that certain OpenAPI specs fail due to odd JSON property names.

For instance, the market v1.2 API here contains a property '@' in one of the JSON responses.

Usually with Jackson or Gson you would use an annotation to handle this type of incompatible mapping. For instance:

   // gson
   @SerializedName("@")
   private String at;

   // jackson
   @JsonProperty("@")
   private String alsoAt;

Is there currently a way to handle property names that don't cleanly map to object properties with Kiota, and if not might I suggest an override registry?

Client library/SDK language

Java

Describe the solution you'd like

Was thinking about this and I don't know if there'd be an easy way to provide meaningful names if you were to automatically replace generated names that don't map to valid object property names appropriately, but it might be easy enough to provide the user a way to supply/override the default behavior with custom mappings as part of a JSON or YML config file, and then just consult this override registry when doing code generation.

That way if a property needs to serialize to a JSON key value that cannot come directly from the object property name due to the name being invalid as an object property, or deserialize from a JSON key value that cannot map to an object property name for the same reason, the user could just supply the replacements.

I think something along the lines of this would work:

 {
    "path": "", // an optional path to allow for explicit serialization/deserialization at the path level, maybe using JSON dot notation?
    "serializeAs": "@",   // JSON key to send over the wire for this object property
    "deserializeFrom": "@",   // JSON key to read from the wire and map to this property
    "propertyName": "atSymbol"   // The valid Java object property name
}

Additional context

No response

[The-Funk]

Forked to look into this. Here are my thoughts so far.

• Kiota (this repo) looks like it's SerDe/JSON library agnostic. Probably best to keep it that way.
• Barring additional augmentation in the individual language libraries (I didn't see any) the override registry would need to exist where code is being generated, ergo in the CLI tool, this repository.
• I have not tried this in other languages to see if this issue is specific to just the Java implementation or if it exists in C# as well. I would assume C# probably suffers from this same issue but would need to look farther into it.
• In the corresponding kiota libraries per language we can add generic annotations/decorators for handling overrides/edge cases.

For instance (using Java because it's what I know best and where I discovered this issue) let's assume in the Java library we add two annotations:

SerializeTo("@")
DeserializeFrom("@")

Then we can have the writer parse the overrides file and add these annotations to any fields that require them.

Alternatively properties could be stored in a map instead of being mapped to object properties, but that kinda makes them harder for the library user to work with (in my opinion).

With the annotations in place, the JSON serialization and deserialization implementations could then integrate the generic annotations into their object mappers.

[The-Funk]

Further thoughts, it may be easier to just validate property names and store invalid names in a map, keeping the existing object mapping functionality but using a data structure that supports weird key names for edge cases. That still requires custom serialization and deserialization logic though, and that logic would be based around either a specific object class, which gets a little hairy, or you'd need extra context when using maps, also hairy.

I'm going to check if this issue exists in the C# generated code as well to see how y'all handle things there. Might be that this is a solved problem in the other languages.

The original suggestion of allowing overrides, I think, provides the most flexibility. Even if the problem is solved or doesn't exist in the other languages, allowing customization of naming during serialization and deserialization is still a nice added feature.

[baywet]

Hi @The-Funk
Thank you for using kiota and for reaching out.
Thank you for your patience.

I'm still trying to understand what the scenario/ask is here? Is it to emit two different properties on the model, one used for serialization, another used for deserialization, but they ultimately map to the same JSON field?

[The-Funk]

Hey @baywet sorta, yes. It might be easier to explain in a quick call if you prefer. Also, I'm willing to take a shot at a PR later if this is something that you think should move forward.

The primary goal is to handle mapping JSON fields with names that aren't valid Java object property names, so that the client generation doesn't fail.

For instance if according to the OpenAPI spec there is a POST endpoint that takes a request JSON object that needs a field called "@", kiota attempts but fails to generate the Java model for the request object because "@" is not a valid Java object property name, but the current code generation doesn't have a way to gracefully handle this scenario.

One way to handle this would be to use maps instead of Java objects for request and response JSONs but I think this makes the generated client code more brittle and annoying to work with. You kind of lose some of the benefit of the code generation if everything is just a Map and you have to do lookups for all your JSON field names. To solve this problem and keep the Java object modeling, we could allow for the user to specify an override for what the generated Java model's property name should be, based on the JSON property name.

For most use cases, when a user indicates an override, we would write the user defined property name to the Java file, then we could just put an annotation on top of the Java property to indicate what name the JSON mapper implementation should use when serializing/deserializing that specific object property. We would want to create our own annotation for this so that we don't lock the user into using any specific object mapper. Then the different mapping implementations that are out there for Java (default Gson implementation or Jackson implementation for Quarkus) could rather simply integrate this annotation into how their object mapper is configured (albeit it would require a little bit of reflection).

If we wanted maximum flexibility we could allow for a generated property to serialize/deserialize with different names. For example we could have the property set to deserialize from an API response JSON's given name (for instance "@") to a property name (let's say atSymbol) but if the user wanted to then serialize that object to store in some logs for instance, and they wanted to store a different name than "@" or "atSymbol" they could do so. This would require 2 annotations, one specifically for the deserialization override and one specifically for the serialization override (see the config in the original post). Again this is for maximum flexibility and most use cases would be just fine with scenario 1 where we have a single annotation for overriding serialization/deserialization together.