-
Notifications
You must be signed in to change notification settings - Fork 620
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Is it possible to serialize a generic Map<String, Any> in JVM/JS as a JSON object? #296
Comments
@sandwwraith Can you possibly share an example code to deal with |
@hotchemi You can replace |
@sandwwraith this isnt really a solution for orgs that already have maps heavily ingrained, which is more likely than having JsonObject everywhere. There should be some support for Map<String, Any> without converting it to a JsonObject or converting every map to <String, JsonElement>. |
|
Its about converting Any to JsonElement, not the other way around. Any isnt very type-safe but provides a big use case with Maps. As with other serializers it would just thrown an JsonException on unsupported types. |
@dri94 did you find a way to convert |
It appears there is no way to serialize a dashed variable in Kotlin JS, JSONObject only works on JVM: JetBrains/kotlin-wrappers#339 ERROR: "name contains illegal identifiers that can't appear in javascript identifiers"
EDIT/SOLVED: this[ "grapesjs-lory-slider"] |
This is how I am doing Map to JsonElement for now. I take all primitive to be string (In my case it is fine).
|
I was expecting to be able to use No way to get a generic interface for all the possible serializable types? |
Well to keep this more generic, I'm using a Map where all the elements have a serializer and we can use the kotlin reflection to get the right one for the handled type, so we can get an object out of the map with just: fun buildJsonObject(other: Map<String, Any?>) : JsonElement {
val jsonEncoder = Json{ encodeDefaults = true } // Set this accordingly to your needs
val map = emptyMap<String, JsonElement>().toMutableMap()
other.forEach {
map[it.key] = if (it.value != null)
jsonEncoder.encodeToJsonElement(serializer(it.value!!::class.starProjectedType), it.value)
else JsonNull
}
return JsonObject(map)
} And this will still throw a However, I'm not still fully happy as I'd prefer some more generic serializable type so that can be used with binary when using CBOR serialization, and so where the encoding happens only at the moment we call the Not to mention that a such built object would just fail with Cbor ( |
So, to handle the generic serializer case (such as binary ones), I've crafted some raw So basically mimicking what Polymorphic does, I'm not using any experimental or internal APIs but some of them could improve the result, like reusing the type serialName if any (even though, I'm not sure how i can deserialize that).
Here's a gist, but suggestions are welcome: https://gist.github.com/3v1n0/ecbc5e825e2921bd0022611d7046690b |
See https://youtrack.jetbrains.com/issue/KTOR-3063, also thanks to @kabirsaheb for the first draft of toJsonElement(). The linked youtrack issue has a more advanced mitigation strategy for this issue, where serializing Map, List, String, Number, Boolean, Enum and null is supported. |
Based on @kabirsaheb, I use the next:
|
like is code
will be throw IllegalStateException |
I'm surprised that such conversion is not supported by the library, it seems to be quite common when working with Java code. Thanks for the workarounds though! |
|
Thanks, @kabirsaheb for your solution. fun Collection<*>.toJsonElement(): JsonElement = JsonArray(mapNotNull { it.toJsonElement() })
fun Map<*, *>.toJsonElement(): JsonElement = JsonObject(
mapNotNull {
(it.key as? String ?: return@mapNotNull null) to it.value.toJsonElement()
}.toMap(),
)
fun Any?.toJsonElement(): JsonElement = when (this) {
null -> JsonNull
is Map<*, *> -> toJsonElement()
is Collection<*> -> toJsonElement()
else -> JsonPrimitive(toString())
} |
Hi, do I need to configure anything else to make this work in ktor js client? I have this error:
Here is my configuration:
|
Ignore my comment, I need to set manually |
try this: @OptIn(InternalSerializationApi::class)
fun Any?.toJsonElement(): JsonElement =
when (this) {
null -> JsonNull
is Map<*, *> -> toJsonElement()
is Collection<*> -> toJsonElement()
is Boolean -> JsonPrimitive(this)
is Number -> JsonPrimitive(this)
is String -> JsonPrimitive(this)
is Enum<*> -> JsonPrimitive(this.toString())
else -> this.javaClass.kotlin.serializer().let { json.encodeToJsonElement(it, this) }
}
private fun Collection<*>.toJsonElement(): JsonElement =
JsonArray(this.map { it.toJsonElement() })
private fun Map<String, Any?>.toJsonElement(): JsonElement {
return JsonObject(this.mapValues { it.value.toJsonElement() })
} |
Base on @migueltorcha considering fun Any?.toJsonElement(): JsonElement = when(this) {
null -> JsonNull
is Map<*, *> -> toJsonElement()
is Collection<*> -> toJsonElement()
is ByteArray -> this.toList().toJsonElement()
is CharArray -> this.toList().toJsonElement()
is ShortArray -> this.toList().toJsonElement()
is IntArray -> this.toList().toJsonElement()
is LongArray -> this.toList().toJsonElement()
is FloatArray -> this.toList().toJsonElement()
is DoubleArray -> this.toList().toJsonElement()
is BooleanArray -> this.toList().toJsonElement()
is Array<*> -> toJsonElement()
is Boolean -> JsonPrimitive(this)
is Number -> JsonPrimitive(this)
is String -> JsonPrimitive(this)
is Enum<*> -> JsonPrimitive(this.toString())
else -> {
throw IllegalStateException("Can't serialize unknown type: $this")
}
}
fun Map<*, *>.toJsonElement(): JsonElement {
val map = mutableMapOf<String, JsonElement>()
this.forEach {key, value ->
key as String
map[key] = value.toJsonElement()
}
return JsonObject(map)
}
fun Collection<*>.toJsonElement(): JsonElement {
return JsonArray(this.map { it.toJsonElement() })
}
fun Array<*>.toJsonElement(): JsonElement {
return JsonArray(this.map { it.toJsonElement() })
}
fun main(args: Array<String>) {
val obj = mapOf<String, Any>(
"int" to 1,
"bool" to true,
"float" to 1.2f,
"double" to 1.2,
"arrayInt" to intArrayOf(1, 2,3),
"arrayInt2" to arrayOf(1, 2,3),
"arrayString" to arrayOf("foo", "bar"),
"listDouble" to listOf(1.1, 2.2, 3.3),
"listString" to listOf("goo", "baz"),
"mapInt" to mapOf("1" to 1, "2" to 2),
).toJsonElement()
println(Json.encodeToString(obj))
}
output{
"int":1,
"bool":true,
"float":1.2,
"double":1.2,
"arrayInt":[
1,
2,
3
],
"arrayInt2":[
1,
2,
3
],
"arrayString":[
"foo",
"bar"
],
"listDouble":[
1.1,
2.2,
3.3
],
"listString":[
"goo",
"baz"
],
"mapInt":{
"1":1,
"2":2
}
} |
### 📝 Description Switch from `jackson` to `kotlinx.serialization` for serialization/deserialization of `GraphQLServerRequest` types. After running benchmarks we where able to identify that deserializing `GraphQLServerRequest` with `kotlinx.serialization` is quite faster than doing it with `jackson`, the reason ? possibly because jackson relies on reflections to identify deserialization process. On the other hand, serialization/deserialization of `GraphQLServerReponse` type is still faster if done with `jackson`, possibly because of how `kotlinx.serialization` library was designed and the poor support for serializing `Any` type: Kotlin/kotlinx.serialization#296, which causes a lot of memory comsumption. As part of this PR also including the benchmarks. For that, i created a separate set of types that are marked with both `jackson` and `kotlinx.serialization` annotations. Benchmarks results: Executed on a MacBookPro 2.6 GHz 6-Core Intel Core i7. #### GraphQLServerRequest Deserialization `GraphQLBatchRequest` 4 batched operations, each operation is aprox: 30kb <img width="1260" alt="image" src="https://github.com/ExpediaGroup/graphql-kotlin/assets/6611331/06e5b218-a35e-4baa-a25e-2be1b3c27a95"> `GraphQLRequest` <img width="1231" alt="image" src="https://github.com/ExpediaGroup/graphql-kotlin/assets/6611331/e5ecba01-fd41-4872-b3e8-5519414cc918"> #### GraphQLServerResponse Serialization `GraphQLBatchResponse` <img width="1240" alt="image" src="https://github.com/ExpediaGroup/graphql-kotlin/assets/6611331/ee84bfa4-d7d1-46b4-b4a8-b3c220998a03"> `GraphQLResponse` <img width="1197" alt="image" src="https://github.com/ExpediaGroup/graphql-kotlin/assets/6611331/c217e05f-45fc-460e-a059-7667975ee49f">
Switch from `jackson` to `kotlinx.serialization` for serialization/deserialization of `GraphQLServerRequest` types. After running benchmarks we where able to identify that deserializing `GraphQLServerRequest` with `kotlinx.serialization` is quite faster than doing it with `jackson`, the reason ? possibly because jackson relies on reflections to identify deserialization process. On the other hand, serialization/deserialization of `GraphQLServerReponse` type is still faster if done with `jackson`, possibly because of how `kotlinx.serialization` library was designed and the poor support for serializing `Any` type: Kotlin/kotlinx.serialization#296, which causes a lot of memory comsumption. As part of this PR also including the benchmarks. For that, i created a separate set of types that are marked with both `jackson` and `kotlinx.serialization` annotations. Benchmarks results: Executed on a MacBookPro 2.6 GHz 6-Core Intel Core i7. `GraphQLBatchRequest` 4 batched operations, each operation is aprox: 30kb <img width="1260" alt="image" src="https://github.com/ExpediaGroup/graphql-kotlin/assets/6611331/06e5b218-a35e-4baa-a25e-2be1b3c27a95"> `GraphQLRequest` <img width="1231" alt="image" src="https://github.com/ExpediaGroup/graphql-kotlin/assets/6611331/e5ecba01-fd41-4872-b3e8-5519414cc918"> `GraphQLBatchResponse` <img width="1240" alt="image" src="https://github.com/ExpediaGroup/graphql-kotlin/assets/6611331/ee84bfa4-d7d1-46b4-b4a8-b3c220998a03"> `GraphQLResponse` <img width="1197" alt="image" src="https://github.com/ExpediaGroup/graphql-kotlin/assets/6611331/c217e05f-45fc-460e-a059-7667975ee49f">
…1937) (#1944) Switch from `jackson` to `kotlinx.serialization` for serialization/deserialization of `GraphQLServerRequest` types. After running benchmarks we where able to identify that deserializing `GraphQLServerRequest` with `kotlinx.serialization` is quite faster than doing it with `jackson`, the reason ? possibly because jackson relies on reflections to identify deserialization process. On the other hand, serialization/deserialization of `GraphQLServerReponse` type is still faster if done with `jackson`, possibly because of how `kotlinx.serialization` library was designed and the poor support for serializing `Any` type: Kotlin/kotlinx.serialization#296, which causes a lot of memory comsumption. As part of this PR also including the benchmarks. For that, i created a separate set of types that are marked with both `jackson` and `kotlinx.serialization` annotations. Benchmarks results: Executed on a MacBookPro 2.6 GHz 6-Core Intel Core i7. `GraphQLBatchRequest` 4 batched operations, each operation is aprox: 30kb <img width="1260" alt="image" src="https://github.com/ExpediaGroup/graphql-kotlin/assets/6611331/06e5b218-a35e-4baa-a25e-2be1b3c27a95"> `GraphQLRequest` <img width="1231" alt="image" src="https://github.com/ExpediaGroup/graphql-kotlin/assets/6611331/e5ecba01-fd41-4872-b3e8-5519414cc918"> `GraphQLBatchResponse` <img width="1240" alt="image" src="https://github.com/ExpediaGroup/graphql-kotlin/assets/6611331/ee84bfa4-d7d1-46b4-b4a8-b3c220998a03"> `GraphQLResponse` <img width="1197" alt="image" src="https://github.com/ExpediaGroup/graphql-kotlin/assets/6611331/c217e05f-45fc-460e-a059-7667975ee49f"> ### 📝 Description ### 🔗 Related Issues
Using some of what has been given in this issue as well as messing around with it myself, here is some code that (should) be able to serialize anything that kotlinx.serialization can serialize using Do note: internal fun Any?.toJsonElement(): JsonElement {
val serializer = this?.let { Json.serializersModule.serializerOrNull(this::class.java) }
return when {
this == null -> JsonNull
serializer != null -> Json.encodeToJsonElement(serializer, this)
this is Map<*, *> -> toJsonElement()
this is Array<*> -> toJsonElement()
this is BooleanArray -> toJsonElement()
this is ByteArray -> toJsonElement()
this is CharArray -> toJsonElement()
this is ShortArray -> toJsonElement()
this is IntArray -> toJsonElement()
this is LongArray -> toJsonElement()
this is FloatArray -> toJsonElement()
this is DoubleArray -> toJsonElement()
this is UByteArray -> toJsonElement()
this is UShortArray -> toJsonElement()
this is UIntArray -> toJsonElement()
this is ULongArray -> toJsonElement()
this is Collection<*> -> toJsonElement()
this is Boolean -> JsonPrimitive(this)
this is Number -> JsonPrimitive(this)
this is String -> JsonPrimitive(this)
this is Enum<*> -> JsonPrimitive(this.name)
this is Pair<*, *> -> JsonObject(
mapOf(
"first" to first.toJsonElement(),
"second" to second.toJsonElement(),
)
)
this is Triple<*, *, *> -> JsonObject(
mapOf(
"first" to first.toJsonElement(),
"second" to second.toJsonElement(),
"third" to third.toJsonElement(),
)
)
else -> error("Can't serialize '$this' as it is of an unknown type")
}
}
internal fun Map<*, *>.toJsonElement(): JsonElement {
return buildJsonObject {
forEach { (key, value) ->
if (key !is String)
error("Only string keys are supported for maps")
put(key, value.toJsonElement())
}
}
}
internal fun Collection<*>.toJsonElement(): JsonElement = buildJsonArray {
forEach { element ->
add(element.toJsonElement())
}
}
internal fun Array<*>.toJsonElement(): JsonElement = buildJsonArray {
forEach { element ->
add(element.toJsonElement())
}
}
internal fun BooleanArray.toJsonElement(): JsonElement = buildJsonArray { forEach { add(JsonPrimitive(it)) } }
internal fun ByteArray.toJsonElement(): JsonElement = buildJsonArray { forEach { add(JsonPrimitive(it)) } }
internal fun CharArray.toJsonElement(): JsonElement = buildJsonArray { forEach { add(JsonPrimitive(it.toString())) } }
internal fun ShortArray.toJsonElement(): JsonElement = buildJsonArray { forEach { add(JsonPrimitive(it)) } }
internal fun IntArray.toJsonElement(): JsonElement = buildJsonArray { forEach { add(JsonPrimitive(it)) } }
internal fun LongArray.toJsonElement(): JsonElement = buildJsonArray { forEach { add(JsonPrimitive(it)) } }
internal fun FloatArray.toJsonElement(): JsonElement = buildJsonArray { forEach { add(JsonPrimitive(it)) } }
internal fun DoubleArray.toJsonElement(): JsonElement = buildJsonArray { forEach { add(JsonPrimitive(it)) } }
internal fun UByteArray.toJsonElement(): JsonElement = buildJsonArray { forEach { add(JsonPrimitive(it)) } }
internal fun UShortArray.toJsonElement(): JsonElement = buildJsonArray { forEach { add(JsonPrimitive(it)) } }
internal fun UIntArray.toJsonElement(): JsonElement = buildJsonArray { forEach { add(JsonPrimitive(it)) } }
internal fun ULongArray.toJsonElement(): JsonElement = buildJsonArray { forEach { add(JsonPrimitive(it)) } } Outputpublic fun main() {
val obj = mapOf(
"bool" to true,
"byte" to 1.toByte(),
"char" to '2',
"short" to 3.toShort(),
"int" to 4,
"long" to 5.toLong(),
"float" to 1.2f,
"double" to 1.2,
"ubyte" to 0.toUByte(),
"ushort" to 1.toUShort(),
"uint" to 2u,
"ulong" to 3uL,
"enum" to YesNo.YES,
"pair" to ("foo" to "bar"),
"triple" to Triple("foo", "bar", "baz"),
"unit" to Unit,
"duration" to 1.seconds,
"boolArray" to booleanArrayOf(true, false, true),
"byteArray" to byteArrayOf(1, 2, 3),
"charArray" to charArrayOf('1', '2', '3'),
"shortArray" to shortArrayOf(1, 2, 3),
"intArray" to intArrayOf(1, 2, 3),
"longArray" to longArrayOf(1, 2, 3),
"floatArray" to floatArrayOf(1.0f, 1.1f, 1.2f, 1.3f),
"doubleArray" to doubleArrayOf(1.0, 1.1, 1.2, 1.3),
"ubyteArray" to ubyteArrayOf(1u, 2u, 3u),
"ushortArray" to ushortArrayOf(1u, 2u, 3u),
"uintArray" to uintArrayOf(1u, 2u, 3u),
"ulongArray" to ulongArrayOf(1u, 2u, 3u),
"arrayOfInt" to arrayOf(1, 2, 3),
"arrayOfString" to arrayOf("foo", "bar"),
"listOfDouble" to listOf(1.1, 2.2, 3.3),
"listOfString" to listOf("foo", "bar"),
"setOfString" to setOf("foo", "bar", "baz"),
"mapOfStringInt" to mapOf("1" to 1, "2" to 2),
).toJsonElement()
println(Json.encodeToString(obj))
}
public enum class YesNo {
YES, NO
} {
"bool": true,
"byte": 1,
"char": "2",
"short": 3,
"int": 4,
"long": 5,
"float": 1.2,
"double": 1.2,
"ubyte": 0,
"ushort": 1,
"uint": 2,
"ulong": 3,
"enum": "YES",
"pair": [ "foo", "bar" ],
"triple": [ "foo", "bar", "baz" ],
"unit": {},
"duration": "PT1S",
"boolArray": [ true, false, true ],
"byteArray": [ 1, 2, 3 ],
"charArray": [ "1", "2", "3" ],
"shortArray": [ 1, 2, 3 ],
"intArray": [ 1, 2, 3 ],
"longArray": [ 1, 2, 3 ],
"floatArray": [ 1.0, 1.1, 1.2, 1.3 ],
"doubleArray": [ 1.0, 1.1, 1.2, 1.3 ],
"ubyteArray": [ 1, 2, 3 ],
"ushortArray": [ 1, 2, 3 ],
"uintArray": [ 1, 2, 3 ],
"ulongArray": [ 1, 2, 3 ],
"arrayOfInt": [ 1, 2, 3 ],
"arrayOfString": [ "foo", "bar" ],
"listOfDouble": [ 1.1, 2.2, 3.3 ],
"listOfString": [ "foo", "bar" ],
"setOfString": [ "foo", "bar", "baz" ],
"mapOfStringInt": {
"1": 1,
"2": 2
}
} this should also be able to serialize/deserialize Edit 1: Fixed a bug with CharArray.toJsonElement() serializing to integers instead of strings |
I'm trying to figure out a way to serialize an arbitrary
Map<String, Any>
into JSON. In my case, I can guarantee that in runtime any value of the map is either a primitive, a list or a map. In case of lists and maps, the values of them are either primitives or lists or maps of the same pattern. Seems, it lays nicely on a json object.I do not need to deserialize such an object, serialization only.
For now, I'm trying to write a custom serializer for such case but no success yet. How can it be done for both JVM and JS?
The text was updated successfully, but these errors were encountered: