Deserialization of Map<Enum, Int> fails with generic implementation, works with concrete? (Kotlin)

[stefanhendriks]

I use:

kotlin: 1.4.10
com.fasterxml.jackson.core:jackson-annotations:2.10.1
com.fasterxml.jackson.core:jackson-databind:2.10.1
com.fasterxml.jackson.datatype:jackson-datatype-jdk8:2.10.1
(spring-boot: 2.4.0)

I try to deserialize a map which has keys which should be an enum type (LuggageType).

When writing a test, it works as expected:

@Test
    fun deserializeAsMapWithEnumKey() {
        val mapper = ObjectMapper()
        mapper.registerModule(KotlinModule())
        
        // create JSON first:
        var theMap: Map<LuggageType, Int> = mapOf(LuggageType.LARGE to 3, LuggageType.SMALL to 2)

        var json = mapper.writeValueAsString(theMap)
        System.out.println(json) // {"LARGE":3,"SMALL":2}
        
        // Now read the JSON:
        val typeRef: TypeReference<HashMap<LuggageType, Int>> = object : TypeReference<HashMap<LuggageType, Int>>() {}
        val mapRead = mapper.readValue(json, typeRef)

        System.out.println(mapRead) // {LARGE=3, SMALL=2}
        
        Assert.assertEquals(2, mapRead.keys.size)
        // Check enums are read
        Assertions.assertThat(mapRead.keys).contains(LuggageType.SMALL, LuggageType.LARGE)
    }

However, we use a util class (called SwissKnife) with some generics stuff in it. Which does:

object SwissKnife {

// .... bunch of more stuff ....

    private val objectMapper: ObjectMapper = ObjectMapper().registerModule(KotlinModule())

// .... bunch of more stuff ....

    @Throws(JsonProcessingException::class)
    fun <T> toJson(obj: T): String {
        return objectMapper.writeValueAsString(obj)
    }

    @Throws(JsonProcessingException::class, JsonMappingException::class)
    fun <K, V> getJsonValuesAsMapFriendly(jsonData: String?): Map<K, V> {
        if (StringUtils.isEmpty(jsonData)) {
            return HashMap<K, V>()
        }
        val typeRef: TypeReference<HashMap<K, V>> = object : TypeReference<HashMap<K, V>>() {}
        return objectMapper.readValue(jsonData, typeRef)
    }

// .... bunch of more stuff ....
}

The only real difference is using a K,V for the TypeReference. Now using that in another test, it seems to fail:

    @Test
    fun getJsonValuesAsMapFriendly_withEnumAsKey_alongWithJackson() {
        val mapper = ObjectMapper()
        mapper.registerModule(KotlinModule())

        // create JSON first:
        var theMap: Map<LuggageType, Int> = mapOf(LuggageType.LARGE to 3, LuggageType.SMALL to 2)

        var json = mapper.writeValueAsString(theMap)
        var swissKnifeJson = SwissKnife.toJson(theMap)
        System.out.println("JSON: " + json) // {"LARGE":3,"SMALL":2}
        System.out.println("SWISSKNIFE: " + swissKnifeJson)
        Assert.assertEquals(json, swissKnifeJson)

        // Now read the JSON:
        val typeRef: TypeReference<HashMap<LuggageType, Int>> = object : TypeReference<HashMap<LuggageType, Int>>() {}
        val mapRead = mapper.readValue(json, typeRef)
        System.out.println("MAPREAD: " + mapRead) // {LARGE=3, SMALL=2}
        val swissKnifeMap = SwissKnife.getJsonValuesAsMapFriendly<LuggageType, Int>(swissKnifeJson)
        System.out.println("SWISSKNIFE: " + swissKnifeMap) // {LARGE=3, SMALL=2}

        Assert.assertEquals(2, swissKnifeMap.keys.size)
        // Check enums are read (FAILS!)
        Assertions.assertThat(swissKnifeMap.keys).contains(LuggageType.SMALL, LuggageType.LARGE)
    }

This test basically is a copy of the first, but also calls the SwissKnife versions and compares them. At the end, it simply checks if the map will have 2 enum keys. However, it doesn't, it fails because of:

Expecting ArrayList:
 <["SMALL", "LARGE"]>
to contain:
 <[SMALL, LARGE]>
but could not find the following element(s):
 <[SMALL, LARGE]>

So it is as if the keys are still String.

For reference, the LuggageType enum is:

package nl.kaizenbv.trip.model

import com.fasterxml.jackson.annotation.JsonCreator
import com.fasterxml.jackson.annotation.JsonValue


enum class LuggageType(@get:JsonValue val code: String) {

    SMALL("SMALL"),

    LARGE("LARGE"),
    
    UNKNOWN("UNKNOWN");
    
//    companion object {
//        @JsonCreator(mode=JsonCreator.Mode.DELEGATING) 
//        @JvmStatic fun fromString(str: String) =
//                values().firstOrNull { it.toString() == str } ?: UNKNOWN
//    }
}

Even uncommenting the @JsonCreator will not make a difference (hence I don't even see it being called!?).

Your question
Can anyone elaborate/explain what is going on? Is this a bug? Or is there something I'm doing wrong?

I get confused because the only real difference is changing explicit types to generics, then again the calls are the same and the resulting objects seem to be the same. Calling toString on the swissKnifeMap gives the impression it works, but since the keys are String you wont be able to tell the difference.

As you can see, the keys are truely strings:

[stefanhendriks]

As a test I created a function out of these 2 lines:

val typeRef: TypeReference<HashMap<LuggageType, Int>> = object : TypeReference<HashMap<LuggageType, Int>>() {}
val mapRead = mapper.readValue(json, typeRef)

into the SwissKnife:

@Throws(JsonProcessingException::class, JsonMappingException::class)
fun <LuggageType, Int> getJsonValuesAsMapFriendly2(jsonData: String?): Map<LuggageType, Int> {
   val typeRef: TypeReference<HashMap<LuggageType, Int>> = object : TypeReference<HashMap<LuggageType, Int>>() {}
   return objectMapper.readValue(jsonData, typeRef)
}

But when using that function in the test which worked fine (ie, replacing the 2 lines above, with the function), it fails ?!

    @Test
    fun deserializeAsMapWithEnumKey_with2LineFunction() {
        val mapper = ObjectMapper()
        mapper.registerModule(KotlinModule())
        
        // create JSON first:
        var theMap: Map<LuggageType, Int> = mapOf(LuggageType.LARGE to 3, LuggageType.SMALL to 2)

        var json = mapper.writeValueAsString(theMap)
        System.out.println(json) // {"LARGE":3,"SMALL":2}
        
        // Now read the JSON: (replace these 2 lines, use the function instead)
//        val typeRef: TypeReference<HashMap<LuggageType, Int>> = object : TypeReference<HashMap<LuggageType, Int>>() {}
//        val mapRead = mapper.readValue(json, typeRef)
        val mapRead = SwissKnife.getJsonValuesAsMapFriendly<LuggageType, Int>(json)

        System.out.println(mapRead) // {LARGE=3, SMALL=2}
        
        Assert.assertEquals(2, mapRead.keys.size)
        // Check enums are read
        Assertions.assertThat(mapRead.keys).contains(LuggageType.SMALL, LuggageType.LARGE)
    }

This fails at the assertion because of the same reason.

Reverting back to the 2 lines in the test , makes the test pass again?

[stefanhendriks]

I narrowed it down to a Kotlin feature, but I don't know why it happens:

The following function fails:

    @Throws(JsonProcessingException::class, JsonMappingException::class)
    fun <LuggageType, Int> getJsonValuesAsMapFriendly2(jsonData: String?): Map<LuggageType, Int> {
        val typeRef: TypeReference<HashMap<LuggageType, Int>> = object : TypeReference<HashMap<LuggageType, Int>>() {}
        return objectMapper.readValue(jsonData, typeRef)
    }

But removing the <LuggageType, Int> before the function name (which was generics in the first place) makes it work.

Ie, the function:

    @Throws(JsonProcessingException::class, JsonMappingException::class)
    fun getJsonValuesAsMapFriendly2(jsonData: String?): Map<LuggageType, Int> {
        val typeRef: TypeReference<HashMap<LuggageType, Int>> = object : TypeReference<HashMap<LuggageType, Int>>() {}
        return objectMapper.readValue(jsonData, typeRef)
    }

works!

This is unfortunate, because my actual goal is to have a generics like function like so:

    @Throws(JsonProcessingException::class, JsonMappingException::class)
    fun <K, V> getJsonValuesAsMapFriendly(jsonData: String?): Map<K, V> {
        if (StringUtils.isEmpty(jsonData)) {
            return HashMap<K, V>()
        }
        return objectMapper.readValue(jsonData, HashMap::class.java) as Map<K, V>
    }

But - as I would expect now - still fails due the same reason.

I am not sufficient enough in Kotlin to explain this behavior. Or perhaps it is a Kotlin/Jackson mixture that plays?

[stefanhendriks]

I found it.

I should have known, but somehow I overlooked.

The whole reason this won't work how it is written now is because of Type Erasure. This answer at stackoverflow explains it. So the whole TypeReference cannot be used like this.

The reason the code worked without <LuggageType, Int> before the method name was because then there would be no type erasure. If you do specify <LuggageType, Int> before method name then the words LuggageType and Int become generic types (much like simply naming them K,V). In fact, my IDE was so smart to color them differently, but I didn't notice.

However Kotlin to the rescue!

It is possible to make a generic function to read a Map with generic types. Using inline and reified from Kotlin.

With that, I made the function as following (which works):

    @Throws(JsonProcessingException::class, JsonMappingException::class)
    inline fun <reified K, V> getJsonValuesAsMapFriendly(jsonData: String?): Map<K, V> {
        if (StringUtils.isEmpty(jsonData)) {
            return HashMap<K, V>()
        }
        val typeRef: TypeReference<Map<K, V>> = object : TypeReference<Map<K, V>>() {}
        return objectMapper.readValue(jsonData, typeRef)
    }

(objectMapper must be public though)

This simply will inline the function everywhere where it is used, which can be a downside (makes the code a bit bigger). But now it will work with any kind of map you pass in it and it is statically typed 🥳

I guess I answered my own question here.

Final test now looks like:

    @Test
    fun deserializeAsMapWithEnumKey() {
        var json = "{\"LARGE\":3,\"SMALL\":2}"
        
        // Act
        val map = SwissKnife.getJsonValuesAsMapFriendly<LuggageType, Int>(json)
        
        // Assert
        Assert.assertEquals(2, map.keys.size)
        Assertions.assertThat(map.keys).contains(LuggageType.SMALL, LuggageType.LARGE)
    }

[stefanhendriks]

Although the unit test works, when further working with the data, directly in to a Json object, I get:

Could not write JSON: class java.lang.String cannot be cast to class java.lang.Enum
		(java.lang.String and java.lang.Enum are in module java.base of loader 'bootstrap'); nested exception is
		com.fasterxml.jackson.databind.JsonMappingException: class java.lang.String cannot be cast to class
		java.lang.Enum (java.lang.String and java.lang.Enum are in module java.base of loader 'bootstrap')
		(through reference chain:
		java.util.ArrayList[0]->nl.kaizenbv.trip.dto.trip.TripDto["airportData"]->nl.kaizenbv.trip.dto.reservation.AirportMetadataJson["luggageType"])

Sigh.

[dinomite]

Are you able to recreate that failure in a test? If so, would you share a branch of j-m-k (branch off of 2.13) with your test case?