swagger-core serializes enum differently when using custom enum serializer

[vojkny]

Describe the bug
We have a custom jackson object mapper which serializes enums to lowercase. The same objectMapper is used by OpenApi for serialization, but the enums are still shown uppercase.

To Reproduce

Minimal example: https://gitlab.com/gooutopensource/openuidemo/-/tree/main

If you open the /docs endpoint, it will show enums uppercased, response will have them lowercased - see screenshots below:

Expected behavior

• Enums should show lowercase both for OpenApi and for Spring responses.

Screenshots
Response enum is correctly lowercased:

OpenApi enum is uppercased.

[bnasslahsen]

@knyttl,

You can customize serialization, by just registering your modules.
See answer here #953.

I cannot use your project. Please, make sure you add a complete spring-boot sample if you need help.

[vojkny]

I cannot use your project. Please, make sure you add a complete spring-boot sample if you need help.

I migrated to Spring Boot, see now (was using Vanilla Spring MVC). I can now run it with:

./gradlew bootRun

And then I can see the demo for:

1. http://localhost:8080/v3/api-docs
2. http://localhost:8080/test

You can customize serialization, by just registering your modules.

This is what I am actually doing in the project, but it does seem to work ONLY for endpoint outputs, not for OpenApi.

(excerpt from the project)

val lowerCaseEnumJacksonSerializerModule = SimpleModule().also {
    val lowerCaseEnumKeySerializer = object : StdSerializer<Enum<*>>(Enum::class.java) {
        override fun serialize(value: Enum<*>?, json: JsonGenerator, provider: SerializerProvider) {
            json.writeFieldName(value?.name?.lowercase())
        }
    }
    val lowerCaseEnumValueSerializer = object : StdSerializer<Enum<*>>(Enum::class.java) {
        override fun serialize(value: Enum<*>?, json: JsonGenerator, provider: SerializerProvider) {
            json.writeString(value?.name?.lowercase())
        }
    }
    it.addKeySerializer(Enum::class.java, lowerCaseEnumKeySerializer)
    it.addSerializer(Enum::class.java, lowerCaseEnumValueSerializer)
}

val jackson = ObjectMapper().registerModule(KotlinModule())
    .registerModule(lowerCaseEnumJacksonSerializerModule)

...

@Bean
open fun objectMapper(): ObjectMapper = jackson

[bnasslahsen]

@knyttl,

This will not work as swagger-core scans directly for enum values based on enumConstants / enumIndex or toString.
It also scans for @JsonValue annotation, but does not reuse custom enum serializers which is your case.

https://github.com/swagger-api/swagger-core/blob/6ccd2f8132196ba8e51e3e0029c8c3ea4448825b/modules/swagger-core/src/main/java/io/swagger/v3/core/jackson/ModelResolver.java#L936-L974

As workaround, you have to provide your own ModelConverter as a spring bean:

• https://github.com/swagger-api/swagger-core/wiki/Swagger-2.X---Extensions#extending-core-resolver

This a sample code that you can adapt:

@Component
class EnumLowerCaseConverter : ModelConverter {
	override fun resolve(
		type: AnnotatedType,
		context: ModelConverterContext,
		chain: Iterator<ModelConverter>
	): Schema<*>? {
		val javaType = Json.mapper().constructType(type.type)
		if (javaType != null && javaType.isEnumType) {
			val enumClass = javaType.rawClass as Class<Enum<*>>
			val enumConstants = enumClass.enumConstants
			val stringSchema = StringSchema()
			for (en in enumConstants) {
				val enumValue = en.name.lowercase(Locale.getDefault())
				stringSchema.addEnumItem(enumValue)
			}
			return stringSchema
		}
		return if (chain.hasNext()) chain.next().resolve(type, context, chain) else null
	}
}

[vojkny]

Hi, thanks, I will try this.

However - why don't you use the given jackson directly for enum serialization?

[bnasslahsen]

As explained, Jackson configuration is directly managed by swagger-core and not in springdoc-openapi..

[ch4mpy]

@vojkny of course, swagger-core rules only Swagger stuff, and not what Spring uses at runtime to (de)serialize REST APIs payloads (otherwise , what we get in the OpenAPI spec generated by Swagger and as input / output of Spring REST APIs would be the same).

As the schema generation happens in a running Spring application, it is possible to scan the app context for registered converters and then use it in a Swagger ModelConverter to put in the schema the enumerated values that Spring APIs actually accept and produce. It could be possible to register such a smart ModelConverter for enums in Spring apps in springdoc-openapi, but the owner of the repo apparently doesn't want to.

If working with a Spring Boot 3 servlet, you can try to add com.c4-soft.springaddons:spring-addons-starter-openapi to your dependencies. This should auto-configure a ModelConverter using Spring:

• HttpMessageConverter which is what Spring uses for @RequestBody and @ResponseBody
• FormattingConversionService which is what Spring uses for @RequestParam, @RequestHeader, @PathVariable, @MatrixVariable and @CookieValue

If you don't want to use the starter (which is very, very small: it loads only one config file which instantiate a single bean), you can still copy from the source, but you might introduce some tight coupling to swagger-core everywhere you copy this ModelConverter.

Unfortunately, I haven't found a way to synchronize the HttpMessageWriter used to serialize @ResponseBody in reactive apps, so the starter doesn't work in WebFlux apps (yet?).