WebFlux incorrectly falls back to application/json for method that produces application/octet-stream and returns ResponseEntity<Object> [SPR-15910]

[spring-issuemaster]

Andy Wilkinson opened SPR-15910 and commented

The problem is illustrated by this sample application:

package com.example.demo;

import org.reactivestreams.Publisher;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.core.io.ByteArrayResource;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

import reactor.core.publisher.Mono;

@SpringBootApplication
@RestController
public class WebFluxResponseEntityBugApplication {

	public static void main(String[] args) {
		SpringApplication.run(WebFluxResponseEntityBugApplication.class, args);
	}

	@RequestMapping(path="publisher-wildcard", produces=MediaType.APPLICATION_OCTET_STREAM_VALUE)
	public Publisher<ResponseEntity<?>> publisherWildcard() {
		return Mono.just(new ResponseEntity<>(new ByteArrayResource("foo".getBytes()), HttpStatus.OK));
	}

	@RequestMapping(path="publisher-object", produces=MediaType.APPLICATION_OCTET_STREAM_VALUE)
	public Publisher<ResponseEntity<Object>> publisherObject() {
		return Mono.just(new ResponseEntity<>(new ByteArrayResource("bar".getBytes()), HttpStatus.OK));
	}

	@RequestMapping(path="wildcard", produces=MediaType.APPLICATION_OCTET_STREAM_VALUE)
	public ResponseEntity<?> wildcard() {
		return new ResponseEntity<>(new ByteArrayResource("foo".getBytes()), HttpStatus.OK);
	}

	@RequestMapping(path="object", produces=MediaType.APPLICATION_OCTET_STREAM_VALUE)
	public ResponseEntity<Object> object() {
		return new ResponseEntity<>(new ByteArrayResource("bar".getBytes()), HttpStatus.OK);
	}

}

Requests to wildcard and publisher-wildcard produce the expected 200 response. Requests to object and publisher-object produce a 500 triggered by this exception:

org.springframework.core.codec.CodecException: Type definition error: [simple type, class java.io.ByteArrayInputStream]; nested exception is com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class java.io.ByteArrayInputStream and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) (through reference chain: org.springframework.core.io.ByteArrayResource["inputStream"])
	at org.springframework.http.codec.json.AbstractJackson2Encoder.encodeValue(AbstractJackson2Encoder.java:133) ~[spring-web-5.0.0.BUILD-SNAPSHOT.jar:5.0.0.BUILD-SNAPSHOT]
	at org.springframework.http.codec.json.AbstractJackson2Encoder.lambda$encode$0(AbstractJackson2Encoder.java:97) ~[spring-web-5.0.0.BUILD-SNAPSHOT.jar:5.0.0.BUILD-SNAPSHOT]
	at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:107) [reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.FluxJust$WeakScalarSubscription.request(FluxJust.java:91) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:156) [reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at org.springframework.http.server.reactive.ChannelSendOperator$WriteBarrier.onSubscribe(ChannelSendOperator.java:143) ~[spring-web-5.0.0.BUILD-SNAPSHOT.jar:5.0.0.BUILD-SNAPSHOT]
	at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:90) [reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.FluxJust.subscribe(FluxJust.java:68) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.FluxMapFuseable.subscribe(FluxMapFuseable.java:63) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at org.springframework.http.server.reactive.ChannelSendOperator.subscribe(ChannelSendOperator.java:76) ~[spring-web-5.0.0.BUILD-SNAPSHOT.jar:5.0.0.BUILD-SNAPSHOT]
	at reactor.core.publisher.Mono.subscribe(Mono.java:2769) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:172) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.MonoFlatMap.subscribe(MonoFlatMap.java:53) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.MonoOnErrorResume.subscribe(MonoOnErrorResume.java:44) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:148) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1010) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:238) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1010) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.MonoIgnoreThen$ThenAcceptInner.onNext(MonoIgnoreThen.java:288) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:72) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onNext(FluxPeekFuseable.java:198) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:1567) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.request(FluxPeekFuseable.java:139) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.set(Operators.java:1381) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onSubscribe(FluxOnErrorResume.java:67) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onSubscribe(FluxPeekFuseable.java:173) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:161) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.MonoFlatMap.subscribe(MonoFlatMap.java:53) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.MonoPeekFuseable.subscribe(MonoPeekFuseable.java:74) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.MonoOnErrorResume.subscribe(MonoOnErrorResume.java:44) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.drain(MonoIgnoreThen.java:147) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.MonoIgnoreThen.subscribe(MonoIgnoreThen.java:56) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:148) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:67) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.MonoNext$NextSubscriber.onNext(MonoNext.java:76) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.innerNext(FluxConcatMap.java:270) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.FluxConcatMap$ConcatMapInner.onNext(FluxConcatMap.java:790) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:115) [reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:1567) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:156) [reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.set(Operators.java:1381) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onSubscribe(Operators.java:1255) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:90) [reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.MonoJust.subscribe(MonoJust.java:54) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.MonoMapFuseable.subscribe(MonoMapFuseable.java:59) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.Mono.subscribe(Mono.java:2769) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.drain(FluxConcatMap.java:414) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.onSubscribe(FluxConcatMap.java:210) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:91) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:55) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.FluxConcatMap.subscribe(FluxConcatMap.java:121) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.MonoNext.subscribe(MonoNext.java:40) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.MonoSwitchIfEmpty.subscribe(MonoSwitchIfEmpty.java:44) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.MonoFlatMap.subscribe(MonoFlatMap.java:60) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.MonoFlatMap.subscribe(MonoFlatMap.java:60) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.MonoOnErrorResume.subscribe(MonoOnErrorResume.java:44) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.MonoOnErrorResume.subscribe(MonoOnErrorResume.java:44) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.Mono.subscribe(Mono.java:2769) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.drain(MonoIgnoreThen.java:165) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.MonoIgnoreThen.subscribe(MonoIgnoreThen.java:56) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.MonoOnErrorResume.subscribe(MonoOnErrorResume.java:44) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.core.publisher.MonoPeekTerminal.subscribe(MonoPeekTerminal.java:61) ~[reactor-core-3.1.0.M3.jar:3.1.0.M3]
	at reactor.ipc.netty.channel.ChannelOperations.applyHandler(ChannelOperations.java:380) ~[reactor-netty-0.7.0.M1.jar:0.7.0.M1]
	at reactor.ipc.netty.http.server.HttpServerOperations.onHandlerStart(HttpServerOperations.java:354) ~[reactor-netty-0.7.0.M1.jar:0.7.0.M1]
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) ~[netty-common-4.1.13.Final.jar:4.1.13.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403) ~[netty-common-4.1.13.Final.jar:4.1.13.Final]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:462) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) ~[netty-common-4.1.13.Final.jar:4.1.13.Final]
	at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_121]
Caused by: com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class java.io.ByteArrayInputStream and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) (through reference chain: org.springframework.core.io.ByteArrayResource["inputStream"])
	at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77) ~[jackson-databind-2.9.0.jar:2.9.0]
	at com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191) ~[jackson-databind-2.9.0.jar:2.9.0]
	at com.fasterxml.jackson.databind.DatabindContext.reportBadDefinition(DatabindContext.java:305) ~[jackson-databind-2.9.0.jar:2.9.0]
	at com.fasterxml.jackson.databind.ser.impl.UnknownSerializer.failForEmpty(UnknownSerializer.java:71) ~[jackson-databind-2.9.0.jar:2.9.0]
	at com.fasterxml.jackson.databind.ser.impl.UnknownSerializer.serialize(UnknownSerializer.java:33) ~[jackson-databind-2.9.0.jar:2.9.0]
	at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[jackson-databind-2.9.0.jar:2.9.0]
	at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[jackson-databind-2.9.0.jar:2.9.0]
	at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[jackson-databind-2.9.0.jar:2.9.0]
	at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480) ~[jackson-databind-2.9.0.jar:2.9.0]
	at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319) ~[jackson-databind-2.9.0.jar:2.9.0]
	at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1396) ~[jackson-databind-2.9.0.jar:2.9.0]
	at com.fasterxml.jackson.databind.ObjectWriter._configAndWriteValue(ObjectWriter.java:1120) ~[jackson-databind-2.9.0.jar:2.9.0]
	at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:950) ~[jackson-databind-2.9.0.jar:2.9.0]
	at org.springframework.http.codec.json.AbstractJackson2Encoder.encodeValue(AbstractJackson2Encoder.java:130) ~[spring-web-5.0.0.BUILD-SNAPSHOT.jar:5.0.0.BUILD-SNAPSHOT]
	... 69 common frames omitted

This appears to happen because the Object in ResponseEntity<Object> causes AbstractMessageWriterResultHandler to use EncoderHttpMessageWritter rather than ResourceHttpMessageWriter.

All four request mappings work as expected with Spring MVC. I have tested with both Spring Framework 5.0.0.RC3 and 5.0.0.BUILD-SNAPSHOT.

---

Affects: 5.0 RC3

Issue Links:

• #20495 Consistent type resolution for ? and Object element type

Referenced from: commits 03eb6f7, 5d4ee09

[spring-issuemaster]

Sébastien Deleuze commented

This issue raises more question that expected initially and is not straightforward to solve.

After various thoughts and tries, I have ended up crafting this pull-request where I would like your feedback Andy Wilkinson, Arjen Poutsma, Rossen Stoyanchev and Brian Clozel.

[spring-issuemaster]

Rossen Stoyanchev commented

I've pushed a straight-forward improvement that treats wildcard and object consistently and makes the 4 use cases described here work (note the build is currently failing for unrelated reasons so a snapshot is not available yet).

I do agree it would be useful to perform a broader review of related use cases and also compare to Spring MVC. In general with flexible method signatures we try and use the available information if that's feasible.

Andy Wilkinson can you clarify the motivation for the above signatures? I presume you need to returning different types?

[spring-issuemaster]

Andy Wilkinson commented

Yes. Signatures like those illustrated above are using in the WebFlux part of Spring Boot 2.0's new Actuator endpoint infrastructure. An endpoint can return pretty much anything which we then wrap in a ResponseEntity so that we can control the status code.

[spring-issuemaster]

Sébastien Deleuze commented

I have pushed my Jackson fix as well, and I am preparing a broader review of related use cases in order to evaluate more concretely the potential problems raised on the PR.

[spring-issuemaster]

Sébastien Deleuze commented

Since Spring Boot use case is fixed by the 2 commit in master, I resolve this issue and have created #20495 to evaluate if we go further with support for handler methods declaring following types
and returning an actual Mono<Resource>:

• ResponseEntity<Publisher<Object>>
• ResponseEntity<Publisher<?>>
• Publisher<?>
• Publisher<Object>