Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi-part upload of a byte[] with WebTestClient is Base64 encoded (?) and assigned a JSON content type [SPR-16350] #20897

Closed
spring-issuemaster opened this issue Jan 5, 2018 · 4 comments

Comments

Projects
None yet
2 participants
@spring-issuemaster
Copy link
Collaborator

commented Jan 5, 2018

Andy Wilkinson opened SPR-16350 and commented

Consider the following code:

MultiValueMap<String, Object> multipartData = new LinkedMultiValueMap<>();
multipartData.add("file", new byte[] { 'a', 'b', 'c', 'd' });
WebTestClient.bindToRouterFunction(RouterFunctions.route(POST("/foo"), (req) -> {
	MultiValueMap<String, Part> parts = req.body(BodyExtractors.toMultipartData())
			.block();
	Part filePart = parts.getFirst("file");
	ByteArrayOutputStream contentStream = new ByteArrayOutputStream();
	DataBufferUtils.write(filePart.content(), contentStream).blockFirst();
	System.out.println(new String(contentStream.toByteArray()));
	System.out.println(filePart.headers());
	return null;
})).configureClient().baseUrl("http://localhost").build().post().uri("/foo")
		.body(BodyInserters.fromMultipartData(multipartData)).exchange()
		.expectBody().returnResult();

With recent 5.0.3 snapshots it produces the following output:

["YWJjZA=="]
{content-disposition=[form-data; name="file"], content-type=[application/json;charset=UTF-8]}

With 5.0.2.RELEASE the following output is produced:

abcd
{content-disposition=[form-data; name="file"]}

There also appears to be a related regression when using a ByteArrayResource that results in an exception, presumably because it's being treated as JSON:

reactor.core.Exceptions$ErrorCallbackNotImplemented: org.springframework.core.codec.CodecException: Type definition error: [simple type, class java.io.ByteArrayInputStream]; nested exception is com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class java.io.ByteArrayInputStream and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) (through reference chain: java.util.LinkedList[0]->org.springframework.restdocs.webtestclient.WebTestClientRequestConverterTests$1["inputStream"])
Caused by: org.springframework.core.codec.CodecException: Type definition error: [simple type, class java.io.ByteArrayInputStream]; nested exception is com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class java.io.ByteArrayInputStream and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) (through reference chain: java.util.LinkedList[0]->org.springframework.restdocs.webtestclient.WebTestClientRequestConverterTests$1["inputStream"])
	at org.springframework.http.codec.json.AbstractJackson2Encoder.encodeValue(AbstractJackson2Encoder.java:136)
	at org.springframework.http.codec.json.AbstractJackson2Encoder.lambda$encode$0(AbstractJackson2Encoder.java:100)
	at org.springframework.http.codec.json.AbstractJackson2Encoder$$Lambda$129/1362546706.apply(Unknown Source)
	at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:107)
	at reactor.core.publisher.FluxJust$WeakScalarSubscription.request(FluxJust.java:91)
	at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:156)
	at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.set(Operators.java:1463)
	at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onSubscribe(Operators.java:1337)
	at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:90)
	at reactor.core.publisher.FluxJust.subscribe(FluxJust.java:68)
	at reactor.core.publisher.FluxMapFuseable.subscribe(FluxMapFuseable.java:63)
	at reactor.core.publisher.Flux.subscribe(Flux.java:6621)
	at reactor.core.publisher.FluxConcatArray$ConcatArraySubscriber.onComplete(FluxConcatArray.java:200)
	at reactor.core.publisher.FluxConcatArray.subscribe(FluxConcatArray.java:80)
	at reactor.core.publisher.MonoIgnoreElements.subscribe(MonoIgnoreElements.java:37)
	at reactor.core.publisher.Mono.subscribe(Mono.java:3008)
	at reactor.core.publisher.Mono.subscribeWith(Mono.java:3116)
	at reactor.core.publisher.Mono.subscribe(Mono.java:2893)
	at org.springframework.http.codec.multipart.MultipartHttpMessageWriter.encodePart(MultipartHttpMessageWriter.java:262)
	at org.springframework.http.codec.multipart.MultipartHttpMessageWriter.lambda$encodePartValues$3(MultipartHttpMessageWriter.java:225)
	at org.springframework.http.codec.multipart.MultipartHttpMessageWriter$$Lambda$127/57497692.apply(Unknown Source)
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
	at java.util.LinkedList$LLSpliterator.forEachRemaining(LinkedList.java:1235)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:512)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:502)
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
	at org.springframework.http.codec.multipart.MultipartHttpMessageWriter.encodePartValues(MultipartHttpMessageWriter.java:225)
	at org.springframework.http.codec.multipart.MultipartHttpMessageWriter.lambda$writeMultipart$2(MultipartHttpMessageWriter.java:209)
	at org.springframework.http.codec.multipart.MultipartHttpMessageWriter$$Lambda$80/576020159.apply(Unknown Source)
	at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.drain(FluxConcatMap.java:357)
	at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.onSubscribe(FluxConcatMap.java:210)
	at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:128)
	at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:61)
	at reactor.core.publisher.FluxConcatMap.subscribe(FluxConcatMap.java:121)
	at reactor.core.publisher.Flux.subscribe(Flux.java:6621)
	at reactor.core.publisher.FluxConcatArray$ConcatArraySubscriber.onComplete(FluxConcatArray.java:200)
	at reactor.core.publisher.FluxConcatArray.subscribe(FluxConcatArray.java:80)
	at reactor.core.publisher.FluxPeek.subscribe(FluxPeek.java:83)
	at reactor.core.publisher.FluxPeek.subscribe(FluxPeek.java:83)
	at reactor.core.publisher.FluxPeek.subscribe(FluxPeek.java:83)
	at reactor.core.publisher.FluxPeek.subscribe(FluxPeek.java:83)
	at reactor.core.publisher.Flux.subscribe(Flux.java:6621)
	at reactor.core.publisher.Flux.subscribeWith(Flux.java:6788)
	at reactor.core.publisher.Flux.subscribe(Flux.java:6614)
	at reactor.core.publisher.Flux.subscribe(Flux.java:6578)
	at org.springframework.http.codec.multipart.SynchronossPartHttpMessageReader$SynchronossPartGenerator.accept(SynchronossPartHttpMessageReader.java:133)
	at org.springframework.http.codec.multipart.SynchronossPartHttpMessageReader$SynchronossPartGenerator.accept(SynchronossPartHttpMessageReader.java:109)
	at reactor.core.publisher.FluxCreate.subscribe(FluxCreate.java:92)
	at reactor.core.publisher.MonoCollect.subscribe(MonoCollect.java:66)
	at reactor.core.publisher.MonoMapFuseable.subscribe(MonoMapFuseable.java:59)
	at reactor.core.publisher.Mono.block(Mono.java:1161)
	at org.springframework.restdocs.webtestclient.WebTestClientRequestConverterTests.lambda$8(WebTestClientRequestConverterTests.java:187)
	at org.springframework.restdocs.webtestclient.WebTestClientRequestConverterTests$$Lambda$10/1638215613.handle(Unknown Source)
	at org.springframework.web.reactive.function.server.RouterFunctions.lambda$null$3(RouterFunctions.java:232)
	at org.springframework.web.reactive.function.server.RouterFunctions$$Lambda$116/945722724.get(Unknown Source)
	at org.springframework.web.reactive.function.server.RouterFunctions.wrapException(RouterFunctions.java:240)
	at org.springframework.web.reactive.function.server.RouterFunctions.lambda$null$4(RouterFunctions.java:232)
	at org.springframework.web.reactive.function.server.RouterFunctions$$Lambda$110/769798433.apply(Unknown Source)
	at reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:141)
	at reactor.core.publisher.MonoFlatMap.subscribe(MonoFlatMap.java:53)
	at reactor.core.publisher.MonoFlatMap.subscribe(MonoFlatMap.java:60)
	at reactor.core.publisher.MonoOnErrorResume.subscribe(MonoOnErrorResume.java:44)
	at reactor.core.publisher.Mono.subscribe(Mono.java:3008)
	at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.drain(MonoIgnoreThen.java:167)
	at reactor.core.publisher.MonoIgnoreThen.subscribe(MonoIgnoreThen.java:56)
	at reactor.core.publisher.Mono.subscribe(Mono.java:3008)
	at reactor.core.publisher.Mono.subscribeWith(Mono.java:3116)
	at reactor.core.publisher.Mono.subscribe(Mono.java:3002)
	at reactor.core.publisher.Mono.subscribe(Mono.java:2969)
	at reactor.core.publisher.Mono.subscribe(Mono.java:2941)
	at org.springframework.test.web.reactive.server.HttpHandlerConnector.lambda$connect$1(HttpHandlerConnector.java:90)
	at org.springframework.test.web.reactive.server.HttpHandlerConnector$$Lambda$65/1108924067.apply(Unknown Source)
	at org.springframework.mock.http.client.reactive.MockClientHttpRequest.lambda$null$2(MockClientHttpRequest.java:125)
	at org.springframework.mock.http.client.reactive.MockClientHttpRequest$$Lambda$95/945591847.get(Unknown Source)
	at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:44)
	at reactor.core.publisher.Mono.subscribe(Mono.java:3008)
	at reactor.core.publisher.FluxConcatIterable$ConcatIterableSubscriber.onComplete(FluxConcatIterable.java:141)
	at reactor.core.publisher.FluxConcatIterable.subscribe(FluxConcatIterable.java:60)
	at reactor.core.publisher.MonoSourceFlux.subscribe(MonoSourceFlux.java:47)
	at reactor.core.publisher.Mono.subscribe(Mono.java:3008)
	at reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:172)
	at reactor.core.publisher.MonoFlatMap.subscribe(MonoFlatMap.java:53)
	at reactor.core.publisher.Mono.subscribe(Mono.java:3008)
	at reactor.core.publisher.Mono.subscribeWith(Mono.java:3116)
	at reactor.core.publisher.Mono.subscribe(Mono.java:3002)
	at reactor.core.publisher.Mono.subscribe(Mono.java:2969)
	at reactor.core.publisher.Mono.subscribe(Mono.java:2941)
	at org.springframework.test.web.reactive.server.HttpHandlerConnector.connect(HttpHandlerConnector.java:101)
	at org.springframework.test.web.reactive.server.WiretapConnector.connect(WiretapConnector.java:73)
	at org.springframework.web.reactive.function.client.ExchangeFunctions$DefaultExchangeFunction.exchange(ExchangeFunctions.java:74)
	at org.springframework.web.reactive.function.client.DefaultWebClient$DefaultRequestBodyUriSpec.exchange(DefaultWebClient.java:326)
	at org.springframework.test.web.reactive.server.DefaultWebTestClient$DefaultRequestBodyUriSpec.exchange(DefaultWebTestClient.java:282)
	at org.springframework.restdocs.webtestclient.WebTestClientRequestConverterTests.multipartUploadFromResource(WebTestClientRequestConverterTests.java:191)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:678)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
Caused by: com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class java.io.ByteArrayInputStream and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) (through reference chain: java.util.LinkedList[0]->org.springframework.restdocs.webtestclient.WebTestClientRequestConverterTests$1["inputStream"])
	at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77)
	at com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191)
	at com.fasterxml.jackson.databind.DatabindContext.reportBadDefinition(DatabindContext.java:312)
	at com.fasterxml.jackson.databind.ser.impl.UnknownSerializer.failForEmpty(UnknownSerializer.java:71)
	at com.fasterxml.jackson.databind.ser.impl.UnknownSerializer.serialize(UnknownSerializer.java:33)
	at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
	at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
	at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
	at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serializeContents(CollectionSerializer.java:145)
	at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:107)
	at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:25)
	at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
	at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:400)
	at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1392)
	at com.fasterxml.jackson.databind.ObjectWriter._configAndWriteValue(ObjectWriter.java:1120)
	at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:950)
	at org.springframework.http.codec.json.AbstractJackson2Encoder.encodeValue(AbstractJackson2Encoder.java:133)
	... 117 more

Affects: 5.0.3

Issue Links:

  • #20854 Support Publishers for multipart data in BodyInserters

Referenced from: commits 93a522f

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Jan 5, 2018

Andy Wilkinson commented

The first problem can be avoided by explicitly setting the part's content type:

MultipartBodyBuilder bodyBuilder = new MultipartBodyBuilder();
bodyBuilder.part("file", new byte[] { 1, 2, 3, 4 },
		MediaType.APPLICATION_OCTET_STREAM);
WebTestClient.bindToRouterFunction(RouterFunctions.route(POST("/foo"), (req) -> {
	req.body(BodyExtractors.toMultipartData()).block();
	return null;
})).configureClient().baseUrl("http://localhost").build().post().uri("/foo")
		.body(BodyInserters.fromObject(bodyBuilder.build())).exchange()
		.expectBody().returnResult();
@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Jan 5, 2018

Andy Wilkinson commented

As can the second:

MultipartBodyBuilder bodyBuilder = new MultipartBodyBuilder();
bodyBuilder.part("file", new ByteArrayResource(new byte[] { 1, 2, 3, 4 }) {

	@Override
	public String getFilename() {
		return "image.png";
	}

}, MediaType.IMAGE_PNG);
WebTestClient.bindToRouterFunction(RouterFunctions.route(POST("/foo"), (req) -> {
	req.body(BodyExtractors.toMultipartData()).block();
	return null;
})).configureClient().baseUrl("http://localhost").build().post().uri("/foo")
		.body(BodyInserters.fromObject(bodyBuilder.build())).exchange()
		.expectBody().returnResult();

However, this rather seems to defeat the purpose of using a Resource where, in 5.0.2, the content type of the part was inferred from its file name.

@spring-issuemaster

This comment has been minimized.

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Jan 5, 2018

Rossen Stoyanchev commented

Indeed this was introduced very recently with #7035ee when BodyInserters was updated to use MultipartBodyBuilder internally. The part method is only prepared to take a single value but we were passing the value from a MultiValueMap instead (i.e. a List) which caused Jackson to get involved. Thanks for catching it before it was even released!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.