Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reactive WebClient corrupts uploaded files [SPR-16246] #20793

Closed
spring-issuemaster opened this issue Nov 30, 2017 · 3 comments

Comments

Projects
None yet
2 participants
@spring-issuemaster
Copy link
Collaborator

commented Nov 30, 2017

Matúš Sekáč opened SPR-16246 and commented

When I try to upload some ordinary file through reactive WebClient, the file ends up corrupted. It happens with audio files as well as text files. I tried creating request body manually or via MultiPartBodyBuilder in newer version and the result was same. The behavior is same when using WebTestClient.

I also tried uploading to Apache web server and the files were still corrupted.

Sample test:

	@Test
	public void audioFile() throws IOException {
		ClassPathResource resource = new ClassPathResource(TEST_WAV_NAME);

		LinkedMultiValueMap<String, HttpEntity<?>> body = new LinkedMultiValueMap<>();
		HttpHeaders headers = new HttpHeaders();
		headers.setContentType(MediaType.APPLICATION_OCTET_STREAM);
		HttpEntity<ClassPathResource> part = new HttpEntity<>(resource, headers);
		body.add("file", part);

		doTest(hash(resource), body);
	}

	private static long hash(AbstractFileResolvingResource resource) throws IOException {
		CRC32 crc = new CRC32();
		crc.update(Files.readAllBytes(resource.getFile().toPath()));
		return crc.getValue();
	}

	private void doTest(long hash, LinkedMultiValueMap<String, HttpEntity<?>> body) throws IOException {
		String result = WebClient.builder()
				.baseUrl("http://localhost:8080")
				.build()
				.post()
				.uri("/upload")
				.body(BodyInserters.fromMultipartData(body))
				.retrieve()
				.bodyToMono(String.class)
				.block();
		Assert.assertEquals(String.valueOf(hash), result);
	}

And corresponding controller:

@PostMapping(value = "/upload", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
public Mono<String> upload(@RequestPart("file") FilePart part) throws IOException {
     File tf = Files.createTempFile("spring-test", "temp").toFile();
     part.transferTo(tf);
     CRC32 crc = new CRC32();
     crc.update(Files.readAllBytes(tf.toPath()));
     return Mono.just(String.valueOf(crc.getValue()));
}

The whole sample project with tested files is attached.


Affects: 5.0 GA

Attachments:

Referenced from: commits b36af8a, 91d3e44

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Nov 30, 2017

Rossen Stoyanchev commented

The correct way to code the controller is something like this:

@PostMapping(value = "/upload", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
public Mono<String> upload(@RequestPart("file") FilePart part) throws IOException {

	File tf = Files.createTempFile("spring-test", "temp").toFile();
	Path path = tf.toPath();

	return part.transferTo(tf)
			.then(Mono.defer(() -> {
				CRC32 crc = new CRC32();
				try {
					byte[] bytes = Files.readAllBytes(path);
					System.out.println("length: " + bytes.length);
					crc.update(bytes);
				}
				catch (IOException e) {
					throw new IllegalStateException("");
				}
				long value = crc.getValue();
				System.out.println(value);
				return Mono.just(String.valueOf(value));
			}));
}

That said there does appear to be some issue. The (large) text file in the same project also doesn't upload correctly, but a smaller file does. Possibly an issue in the MultipartHttpMessageWriter based on the fact that it doesn't upload correctly to another server either. I will try to create a test based on that.

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Dec 1, 2017

Rossen Stoyanchev commented

This took a while but I managed to narrow it down. I created a test in our own tests (ignored for now). I suspect a Reactor Netty issue, so I've submitted a ticket there with a repro project.

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Jan 10, 2018

Rossen Stoyanchev commented

There is now a fix in Reactor Netty that I've been able to confirm using both our own test and the tests from the attached sample.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.