AWS putObject is very slow due to single byte reads in AwsChunkedDecodingInputStream

[risdenk]

AwsChunkedDecodingInputStream only implements the read() method which is one byte at a time.

https://github.com/adobe/S3Mock/blame/main/server/src/main/java/com/adobe/testing/s3mock/util/AwsChunkedDecodingInputStream.java#L72

This results in slow uploads due to repeated one byte reads from the stream. A simple solution is to wrap the provided source InputStream with BufferedInputStream which avoids the repeated one byte reads.

We have a test in Apache Solr that was taking 11s to due a 10mb file upload and after this simple change results in the test taking <1s. The simple change being replace this.source = source with this.source = new BufferedInputStream(source); in AwsChunkedDecodingInputStream

https://github.com/adobe/S3Mock/blame/main/server/src/main/java/com/adobe/testing/s3mock/util/AwsChunkedDecodingInputStream.java#L68

This shows a profiler running for the test and ~80% of the samples came from AwsChunkedDecodingInputStream#read

[afranken]

@risdenk thanks for opening this issue.
The fix is simple, unfortunately the S3Mock release is currently blocked due to a problem pushing images to Docker Hub. I hope to get this resolved soon.

[risdenk]

@afranken any luck w/ the releases?

[afranken]

@risdenk finally got the release up and running again, just re-released the changes from January / February today.
I'll look into making the next patch release this week.

[afranken]

@risdenk just released 2.12.2 :)

[risdenk]

Thanks @afranken!

[JJRdec]

Hi all, I'm still seeing slow uploads. This actually results in read timeout exception on upload of large files. We are using the latest (3.1.0) release.

Are we sure this has been fixed @afranken ?

As a workaround, you can set read timeout to infinite (not ideal).

[afranken]

@JJRdec Could you please open a new ticket with a description on how to reproduce the issue?

I just tested uploading a 1GB file:

truncate -s 1G output.file
aws s3 cp ./output.file s3://bucket-a/ --endpoint-url=http://localhost:9090

The upload itself is really fast, at least locally when running S3Mock and copying a 1GB file to a bucket, it uploads with 250MB/s.
AWS CLI is using a multipart upload automatically. What is taking a while is the process of merging all parts into one file during the completion of the multipart.

Please keep in mind that S3Mock is meant to be used for local integration testing.
Usually, small files are sufficient to test this.