Question - MultiPartUpload response content type

[sfali]

Hello,

This is more of a question, I am trying to figure out where is the gap.

We are using Scala as programming language, for reading and writing to S3 we are using Alpakka, which provide us streaming.
We are using multi part upload which works fine when directly using Amazon S3 but when using agains S3Mock we get following error:

Unsupported Content-Type, supported: application/xml, application/octet-stream

We are also tried other S3 mock and it works fine as well.

Any idea what is going on?

Your response will be appreciated.

Regards,

Syed

[agudian]

Interesting. Which request (or part of the multipart upload sequence) is causing the trouble?

I can see that for some of those methods we don't have a Content-Type defined... Would be nice if you could help us sort this out.

[sfali]

After further investigation, looks like it is failing on initiateMultipartUpload, interestingly this method producing application/x-www-form-urlencoded .

[sfali]

I tried changing application/x-www-form-urlencoded to application/octet-stream, the error I got back in my client was:

org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Premature end of file.

[an-tex]

I've just ran into the same issue. The problem is that the S3Mock Server (not the client) returns a Content-Type: application/x-www-form-urlencoded in the response of the initiateMultipartUpload endpoint (https://github.com/adobe/S3Mock/blob/master/server/src/main/java/com/adobe/testing/s3mock/FileStoreController.java#L969). The actual AWS endpoint returns a application/octet-stream (just checked), this (or application/xml) is what the strict Akka HTTP Client (which is used under the hood by Alpakka S3) expects for a XML response and fails otherwise.

The official AWS S3 documentation (https://docs.aws.amazon.com/AmazonS3/latest/API/API_CreateMultipartUpload.html) unfortunately doesn't define a Content-Type for the response. It seems most clients ignore the Content-Type of the Response and just parse it as XML, that's why this issue isn't so common. Nevertheless I believe for the S3 Mock to be an HTTP wise correct and AWS wise accurate implementation, it should also return the application/octet-stream Content-Type at least for this endpoint.

I'd be happy to create a PR if others think the same way.

[DanieleSassoli]

hey @an-tex, very good analysis, helped me understanding the issue a lot better.
Where you planning on raising a PR for this? I had the same issue and worked around it by using https://github.com/findify/s3mock but it seems like overall the project is not as well maintained as this one.

[an-tex]

I hoped for a feedback by the maintainers first if there's a reason for defining explicitly that content type. But seems that's not gonna happen ;) In the meantime I'm using https://github.com/gaul/s3proxy but would also rather switch back to this.

[agudian]

@an-tex thanks for looking into it and finding the discrepancy. I think using the same content type as AWS does is the way to go.

A PR would be very welcome! 👍

[an-tex]

Thanks for the reply @agudian ! I'll make a PR next week. Cheers

[DanieleSassoli]

hey @an-tex, did you find any time to raise a PR for this? Might look into doing this myself if it's of any help...

[an-tex]

Hi @DanieleSassoli . I've made the changes locally just yesterday but some tests started failing. I'll look into it later, if it's something I could need assistance with I'll open a PR with the failing tests so we can continue together? Cheers

[an-tex]

alright the issue is when annotating an endpoint with:

@RequestMapping(/*...*/ produces = {"application/octet-stream"})

the integration tests fail as the S3Mock returns a "400 - Bad Request" for most of the calls. This is the spring framework itself rejecting the request. I'm not sure why. Maybe it just doesn't know how to serialize the response? When I put application/xml it all works. So this might still be a better solution than the previous application/x-www-form-urlencoded? Any thoughts? I'll try out how the strict Akka HTTP Client handles this. If it's fine with this I'll just go ahead and create a PR then we can shift the implementation discussion there.

[DanieleSassoli]

I believe this should be closed following #198?

[timoe]

yepp, you're right.