S3 SDK fails to autodetect content-type, sets everything to "application/octet-stream"

[distinctdan]

Confirm by changing [ ] to [x] below to ensure that it's a bug:

• I've gone though Developer Guide and API reference
• I've checked AWS Forums and StackOverflow for answers
• I've searched for previous similar issues and didn't find any solution

Describe the bug
The S3 sdk fails to autodetect the content-type of files uploaded through s3.putObject and defaults them to "application/octet-stream". There was a previous issue here: #2510, but the only solution given was to manually set the content type for every file. This is unreasonable because the docs imply the sdk is supposed to do this automatically. Out of all the examples on the docs here: https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#putObject-property, none of them manually specify a content type, so this is supposed to be working automatically.

Is the issue in the browser/Node.js?
Node.js

If on Node.js, are you running this on AWS Lambda?
No

Details of the browser/Node.js version
Node v12.11.1

SDK version number
aws-sdk@2.610.0

To Reproduce (observed behavior)

const params = {
    Body: fs.readFileSync(filePath),
    Bucket: 'myBucketName',
    Key: filePath
};

s3.putObject(params, (err, data) => {
    if (err) {
        console.log('ERROR uploading file ' + filePath + ': ' + err);
    } else {
        console.log('Uploaded ' + filePath);
    }
})

[ajredniwja]

Hey @distinctdan thank-you for reaching out to us with your issue.

You are right about the way SDK adds the content type, it will add application/octet-stream if contentType is not passed in the original request. Logic mentioned here, the docs that you are referring to are generated by the service team and some of the SDKs autodetect the contentType so marking them required wont be the solution.

Since the SDK is not trying to detect the contentType at all, I would not call it a bug but a feature-request. We welcome PRs in case you wanna work on it.

[distinctdan]

Hmm, well if it's not actually supported, the docs are specific to the javascript sdk, so it seems that they should be updated to reflect that this sdk doesn't do autodetection. It's really not a big deal to me that it doesn't, it's just that it's not documented, so I had to waste time trying to figure out what was going on.

The docs are already specific to javascript, and this javascript sdk is different from some of the other sdk's in functionality, so making content-type a required property for this sdk shouldn't be a problem, since the reality is that it doesn't have the functionality that some other ones have. At the very least, it should be documented that content-type is not autodetected.

I ended up using another library, https://www.npmjs.com/package/mime-types, to detect the types for my files; I don't know how open you guys are to requiring additional node modules in your repo.

const mime = require('mime-types'); 
const mimeType = mime.lookup(fileName);

[ajredniwja]

Hey @distinctdan apologies that you have to go through the trouble. marking it required would mean that SDK should not result in successful call if the content-type is not mentioned. Which will mean to flag content-type in SDK as required as well which will be a breaking change for many of the customers.

The correct way to do this could be to specify that if no contentType is passed it will default to application/octet-stream will reach out to the service team to ask for a fix.

[distinctdan]

Ok, I gotcha, that sounds good to me.

[A-Circle-Zhang]

Hi, I'm using this SDK for a S3 Compatible storage, they support autodetect content-type on their server side, is there any way to tell SDK don't send the Content-Type header rather than default to application/octet-stream ?
I tried contentType: '' or contentType: null , not working..

I know it is not a AWS question, but i will very much appreciate it if you willing to answer my question..

[osddeitf]

Well, I wasted almost a day because of this. My object displayed with Type column in S3 Console, I thought it had been auto-detected.

[adamdry]

@ajredniwja I might be tempted to work on this feature. Where's the best place to discuss the design of how it might work? I don't want to write all the code, only for the design to be unacceptable 😄

[ajredniwja]

@ajredniwja I might be tempted to work on this feature. Where's the best place to discuss the design of how it might work? I don't want to write all the code, only for the design to be unacceptable 😄

You might wanna check the AWS JAVA SDK implementation for this.

[neilmenon]

When uploading files, the Amazon Web Services S3 Java client will attempt to determine the correct content type if one hasn't been set yet. Users are responsible for ensuring a suitable content type is set when uploading streams. If no content type is provided and cannot be determined by the filename, the default content type, "application/octet-stream", will be used.

https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/model/ObjectMetadata.html

Based on this discussion, the bolded is not applicable, right?