AWS S3 upload access denied ACL public-read

[lowki]

Describe the bug
When I try to upload a file with AWS S3 upload provider, I have a "denied access" error.

Steps to reproduce the behavior

1. Use npm v12.16.1
2. npx create-strapi-app my-project --quickstart
3. cd my-project
4. yarn add strapi-provider-upload-aws-s3
5. Create settings file at extensions/upload/config/settings.json

{
    "provider": "aws-s3",
    "providerOptions": {
      "accessKeyId": "<accesskeyid>",
      "secretAccessKey": "<secretaccesskey>",
      "region": "eu-west-3",
      "params": {
        "Bucket": "<bucket-name>"
      }
    }
  }

6. Go to Media Library in admin panel and upload a file

Expected behavior
The file should be uploaded

Screenshots

System

• Node.js version: 12.6.1
• NPM version: 6.13.4
• Strapi version: 3.0.0-beta.20
• Database: sqlite
• Operating system: Archlinux

Additional context
I've tested my AWS credentials through aws cli to ensure it have access to upload to the bucket.

[derrickmehaffy]

Unsure if this is a bug or not so I'll leave it here.

[lowki]

It's probably a misconfiguration, but in this case the documentation is wrong :)

[B1Z1]

Maybe you have missconfiguration, bacause It works on my server

[lowki]

Maybe, how can I have a more verbose log? I'd like to see what actual credentials are used.

[lowki]

I've just made a successful test in the project, so using the same npm modules.

const fs = require('fs')
const AWS = require('aws-sdk');
const s3 = new AWS.S3({
    region: 'eu-west-3',
    accessKeyId: 'EXAMPLEACCESSKEYID',
    secretAccessKey: 'SECRETEXAMPLEACCESSKEY'
});
 
const fileName = 'test.txt';

const uploadFile = () => {
  fs.readFile(fileName, (err, data) => {
     if (err) throw err;
     const params = {
         Bucket: 'my-bucket-name',
         Key: 'test.txt',
         Body: JSON.stringify(data, null, 2)
     };
     s3.upload(params, function(s3Err, data) {
         if (s3Err) throw s3Err
         console.log(`File uploaded successfully at ${data.Location}`)
     });
  });
};

uploadFile();

[lowki]

I think I've found someting in the headers of the request.

HttpRequest {
  method: 'PUT',
  path: '/backpack_0c14b827a8.png',
  headers: {
    'User-Agent': 'aws-sdk-nodejs/2.658.0 linux/v12.16.1 callback',
    'x-amz-acl': 'public-read',
    'Content-Type': 'image/png',
    'Content-MD5': 'pRQJQ94PQp6z0YhcrrzTRA==',
    'Content-Length': 6120,
    Host: 'my-bucket-name.s3.eu-west-3.amazonaws.com',
    'X-Amz-Content-Sha256': 'UNSIGNED-PAYLOAD',
    'X-Amz-Date': '20200419T144408Z',
    Authorization: 'AWS4-HMAC-SHA256 Credential=##########/20200419/eu-west-3/s3/aws4_request, SignedHeaders=content-md5;host;x-amz-acl;x-amz-content-sha256;x-amz-date, Signature=19d8ca78a640ddf0c7529240d19493928af83680ae9380a1194c5a94165b45c2'
  },

The upload tries to specify a public-read acl to the file. My bucket doesn't have a public access, which is what I want, so It throw an access denied error.

I don't know if it's a bug or an expected behavior, but it would be nice to document it :)

[christianguevara]

I think you DO need to have public read access, if you don't have it, Strapi would not be able to render the content. Right now, all the uploads are meant to be public. Maybe write in the docs that little configuration needed in the S3 bucket.

[vmarchesin]

+1 I have the same error. aws-cli works so credentials are correct. I have my bucket provide the images through a CDN, so I don't need public-read access. I just with to upload the images to the bucket itself and use the CDN links afterwards. Is it possible to bypass this requirement?

[vmarchesin]

So the public-read in the request is indeed the problem. strapi-provider-upload-aws-s3 has ACL: 'public-read' hardcoded, so if your bucket doesn't have public read access it won't work indeed. This is not how that it should work. I have a CDN that serves my bucket files and I don't want the bucket itself to be public.

So I copied the provider, followed the steps in the docs and modified it just a little bit. All I had to do is remove the hardcoded property and change the file.url that is saved to the DB. Now it stores the image in the bucket but the URL points to the CDN, so everything works as expected. When deleting the file it just needs the Key so it deletes correctly as well.

Strapi renders the content fine because the URL it is using is the CDN, not the bucket.

const AWS = require('aws-sdk');

module.exports = {
  init(providerOptions) {
    const s3 = new AWS.S3({
      accessKeyId: providerOptions.accessKeyId,
      secretAccessKey: providerOptions.secretAccessKey,
    });
    const Bucket = providerOptions.params.Bucket;

    return {
      upload(file, customParams = {}) {
        return new Promise((resolve, reject) => {
          const Key = `${file.hash}${file.ext}`;
          s3.upload(
            {
              Key,
              Body: Buffer.from(file.buffer, 'binary'),
              // ACL: 'public-read', // don't use this
              ContentType: file.mime,
              Bucket,
              ...customParams,
            },
            (err, data) => {
              if (err) return reject(err);
              // set the file url to the CDN instead of the bucket itself
              file.url = `https://myCDNurl/${Key}`;
              resolve();
            }
          );
        });
      },
      delete(file, customParams = {}) {
        return new Promise((resolve, reject) => {
          s3.deleteObject(
            {
              Key: `${file.hash}${file.ext}`,
              Bucket,
              ...customParams,
            },
            (err, data) => {
              if (err) return reject(err);
              resolve();
            }
          );
        });
      },
    };
  },
};

This should be changed. Enforcing public-read should not be done.

[lauriejim]

Hello! Here is a video I made about this, maybe it can help.
https://www.youtube.com/watch?v=vrLmZCOTj6Y&list=PL7Q0DQYATmvhlHxHqfKHsr-zFls2mIVTi&index=12&t=0s
If you think there is an issue, can you please submit a PR.

[vmarchesin]

The solution, in my opinion, would be to remove the ACL: 'public-read' property from the s3 call (it can still be passed down using the customParams and pass down some extra config for file.url. If no custom url is passed it saves the file to the bucket itself, keeping the current behavior. If you pass a custom url it will assign the custom url to the file instead. Something like:

if (customParams.customUrl) {
  file.url = `${customParams.customUrl}/${Key}`
} else {
  file.url = data.Location // current behavior
}

This would allow for a custom CDN to be used instead of the bucket, just like in my example above.

Only thing I didn't figure is where those customParams are coming from, I didn't have time to dig deeper.

[southkeys]

I'm having the same issue and have been going at it for the last dozen hours. I have succeeded manually uploading via the aws-sdk with the same credentials.

Is there a particular list of permissions required from aws ?

I have a vanilla strapi instance configured as per docs, a public s3 bucket as per @lauriejim 's video and a user with the following policy attached

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:ListBucket",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::mybucketname",
                "arn:aws:s3:::mybucketname/*"
            ]
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": "s3:ListAllMyBuckets",
            "Resource": "*"
        }
    ]
}

[southkeys]

Also no success registering a new provider, and the only way I got upload to s3 working locally is by commenting out within node_modules/strapi-prodiver-upload-aws-s3/lib/index.js

ACL: 'public-read'

like pointed out by @vmarchesin

[lauriejim]

Hello! Can you please submit a PR with the right way to manage that.
Can you also write a great PR body to help for the review.
We will need to make sure it's also well documented in the readme of the provider.
Thank you!