[V5] S3 Driver doesn't seem to work

[amitavroy]

Based on the documentation, I am trying to move a request.file to S3 using the moveToDisk function.

However, it doesn't seem to work.

The code is quite simple

await postData.banner.moveToDisk('test', {}, 's3')

Where postData.banner is validated using schema.create.

The error which I am getting is

"message": "E_CANNOT_WRITE_FILE: Cannot write file at location \"test/ckwta651w0001ax6y7m6h15hb.png\"",
    "stack": "CannotWriteFileException: E_CANNOT_WRITE_FILE: Cannot write file at location \"test/ckwta651w0001ax6y7m6h15hb.png\"\n    at Function.invoke (<my-codebase-path>/node_modules/@adonisjs/drive/build/src/Exceptions/index.js:18:23)\n    at S3Driver.putStream (<my-codebase-path>/node_modules/@adonisjs/drive-s3/build/src/Drivers/S3.js:234:57)\n    at processTicksAndRejections (internal/process/task_queues.js:95:5)\n    at File.moveToDisk (<my-codebase-path>/node_modules/@adonisjs/bodyparser/build/src/Multipart/File.js:169:9)\n    at ExpensesController.uploadFile (<my-codebase-path>/app/Modules/Expenses/Http/ExpensesController.ts:48:5)\n    at Object.PreCompiler.runRouteHandler [as fn] (<my-codebase-path>/node_modules/@adonisjs/http-server/build/src/Server/PreCompiler/index.js:44:31)\n    at Server.handleRequest (<my-codebase-path>/node_modules/@adonisjs/http-server/build/src/Server/index.js:108:13)",
    "code": "E_CANNOT_WRITE_FILE"

I know the credentials and settings are correct because the same settings are working on a sample Laravel codebase.

[thetutlage]

Drive exceptions have an original property. You can share what's the value of error.original?

[thetutlage]

Okay. So I did a bit of reading on this topic and seems like the correct way is to disable the ACL and use bucket policies. In your case, the ACL was earlier disabled, but I think the bucket did not have any policy.

Here's an article I read https://redlock.io/blog/aws-security-tips-understanding-access-controls-amazon-s3. It says, that ACL is the simplest way to go, but the Policies are the correct way to go.

---

For now, I will not add the permissions recommendation in the doc, coz I am not an expert on AWS policies. So I do not want to guide people in the wrong direction.

The best we can do is. Drop a note saying "Majority of the times, the S3 operations fails because of mis-configured bucket policies. Make sure to double check you have correct policies in place and then share a sample policy with the user"

Ability to do following operations is what Drive needs.

"s3:ListBucket",
"s3:GetObject",
"s3:GetObjectAcl",
"s3:PutObject",
"s3:PutObjectAcl",
"s3:ReplicateObject",
"s3:DeleteObject"

[ad-on-is]

"s3:ListBucket",
"s3:GetObject",
"s3:GetObjectAcl",
"s3:PutObject",
"s3:PutObjectAcl",
"s3:ReplicateObject",
"s3:DeleteObject"

Could you please add this to the main documentation. I stumbled upon this a few months ago, and needed it right now. It was hard to find/remember.

[EwertonDutra]

Hi @thetutlage So, I got the problem and was able to solve it myself.

The S3 bucket "ACLs enabled" was required and I kept the ownership at bucket level. Is this correct thing? If yes, should we add that as part of the documentation? (Your thoughts on this)

Took quite some time to figure this out.

I needed to do this too.

I did it:
Block all public access
Off

In addition to what was shown in the picture

[erickximenes]

Has anyone found a solution where you don't need to disable Block all public access?

[erickximenes]

I found a solution where it is not necessary to activate the option to Block all public accesses. Just put visibility: 'private' in options.

Example:

await Drive.putStream(fileName, fileStream, {
            visibility: 'private',
            contentType: 'application/pdf'
})

[thanhtikeaung-solinxtechnology]

how about digitalocean s3 , how to find ACL disable , I encounter the error you mentioning

[sergenkabakci]

I had the same issue. The problem was resolved when I left the S3_ENDPOINT variable empty in the env file.

[corysus]

I had a similar problem, I got the error E_CANNOT_WRITE_FILE, but the original error was 413: Unknown error where 413 indicates "Request entity too large". In my case, I used the NPM proxy manager with MinIO, and the problem was with NGINX, which limited client_max_body_size to less than 2GB.

For uploading a file, I used the following code:

const readStream = fs.createReadStream(filePath);
await this.idrive.putStream("YOUR_FILE_NAME", readStream, {
  visibility: 'private',
});

I resolved my problem by changing S3_ENDPOINT to use the MinIO IP instead of using NPM with the domain name. However, the problem could also be resolved by increasing the body_size limit.

I wrote this in case someone has a similar problem.