Rclone Copy command - Access denied

[RahulVadisetty91]

What is your current rclone version (output from rclone version)?

v1.50

What problem are you are trying to solve?

I'm trying to upload a txt file from aws ec2 instance to s3 bucket
with the help of rclone copy filename remote:bucketname. I get 403 forbidden error access denied when I'm uploading a text file to s3 bucket. And i tried to add checksum to the above command --ignore-checksum it's giving me the same error.

Note
I am able to upload the txt file with aws commands aws s3 cp txtfile buckname --sse aws:kms

How do you think rclone should be changed to solve that?

Need aws s3 Proper documentation on how to upload file with encryption and without encryption.

[JohnKiller]

Hi! I'm experiencing the same problem.
Old rclone version 1.43.1 works OK. Upgraded to 1.50.2 with same configuration and I get

Command failed: rclone copy /srv/l_ac/dbs/ aruba:s3-backup/l_ac
2019/12/30 07:02:45 ERROR : 201912300700.7z: Failed to copy: s3 upload: 403 Forbidden: <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code>
<Message>Access Denied</Message>
<BucketName>s3-backup</BucketName>
</Error>

Server is v2 only.

EDIT: Listing works OK: rclone ls aruba:s3-backup/l_ac/

[gilles-treguer]

Hi, You can add --s3-v2-auth Flag ,
rclone copy --ignore-checksum --s3-v2-auth /srv/l_ac/dbs/ aruba:s3-backup/l_ac

[ncw]

@RahulVadisetty91

I am able to upload the txt file with aws commands aws s3 cp txtfile buckname --sse aws:kms

Have you tried setting the same option with rclone? Which would be --s3-server-side-encryption aws:kms - you can set this in the config file too.

[ncw]

@JohnKiller

Hi! I'm experiencing the same problem.
Old rclone version 1.43.1 works OK. Upgraded to 1.50.2 with same configuration and I get

I think you may be experiencing this issue #4297 - I've just posted a fix for that can you check it out?

[JohnKiller]

@gilles-treguer
Thanks a lot, that was it. Adding --s3-v2-auth fixes the problem

@ncw
Hi, thanks for your reply. It doesn't work, it has the same error. Is the need for --s3-v2-auth a breaking change? It was working fine without this flag on the old version

[ncw]

@JohnKiller

Hi, thanks for your reply. It doesn't work, it has the same error. Is the need for --s3-v2-auth a breaking change? It was working fine without this flag on the old version

What was your previous version? Which cloud storage are you accessing BTW?

[JohnKiller]

What was your previous version? Which cloud storage are you accessing BTW?

1.43.1

Cloud storage is the italian "Aruba SPA" which implements only the older version of the s3 protocol

[ncw]

What was your previous version? Which cloud storage are you accessing BTW?

1.43.1

That is quite an old version...

Cloud storage is the italian "Aruba SPA" which implements only the older version of the s3 protocol

Ah!

What do you have set as your "provider" in your config file?

Did you try the --s3-path-style=false option - you can set that in the config file too.

[JohnKiller]

[aruba]
type = s3
env_auth = false
access_key_id = username
secret_access_key = password
region = other-v2-signature
endpoint = https://r1-it.storage.cloud.it
location_constraint = r1-it
acl = private
server_side_encryption =
storage_class =.

Commands i've tried:

rclone 1.43.1 works ok

 rclone copy /srv/l_ac/dbs/ aruba:s3-backup/l_ac

rclone 1.52.0 returns 403 forbidden

rclone copy /srv/l_ac/dbs/ aruba:s3-backup/l_ac

rclone 1.52.0 works ok

rclone copy --s3-v2-auth /srv/l_ac/dbs/ aruba:s3-backup/l_ac

rclone 1.52.0 Error: unknown flag: --s3-path-style

rclone copy --s3-path-style=false /srv/l_ac/dbs/ aruba:s3-backup/l_ac

Is there a way to add --s3-v2-auth to the config file?

[ncw]

Sorry it is --s3-force-path-style=false and force_path_style=false for the config file.

rclone 1.52.0 works ok

rclone copy --s3-v2-auth /srv/l_ac/dbs/ aruba:s3-backup/l_ac

If that works OK then stick v2_auth = true in the config file and I think that is a solution.

[ivandeex]

@JohnKiller Does your problem persist?

[ivandeex]

@ncw maybe add a note in S3.md ?

[federicoballarini]

Hi @ivandeex @ncw
I'm getting a similar error with Aruba Cloud Object Storage. I created this configuration:

[aruba]
type = s3
env_auth = false
v2_auth = true
access_key_id = myuser
secret_access_key = mypass
region = other-v2-signature
endpoint = r1-it.storage.cloud.it
location_constraint = r1-it
acl = private

But when I try to upload a file:

[root@dev rclone]# rclone copy /root/kickstart-post.log aruba:qgbackup/test --s3-force-path-style=false
2020/12/26 02:30:46 ERROR : Attempt 1/3 failed with 1 errors and: Forbidden: Forbidden
        status code: 403, request id: , host id:
2020/12/26 02:30:46 ERROR : Attempt 2/3 failed with 1 errors and: Forbidden: Forbidden
        status code: 403, request id: , host id:
2020/12/26 02:30:46 ERROR : Attempt 3/3 failed with 1 errors and: Forbidden: Forbidden
        status code: 403, request id: , host id:
2020/12/26 02:30:46 Failed to copy: Forbidden: Forbidden
        status code: 403, request id: , host id:

Instead, ls works fine without --s3-force-path-style=false and doesn't work with this option.

Can you help me?

[root@dev rclone]# rpm -q rclone
rclone-1.47.0-1.el7.x86_64

I can normally upload with s3cmd

s3cmd put /root/kickstart-post.log s3://qgbackup

that has this configuration

[root@dev rclone]# cat /root/.s3cfg
# Setup endpoint
host_base = r1-it.storage.cloud.it
host_bucket = r1-it.storage.cloud.it
bucket_location = us-east-1
use_https = True

# Setup access keys
access_key =  myuser
secret_key = mypass

# Enable S3 v4 signature APIs
signature_v2 = True

[federicoballarini]

This works fine: https://forum.rclone.org/t/aruba-cloud-object-storage/19704
I had to update rclone version to the latest.

[ncw]

I think this is a bit of a confused issue.

Configuring the S3 backend is hard unfortunately as there are so many option.

We try to make the docs as helpful as possible, so if anyone has some suggestions (or better a pull request) I can put them in.

In the mean time if the docs aren't sufficient then please ask for help on the rclone forum.

I'm going to close this now as I don't think there are any active issues here any more.