Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

S3(Alibaba): bucket ACL changed after rclone mkdir #7889

Closed
Thearas opened this issue Jun 3, 2024 · 4 comments · Fixed by #7907
Closed

S3(Alibaba): bucket ACL changed after rclone mkdir #7889

Thearas opened this issue Jun 3, 2024 · 4 comments · Fixed by #7907

Comments

@Thearas
Copy link
Contributor

Thearas commented Jun 3, 2024 •
edited
Loading

The associated forum post URL from https://forum.rclone.org

No.

What is the problem you are having with rclone?

rclone mkdir will CreateBucket (aka PutBucket in Alibaba (OSS)), and will change the ACL of an existing bucket.

To fix it, we need to avoid calling CreateBucket when bucket exists on Alibaba (OSS), maybe use GetBucketInfo to check when the provider = Alibaba?

P.S. Standard AWS S3 SDK does not have GetBucketInfo API, so we need to introduce aliyun-oss-go-sdk.

What is your rclone version (output from rclone version)

rclone v1.66.0

  • os/version: darwin 14.4.1 (64 bit)
  • os/kernel: 23.4.0 (arm64)
  • os/type: darwin
  • os/arch: arm64 (ARMv8 compatible)
  • go/version: go1.22.1
  • go/linking: dynamic
  • go/tags: none

Which OS you are using and how many bits (e.g. Windows 7, 64 bit)

darwin, 64bit

Which cloud storage system are you using? (e.g. Google Drive)

S3 (provider Alibaba)

The command you were trying to run (e.g. rclone copy /tmp remote:tmp)

rclone mkdir xxx

A log from the command with the -vv flag (e.g. output from rclone -vv copy /tmp remote:tmp)

How to use GitHub

  • Please use the 👍 reaction to show that you are affected by the same issue.
  • Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
  • Subscribe to receive notifications on status change and new comments.
@Thearas Thearas changed the title S3(Alibaba): bucket ACl changed after rclone mkdir S3(Alibaba): bucket ACL changed after rclone mkdir Jun 3, 2024
@ncw
Copy link
Member

ncw commented Jun 6, 2024

Try adding no_check_bucket = true to your config - I think that should fix it.

If it does, then we should recommend that in the docs.

@Thearas
Copy link
Contributor Author

Thearas commented Jun 11, 2024

@ncw Thanks, it works for me.
Two things:

  1. Recommend no_check_bucket = true in docs
  2. Since the default behavior shouldn't be a bug, maybe we check if the directory exists before makeBucket in Mkdir()?

@ncw
Copy link
Member

ncw commented Jun 11, 2024

What should happen is PutBucket returns error AlreadyOwnedByYou rather than updating the bucket. That is what AWS does, so I'd class this as a bug in the Alibaba implementation.

We should at minimum Recommend no_check_bucket = true in the docs for Alibaba - do you want to send a PR for that?

@Thearas
Copy link
Contributor Author

Thearas commented Jun 12, 2024

We should at minimum Recommend no_check_bucket = true in the docs for Alibaba - do you want to send a PR for that?

OK.
I might submit another PR later to specifically implement a Fs for Alibaba.

Thearas added a commit to Thearas/rclone that referenced this issue Jun 13, 2024
@Thearas
docs: recommend no_check_bucket = true for Alibaba - fixes rclone#7889
b91d7d8 


Change-Id: Ib6246e416ce67dddc3cb69350de69129a8826ce3
@Thearas Thearas mentioned this issue Jun 13, 2024
Merged
5 tasks
@ncw ncw closed this as completed in #7907 Jun 15, 2024
ncw pushed a commit that referenced this issue Jun 15, 2024
@Thearas @ncw
docs: recommend no_check_bucket = true for Alibaba - fixes #7889
1c9da8c 
Change-Id: Ib6246e416ce67dddc3cb69350de69129a8826ce3
miku added a commit to internetarchive/rclone that referenced this issue Jun 18, 2024
@miku
Merge branch 'master' into ia-wt-1168
6c38c7c 
* master: (68 commits)
  build(deps): bump docker/build-push-action from 5 to 6
  s3: fix incorrect region for Magalu provider
  docs: recommend `no_check_bucket = true` for Alibaba - fixes rclone#7889
  docs: tidy .gitignore for docs
  docs: fix hugo warning: found no layout file for "html" for kind "term"
  docs: remove slug and url from command pages since they are no longer needed
  docs: fix hugo warning: found no layout file for "html" for kind "section"
  serve dlna: fix panic: invalid argument to Int63n
  Start v1.68.0-DEV development
  Version v1.67.0
  s3: fix 405 error on HEAD for delete marker with versionId
  gitannex: make tests run more quietly - use go test -v for more info
  jottacloud: set metadata on server side copy and move - fixes rclone#7900
  qingstor: disable integration tests as test account suspended
  operations: add operations.ReadFile to read the contents of a file into memory
  fs: make ConfigFs take an fs.Info which makes it more useful
  touch: fix using -R on certain backends
  serve s3: fix XML of error message
  fs/logger: make the tests deterministic
  zoho: sleep for 60 seconds if rate limit error received
  ...
gotenksIN added a commit to gotenksIN/rclone that referenced this issue Jul 3, 2024
@gotenksIN
Merge github.com:rclone/rclone
d956bf2 
* github.com:rclone/rclone: (135 commits)
  swift: add workarounds for bad listings in Ceph RGW
  sftp: fix docs on connections not to refer to concurrency
  docs: remove warp as silver sponsor
  onedrive: fix nil pointer error when uploading small files
  vfs: fix fatal error: sync: unlock of unlocked mutex in panics
  Add Filipe Herculano to contributors
  Add Thearas to contributors
  pikpak: implement custom hash to replace wrong sha1
  pikpak: improves data consistency by ensuring async tasks complete
  build(deps): bump docker/build-push-action from 5 to 6
  s3: fix incorrect region for Magalu provider
  docs: recommend `no_check_bucket = true` for Alibaba - fixes rclone#7889
  docs: tidy .gitignore for docs
  docs: fix hugo warning: found no layout file for "html" for kind "term"
  docs: remove slug and url from command pages since they are no longer needed
  docs: fix hugo warning: found no layout file for "html" for kind "section"
  serve dlna: fix panic: invalid argument to Int63n
  Start v1.68.0-DEV development
  Version v1.67.0
  s3: fix 405 error on HEAD for delete marker with versionId
  ...
Fornax96 pushed a commit to Fornaxian/rclone that referenced this issue Jul 30, 2024
@Thearas @Fornax96
docs: recommend no_check_bucket = true for Alibaba - fixes rclone#7889
7b66732 


Change-Id: Ib6246e416ce67dddc3cb69350de69129a8826ce3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

docs(s3): recommend no_check_bucket = true for Alibaba Thearas/rclone
2 participants
@ncw @Thearas