Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Copy from S3 to S3 fails #230

Closed
gerhardj-b opened this issue May 26, 2023 · 4 comments · Fixed by #231
Closed

Copy from S3 to S3 fails #230

gerhardj-b opened this issue May 26, 2023 · 4 comments · Fixed by #231

Comments

@gerhardj-b
Copy link

When specifying a copy source which is a location in an S3 Bucket, the Operation cuts of the first character from the Bucket name.

Example with the official cli:

root@docker-desktop:/$ aws s3 --endpoint-url http://$MY_S3_NINJA cp s3://testbucket/testfile s3://testbucket/newtestfile
copy failed: s3://testbucket/testfile to s3://testbucket/newtestfile An error occurred (NoSuchBucket) when calling the CopyObject operation: Source bucket 'estbucket' does not exist

I suspect the substring here to be the culprit.

Amazon API Documentation here
jakobvogel added a commit that referenced this issue Jun 16, 2023
@jakobvogel
Adds test case for copy operations within a bucket 👯‍♂️
a0e2c34 
#230
jakobvogel added a commit that referenced this issue Jun 16, 2023
@jakobvogel
Adds test case for copy operations across buckets 🪣
545ff35 
#230
jakobvogel added a commit that referenced this issue Jun 16, 2023
@jakobvogel
Adds missing documentation of a parameter ✍️
c986f17 
#230
jakobvogel added a commit that referenced this issue Jun 16, 2023
@jakobvogel
Adds extra protection against empty strings 🛡️
add5e7a 
#230
jakobvogel added a commit that referenced this issue Jun 16, 2023
@jakobvogel
Supports the case if the copy source does not start with a leading pa…
b6e8529 
…th delimiter 🚑

Unlike the Java library, the `aws` CLI does not send a leading path delimiter, leading to an error further down the line.

This was reported by @gerhardj-b.

#230
jakobvogel added a commit that referenced this issue Jun 16, 2023
@jakobvogel
Revises code 💅
d33afd0 
Uses better variable names, improves code structure.

#230
@jakobvogel jakobvogel mentioned this issue Jun 16, 2023
Merged
@jakobvogel
Copy link
Member

Thanks for reporting this, @gerhardj-b – Apparently, the aws CLI sends the copy header without leading path delimiter while the Java library does. (The new test cases are thus a bit pointless.)

@gerhardj-b
Copy link
Author

Awesome, thanks. I read the API description also as not expecting a leading delimiter. Anyways, thanks for the extra check and solution for both cases.

@jakobvogel
Copy link
Member

For internal reference: The original error is reproduced via with access credentials taken from http://localhost:9444/ui

$ aws configure --profile s3ninja
$ export AWS_PROFILE=s3ninja
$ aws s3 --endpoint-url http://localhost:9444 cp s3://bucket1/file1 s3://bucket2/file2.pdf

jakobvogel added a commit that referenced this issue Jun 16, 2023
@jakobvogel
Fixes start index 🚒
3270af1 
#230
jakobvogel added a commit that referenced this issue Jun 19, 2023
@jakobvogel
Fixes support for special characters in keys 🚑
ff4a276 
#230
@jakobvogel
Copy link
Member

Released with version 8.3.2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Bugfix: Copying Objects with aws CLI scireum/s3ninja
2 participants
@jakobvogel @gerhardj-b