Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S3 bucket slashes #1587

Merged
merged 4 commits into from
Jun 20, 2017
Merged

S3 bucket slashes #1587

merged 4 commits into from
Jun 20, 2017

Conversation

chrisradek
Copy link
Contributor

Updates s3.upload to use the same signature version as the client calling it.

Also allows forward slashes in S3 bucket names when an operation supports a Bucket and Key parameter. This is to maintain backwards compatibility with code written when signature version 2 was the default.

/cc @jeskew

@codecov-io
Copy link

codecov-io commented Jun 20, 2017

Codecov Report

Merging #1587 into master will increase coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1587      +/-   ##
==========================================
+ Coverage   95.56%   95.56%   +<.01%     
==========================================
  Files         183      183              
  Lines        6469     6478       +9     
  Branches     1325     1329       +4     
==========================================
+ Hits         6182     6191       +9     
  Misses        287      287
Impacted Files Coverage Δ
lib/services/s3.js 98.11% <100%> (+0.03%) ⬆️
lib/s3/managed_upload.js 93.89% <100%> (+0.04%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 17612f9...9a0c6eb. Read the comment docs.

req.params.Key = prefix + '/' + key;
req.params.Bucket = bucket.substr(0, slashIndex);
} else {
var msg = 'Bucket names cannot contain forward slashes. Bucket: ' + bucket;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't this run into the same BC issue as throwing whenever someone uses a bucket + prefix instead of just a bucket name? Do we currently allow users to provide a bucket name of foo/bar and no key when they want to access the key bar in the bucket foo?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a user did not supply a key at all, then they would receive a missing required param error. That said, it is possible to turn off param validation so maybe I should just make sure Key exists as a possible parameter.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds like a fairly extreme edge case. I'd be fine with leaving as is.

Scenario: Sigv2 Bucket with trailing slashes with Key
Given I use signatureVersion "v2"
When I put "" to the key "foo" with bucket suffix "/a/b/"
Then the object "a/b//foo" should exist
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't change it, but this behavior does not seem completely intuitive. Bash at least will treat one or more slashes in a sequence as a single directory separator. Just something to keep in mind for any future high-level interfaces, especially those that allow/encourage users to interact with S3 as if it were a filesystem.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, and I can't imagine this would be desired behavior. I wonder if other SDKs collapse multiple slashes into one.

@@ -205,11 +205,62 @@
done();
});
});
it('using sigv4 and the "Bucket" parameter contains forward slashes and "Key" exists', function(done) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is using sigv2

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy paste error, I'll fix that.

Copy link
Contributor

@jeskew jeskew left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lock
Copy link

lock bot commented Sep 28, 2019

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.

@lock lock bot locked as resolved and limited conversation to collaborators Sep 28, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants