ARROW-10066: [C++] Make sure default AWS region selection algorithm is used #8249
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
I tested with PR with versions 1.7.164 and 1.8.52 of the AWS SDK. |
pitrou
commented
Sep 23, 2020
| @@ -1321,6 +1321,8 @@ def test_s3_real_aws(): | |||
| from pyarrow.fs import S3FileSystem | |||
| fs = S3FileSystem(anonymous=True) | |||
| assert fs.region == 'us-east-1' # default region | |||
|
|
|||
| fs = S3FileSystem(anonymous=True, region='us-east-2') | |||
| entries = fs.get_file_info(FileSelector('ursa-labs-taxi-data')) | |||
There was a problem hiding this comment.
@nealrichardson With the SDK 1.8.52, this doesn't work anymore with the default region.
There was a problem hiding this comment.
But it does with SDK < 1.8? Interesting.
nealrichardson
approved these changes
Sep 23, 2020
…s used When the user doesn't set an explicit region, the default algorithm from the AWS C++ SDK should be used. With SDK versions >= 1.8, this involves reading environment variables and potentially querying the local EC2 metadata server.
pitrou
force-pushed
the
ARROW-10066-aws-region-selection
branch
from
cb54ec1
to
2bacbd5
Compare
|
Successful Travis-CI build here: https://travis-ci.org/github/pitrou/arrow/builds/729705592 |
emkornfield
pushed a commit
to emkornfield/arrow
that referenced
this pull request
Oct 16, 2020
…s used When the user doesn't set an explicit region, the default algorithm from the AWS C++ SDK should be used. With SDK versions >= 1.8, this involves reading environment variables and potentially querying the local EC2 metadata server. Closes apache#8249 from pitrou/ARROW-10066-aws-region-selection Authored-by: Antoine Pitrou <antoine@python.org> Signed-off-by: Neal Richardson <neal.p.richardson@gmail.com>
GeorgeAp
pushed a commit
to sirensolutions/arrow
that referenced
this pull request
Jun 7, 2021
…s used When the user doesn't set an explicit region, the default algorithm from the AWS C++ SDK should be used. With SDK versions >= 1.8, this involves reading environment variables and potentially querying the local EC2 metadata server. Closes apache#8249 from pitrou/ARROW-10066-aws-region-selection Authored-by: Antoine Pitrou <antoine@python.org> Signed-off-by: Neal Richardson <neal.p.richardson@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When the user doesn't set an explicit region, the default algorithm from the AWS C++ SDK should be used.
With SDK versions >= 1.8, this involves reading environment variables and potentially querying the local EC2 metadata server.