A client error (PermanentRedirect) occurred when calling the ListObjects operation #564

Closed
srivest opened this Issue Dec 18, 2013 · 11 comments

Comments

Projects
None yet
10 participants
@srivest

srivest commented Dec 18, 2013

ubuntu@ip-10-166-225-37:~$ aws --version
aws-cli/1.2.8 Python/2.7.5+ Linux/3.11.0-12-generic

ubuntu@ip-10-166-225-37:~$ aws --region us-west-1 s3 sync s3://UtilityStorage/builds/dev /tmp/build
A client error (PermanentRedirect) occurred when calling the ListObjects operation: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.
Completed 1 part(s) with ... file(s) remaining

@jessereynolds

This comment has been minimized.

Show comment
Hide comment
@jessereynolds

jessereynolds Jan 31, 2014

I get this when accessing a different region from the one I used when I created the s3 bucket.

I get this when accessing a different region from the one I used when I created the s3 bucket.

@stefanw

This comment has been minimized.

Show comment
Hide comment
@stefanw

stefanw Feb 17, 2014

A nicer error message would indeed be appreciated.

stefanw commented Feb 17, 2014

A nicer error message would indeed be appreciated.

@vors

This comment has been minimized.

Show comment
Hide comment
@vors

vors Apr 18, 2014

Same problem. Thank you, @jessereynolds, for clarification!

vors commented Apr 18, 2014

Same problem. Thank you, @jessereynolds, for clarification!

@eterps

This comment has been minimized.

Show comment
Hide comment
@eterps

eterps Apr 27, 2014

How does this work, do I understand correctly that an S3 URI is not unique but dependend on the region parameter?

eterps commented Apr 27, 2014

How does this work, do I understand correctly that an S3 URI is not unique but dependend on the region parameter?

@tylercollier

This comment has been minimized.

Show comment
Hide comment
@tylercollier

tylercollier Aug 20, 2014

I didn't realize my data had a region on it. When I'm using the web console, where it normally shows my region in the top right, it instead shows "Global", and when I click the dropdown it says "S3 does not require region selection." So I didn't realize my data DOES have regions. I found it by viewing the file/dir properties S3. Specifying the region as @jessereynolds suggested fixes it right up. Thanks for that clarification.

I agree with @srivest that a nicer error message would help. When I used --debug, it actually had <Endpoint>[the bucket I was using].s3.amazonaws.com</Endpoint>, which only served to confuse me further, because what I should have been using was correct: s3://[the bucket I was using]/[my path].

I didn't realize my data had a region on it. When I'm using the web console, where it normally shows my region in the top right, it instead shows "Global", and when I click the dropdown it says "S3 does not require region selection." So I didn't realize my data DOES have regions. I found it by viewing the file/dir properties S3. Specifying the region as @jessereynolds suggested fixes it right up. Thanks for that clarification.

I agree with @srivest that a nicer error message would help. When I used --debug, it actually had <Endpoint>[the bucket I was using].s3.amazonaws.com</Endpoint>, which only served to confuse me further, because what I should have been using was correct: s3://[the bucket I was using]/[my path].

@kyleknap kyleknap added s3 labels Aug 20, 2014

@bacoboy

This comment has been minimized.

Show comment
Hide comment
@bacoboy

bacoboy Oct 31, 2014

Surprised this is still open...

I stumbled on this using aws-cli with the us-east-1 region. s3 commands to buckets in this region fail unless you explicitly specify the region. If you run: aws s3api get-bucket-region --bucket NAME, it'll return null for us-east-1 but not for others. For instance my us-east-1 bucket says:

{
    "LocationConstraint": null
}

but my Singapore bucket says:

{
    "LocationConstraint": "ap-southeast-1"
}

This seems to be the cause of the error as aws-cli seems to depend on this location resolving to something.

Clearly, I can just specify the --region=us-east-1 option to my commands as a work around, but if the s3 namespace is supposed to be global, my having to tell AWS where it is seems strange -- don't they know?

bacoboy commented Oct 31, 2014

Surprised this is still open...

I stumbled on this using aws-cli with the us-east-1 region. s3 commands to buckets in this region fail unless you explicitly specify the region. If you run: aws s3api get-bucket-region --bucket NAME, it'll return null for us-east-1 but not for others. For instance my us-east-1 bucket says:

{
    "LocationConstraint": null
}

but my Singapore bucket says:

{
    "LocationConstraint": "ap-southeast-1"
}

This seems to be the cause of the error as aws-cli seems to depend on this location resolving to something.

Clearly, I can just specify the --region=us-east-1 option to my commands as a work around, but if the s3 namespace is supposed to be global, my having to tell AWS where it is seems strange -- don't they know?

@jamesls

This comment has been minimized.

Show comment
Hide comment
@jamesls

jamesls Jan 16, 2015

Member

This has been fixed, but it looks like the commit was not linked with this issue. You now get a better error message:

$ aws s3 ls s3://my.bucket/

A client error (PermanentRedirect) occurred when calling the ListObjects operation: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint: my.bucket.s3-ap-southeast-2.amazonaws.com
You can fix this issue by explicitly providing the correct region location using the --region argument, the AWS_DEFAULT_REGION environment variable, or the region variable in the AWS CLI configuration file.  You can get the bucket's location by running "aws s3api get-bucket-location --bucket BUCKET".
Member

jamesls commented Jan 16, 2015

This has been fixed, but it looks like the commit was not linked with this issue. You now get a better error message:

$ aws s3 ls s3://my.bucket/

A client error (PermanentRedirect) occurred when calling the ListObjects operation: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint: my.bucket.s3-ap-southeast-2.amazonaws.com
You can fix this issue by explicitly providing the correct region location using the --region argument, the AWS_DEFAULT_REGION environment variable, or the region variable in the AWS CLI configuration file.  You can get the bucket's location by running "aws s3api get-bucket-location --bucket BUCKET".

@jamesls jamesls closed this Jan 16, 2015

@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost Mar 16, 2015

I also get the location constraint : null.
This is a still a bug, I just went with us-east-1 and it worked, but if it is a us-standard bucket it seems like there should be an option for --region us-standard
Or something similar, no?

ghost commented Mar 16, 2015

I also get the location constraint : null.
This is a still a bug, I just went with us-east-1 and it worked, but if it is a us-standard bucket it seems like there should be an option for --region us-standard
Or something similar, no?

@kyleknap

This comment has been minimized.

Show comment
Hide comment
@kyleknap

kyleknap Mar 16, 2015

Member

@xurume

For buckets located in US Standard, the location constraint will be null. For S3, here is the list of region names with their corresponding regions: http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region. Notice that the location constraint is none for US Standard.

The CLI uses the values in the region column for the --region parameter. So for S3's US Standard, you need to use us-east-1 as the region.

Member

kyleknap commented Mar 16, 2015

@xurume

For buckets located in US Standard, the location constraint will be null. For S3, here is the list of region names with their corresponding regions: http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region. Notice that the location constraint is none for US Standard.

The CLI uses the values in the region column for the --region parameter. So for S3's US Standard, you need to use us-east-1 as the region.

@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost Mar 16, 2015

Undesrtood, thank you!

ghost commented Mar 16, 2015

Undesrtood, thank you!

@zhangruichang

This comment has been minimized.

Show comment
Hide comment
@zhangruichang

zhangruichang Apr 12, 2016

get error when
aws s3 ls s3://path

how to resolve it ?

get error when
aws s3 ls s3://path

how to resolve it ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment