How is region determined if not explicitly passed?

[math4tots]

How does the aws s3 ls s3://... command determine what region to use if the user doesn't specify one?

---

We're working with a bucket where we don't have top level permissions, but only permissions for reading files with a given prefix.

Our code using the Go SDK used to use GetBucketLocation to get the correct region before trying to list files. But now, we don't have top level bucket read permissions so that's not an option anymore.

We were hoping we could learn from what aws-cli does to do the "right thing".

[stealthycoin]

If you send an unauthed HEAD request any s3 region with the header:
Host: bucket-name.s3.amazonaws.com (or any other S3 endpoint) it will return a 307 with both of the following:
x-amz-bucket-region header which will contain the real region.
Location with the url.

To demonstrate:
I created a bucket named stealthycoin.test.bucket in Ireland (eu-west-1).

I used the following curl command:
curl -v -X HEAD --header "Host: stealthycoin.test.bucket.s3.amazonaws.com" s3.us-east-2.amazonaws.com which sends a head bucket request to the us-east-2 region.

Here are the response headers I got:

< HTTP/1.1 307 Temporary Redirect
< x-amz-bucket-region: eu-west-1
< x-amz-request-id: ...
< x-amz-id-2: ...
< Location: http://stealthycoin.test.bucket.s3-eu-west-1.amazonaws.com/
< Content-Type: application/xml
< Transfer-Encoding: chunked
< Date: Thu, 09 Feb 2017 18:09:20 GMT
< Server: AmazonS3

You can see the interesting bits are:
< x-amz-bucket-region: eu-west-1
< Location: http://stealthycoin.test.bucket.s3-eu-west-1.amazonaws.com/

[math4tots]

@stealthycoin Thanks! it seems to work very similar to my situation except that for my stuff I don't get a 307, but a 301, and I'm not getting Location.

Kyumins-MacBook-Pro:~ math4tots$ curl -v -X HEAD --header "Host: ascend-math4tots-demo" s3.eu-west-2.amazonaws.com
* Rebuilt URL to: s3.eu-west-2.amazonaws.com/
*   Trying 52.95.148.4...
* Connected to s3.eu-west-2.amazonaws.com (52.95.148.4) port 80 (#0)
> HEAD / HTTP/1.1
> Host: ascend-math4tots-demo
> User-Agent: curl/7.43.0
> Accept: */*
> 
< HTTP/1.1 301 Moved Permanently
< x-amz-bucket-region: us-east-1
< x-amz-request-id: A01F94C11CE7082B
< x-amz-id-2: QLiZNh3wTvx/3wzT8iU6V7abCIyr4Ugls+Lq1pgLmdHhWLavyB9VwAUb3SH+zbkb8QQxGI74w/0=
< Content-Type: application/xml
< Transfer-Encoding: chunked
< Date: Thu, 09 Feb 2017 20:20:26 GMT
< Server: AmazonS3

Is x-amz-bucket-region: us-east-1 still the correct region? Why is mine different?

[stealthycoin]

Make sure the Host header you send up looks like this:
"Host: stealthycoin.test.bucket.s3.amazonaws.com"
You are just sending up the bucket name part.

To be clear its Host: {bucket-name}.s3.amazonaws.com

[math4tots]

I still get a 301

Kyumins-MacBook-Pro:~ math4tots$ curl -v -X HEAD --header "Host: ascend-math4tots-demo.s3.amazonaws.com" s3.us-east-2.amazonaws.com
* Rebuilt URL to: s3.us-east-2.amazonaws.com/
*   Trying 52.92.78.9...
* Connected to s3.us-east-2.amazonaws.com (52.92.78.9) port 80 (#0)
> HEAD / HTTP/1.1
> Host: ascend-math4tots-demo.s3.amazonaws.com
> User-Agent: curl/7.43.0
> Accept: */*
> 
< HTTP/1.1 301 Moved Permanently
< x-amz-bucket-region: us-east-1
< x-amz-request-id: F84571DAF6838453
< x-amz-id-2: GNZGa7JN+o8PwkvgBWTT/mO6BnGu/n0DKfv5I73THYtxG+40aAiZPEVSjyZjbgyRf6kY4vyZGtY=
< Content-Type: application/xml
< Transfer-Encoding: chunked
< Date: Thu, 09 Feb 2017 21:01:19 GMT
< Server: AmazonS3
< 

[stealthycoin]

So if the bucket is hosted in US Standard (us-east-1) then you will get the 301 and the header will say us-east-1. So you can always rely on the x-amz-bucket-region header.

[math4tots]

Ah cool, thank you, this resolves my issue