Bucket Region

[ET-CS]

How can I connect to S3 bucket without knowing it's Region?
Is this API call implemented in this library? http://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGETlocation.html

or better, In boto (python) aws sdk I can connect to S3 bucket with 'universal' region without the need to find what the bucket region is.

[alimoeeny]

How does boto do it? is there a look up api or something?

[ET-CS]

In python I use the following code (without region):

from boto.s3.connection import S3Connection
s3conn = S3Connection(key, secret)
bucket = s3conn.get_bucket(bucketname)

I am not finding where I read that when region is missing, it's called 'universal' region, but I'm looking for it now.
This is the api : http://boto.readthedocs.org/en/latest/ref/s3.html#module-boto.s3.connection

[ET-CS]

Maybe this will help
http://stackoverflow.com/questions/23239877/how-does-boto-choose-aws-region-to-create-buckets

[ET-CS]

Maybe boto is checking aws GetBucketLocation api, or it's looping through all regions to find where the bucket is. I don't really know. but I know it's working. I can't find a way in goamz to connect to bucket when the region is unknown.

[ET-CS]

And thank you for your quick answer!

[alimoeeny]

I really don't know if there already is a way to to it with goamz or not, if you asked me before this, I would have said goamz does not care, you don't need to know the region, but I have not read that part of the code in ages, maybe people who actually done so should answer and I should shut my mouth :)

[ET-CS]

I am using the following code:.

conn := s3.New(auth, aws.USWest2) // Here is the part which region is needed
bucket := conn.Bucket("bucketName")
file, err := bucket.Get("somefile")

The s3.New() inforce me to supply region. maybe I am doing it wrong. I've tried to look in the source to find another way to create s3 object without region but found nothing.
https://github.com/crowdmob/goamz/blob/master/s3/s3.go#L91
https://godoc.org/launchpad.net/goamz/s3#New

thanks again.

[alimoeeny]

Why not provide it? you can get your region with aws.InstanceRegion()

[ET-CS]

Thank you.
I've tried. aws.InstanceRegion() always returning "unknown".
looking at the source, the url http://169.254.169.254/latest/meta-data/placement/availability-zone seems to be broken.
I've followed again the boto source and haven't found anything regards region in the process of the the functions I call. It's like region is not needed for boto (or I missed, and boto does finds the region by himself). I don't know really how boto do that.

[alimoeeny]

You mean on actual EC2 instances it returns "unknown" ?! really? what region are you trying it on?
If you curl that url on your machine what would you get back?

[ET-CS]

Thank you for your response. My region is USWest2.
I am using the following code:

    // Get Auth object
    auth, err := aws.GetAuth(key, secret, "", time.Time{})
    if err != nil { return err }

    // Get S3 Bucket Connection
    fmt.Print(aws.InstanceRegion())                    // aws.InstanceRegion() == always to 'unknown'
    conn := s3.New(auth, aws.USWest2)
    bucket := conn.Bucket(bucketName)

If I curl http://169.254.169.254/latest/meta-data/placement/availability-zone it returns
curl: (7) couldn't connect to host

[alimoeeny]

if you curl that url on your EC2 instance and you get the could not connect error, the only thing that comes to mind is you have some firewall (iptables?) that prevents that, what kind of machine is this? what ami are you using? to make sure I just tried it and I get:

$ curl http://169.254.169.254/latest/meta-data/placement/availability-zone
us-east-1e$

[ET-CS]

Sorry, I didn't explained well. I am not working on EC2 instance. I develop a software on a local machine - a program that fetchs data from S3 bucket for several key/secret combinations. Currently it's working alright with the above code, but as you noticed - I've explicitly set the region in the s3.New() function to aws.USWest2.
Now, as others will use this application too, and I don't know what region there are in, I am trying to find a way to find the region automatically or better, not supply any (as in boto).
I guess that aws.InstanceRegion() is working on EC2 instance only?

[alimoeeny]

got you, that is why you get unknown region, I don't know of any solution, but I am sure there is one,