S3 actions right after bucket creation: handle 307 redirects

[matthewkmayer]

As seen in #28: if you create a bucket anywhere but us-east-1 and immediately try to upload objects to it, you'll get a 307 temporary redirect. Hyper appears to follow that but the message body does not get re-sent, hence the errors I saw where AWS calculated a body payload of empty string.

This ticket is for reproducing this behavior locally and verifying it's a bug in Hyper, then giving them an easy repro in a ticket.

[matthewkmayer]

Confirmed it's a bug in Hyper's client: hyperium/hyper#641

[matthewkmayer]

While we may be able to get around some of these issues by implementing S3's expect 100-continue behavior, it may not help with, say, changing bucket settings.

If we don't get a fix from Hyper we'll have to manage the retry ourselves. Which means explicitly not following the redirects and handling that in signature.rs. That'd be a good candidate for refactoring signature into the signing part and the request part.

[matthewkmayer]

The "retry ourselves" mechanism is the way to go. Using boto as an example, the python httplib they use does not follow 307 redirects. I tested this with the local Go server mentioned in the Hyper issue above and this Python 2 code:

import httplib, urllib
params = urllib.urlencode({'spam': 1, 'eggs': 2, 'bacon': 0})
conn = httplib.HTTPConnection("localhost:8080")
conn.request("GET", "/", params)
response = conn.getresponse()
print response.status, response.reason
data = response.read()
conn.close()

This means the boto library is handling it instead of its client.

[matthewkmayer]

Copy/paste from Slack: response body will look like this:

<Error><Code>TemporaryRedirect</Code><Message>Please re-send this request to the specified temporary endpoint. Continue to use the original request endpoint for future requests.</Message><Bucket>rusoto1441045966</Bucket><Endpoint>rusoto1441045966.s3-us-west-1.amazonaws.com</Endpoint><RequestId>08DB1CC6602DEFA2</RequestId><HostId>lZYOlF9QwVQS2ach8RWVWxvFmfbX+HvNT0K/DgDN2d+qMQO7KFcPIn3YnqY2knbH96FMrN3JX7o=</HostId></Error>

[matthewkmayer]

This code got removed - this should be retested, as I think this issue is happening again. Reopening to test again.

[matthewkmayer]

We're still good: I bet the DNS-compatible bucket name changes covered this by sending to a different endpoint. See #422 .