Comment on #3015 - Bug not fixed

[mangelozzi]

Note was left in #3015, saying to add a coment open a new issue.

So the solution it says is to add the endpoint URL. This is a workaround and a hack from the root issue that even though boto3 is told the correct region, it does not use it to create the URL.

If the endpoint url is REQUIRED to work, it should not be an optional parameter, that is buggy code.

Its crazy for users to know in what year amazon deployed which services, that it determines whether you need to pass in some unknown parameter that is missing from all the tutorials on the web.

The bug quite likely applies to generate_presigned_post too.

Also for speed, the generated url ideally should not require redirecting, it should use the region_name data to generate the final url.

[tim-finnigan]

Hi @mangelozzi, apologies for closing that issue prematurely. I agree this is something that needs to be addressed.

I escalated this issue with the S3 team earlier and we are waiting to hear back from them. I also brought this up with our team today and we are debating the best way this could be fixed on our end.

(Also I agree that generate_presigned_post is affected too as that was brought up in #2728.)

For more context on how some regions are affected there is this note in the S3 documentation: https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingRouting.html#Redirects

For all Regions that launched after March 20, 2019, if a request arrives at the wrong Amazon S3 location, Amazon S3 returns an HTTP 400 Bad Request error.

So we have to figure out a way to account for that with newer regions. I’m going to close this issue and reopen your original issue to continue on that thread. When we hear back from the S3 team I will post an update to that issue. Thanks for your patience and understanding while we sort this out.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[mangelozzi]

Hi @mangelozzi, apologies for closing that issue prematurely. I agree this is something that needs to be addressed.

I escalated this issue with the S3 team earlier and we are waiting to hear back from them. I also brought this up with our team today and we are debating the best way this could be fixed on our end.

(Also I agree that generate_presigned_post is affected too as that was brought up in #2728.)

For more context on how some regions are affected there is this note in the S3 documentation: https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingRouting.html#Redirects

For all Regions that launched after March 20, 2019, if a request arrives at the wrong Amazon S3 location, Amazon S3 returns an HTTP 400 Bad Request error.

So we have to figure out a way to account for that with newer regions. I’m going to close this issue and reopen your original issue to continue on that thread. When we hear back from the S3 team I will post an update to that issue. Thanks for your patience and understanding while we sort this out.

Thanks @tim-finnigan for following it up. It's not affecting me anyone since I am using the workaround you mentioned (thank you), but I am sure pretty much everyone else using these other regions with presigned URLs will enoucnter this issue, will be great to sort it out for them, its tricky enough being a coder =)!