Use the S3 bucket's region vs the client's

[celaus]

Hi, I am trying to fetch ACLs for all buckets that a user has access to which are spread across several regions. However, it looks like when doing this naively, the endpoint resolver uses the client's region in order to do the GetBucketAcl call and not the actual bucket's. I can't predict the bucket region and there are always going to be several regions.

I have tried different ways of changing the config to use .use_arn_region(true) to no avail, in each case the result is a PermanentRedirect "error".

status="service_error" service error: unhandled error: unhandled error: Error { code: "PermanentRedirect", message: "The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.", request_id: "K7QFCxxxxHT1DP1D", s3_extended_request_id: "nBF37xygPkLk1VbvMxxxxxZEoU4KyPTkYswdNYIxxmcb6ZiDP0oU5x8gYJ1VdFU/+SKE52XcGQ=" } (ServiceError(ServiceError { source: GetBucketAclError ....

I suspect that it could work if I were to recreate the configs for each bucket, however with boto3 this doesn't seem to be an issue? Any input is appreciated!

[ysaito1001]

Hi @celaus, thank you for reporting this! Could you share with us a self-contained example so we can run it on our end (possibly with tailoring) to reproduce what you are seeing? Also, could you let us know what version of the aws-sdk-s3 crate you're on?

Edit:
In the meantime, I believe I have reproduced what you reported. If I have a bucket in the us-east-1 region and run the following code

#[tokio::main]
async fn main() -> Result<(), Error> {
    let shared_config = aws_config::from_env().load().await; // default region is set to us-west-2
    let s3_client = aws_sdk_s3::Client::new(&shared_config);
    s3_client
        .get_bucket_acl()
        .bucket("name-of-bucket-in-us-east-1")
        .send()
        .await
        .unwrap();

    Ok(())
}

The execution fails with a PermanentRedirect error. Whereas when I execute the counterpart in boto3:

import boto3

s3 = boto3.client('s3')
result = s3.get_bucket_acl(Bucket='name-of-bucket-in-us-east-1')
print(result)

It prints the result correctly.

[celaus]

Yes, this is exactly what I am talking about 😄 I was using 0.24 of the SDK - a workaround is to fetch the region and recreate the config and client for each region, but fixing this would save me one request per bucket 😅 .

[rcoh]

The bucket redirect logic in some older SDKs created a ton of complexity (as well as some unexpected behavior in some cases). We've decided to omit that behavior in the Rust SDK (which is why it works in Boto) but not for the Rust SDK.

[celaus]

Ah ok, I saw no mention of that in the docs - so is the best way to recreate the config with the target region? And - as I am listing buckets owned by an account - fetching the locations for each bucket? Or are there other plans?

[github-actions[bot]]

Hello! Reopening this discussion to make it searchable.