Unable to upload S3 file with Multi-region access point

[JulioMakita]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

TF CLI - 1.1.3
AWS Provider - 3.71.0

Affected Resource(s)

aws_s3control_multi_region_access_point
aws_s3_bucket_object

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

resource "aws_s3_bucket_object" "examplebucket_object" {
  key    = "test.txt"
  bucket = "arn:aws:s3::111111111111:accesspoint/my8ka9d9aaaa7.mrap"
  source = "test.txt"
}

Expected Behavior

Terraform should upload file when a S3 multi-region access point is passed.

Actual Behavior

S3 multi-region access point cannot be used. Terraform is complaining the region is not set.
However, multi-region access point does not have a region.

aws_s3_bucket_object.examplebucket_object: Creating... Error: Error uploading object to S3 bucket (arn:aws:s3::111111111111:accesspoint/my8ka9d9aaaa7.mrap): InvalidARNError: invalid ARN caused by: invalid Amazon s3 ARN, region not set, arn:aws:s3::111111111111:accesspoint/my8ka9d9aaaa7.mrap

I tested using the following command in aws cli. it worked well.

aws s3api put-object --bucket arn:aws:s3::111111111111:accesspoint/my8ka9d9aaaa7.mrap --key test.txt --body test.txt

Steps to Reproduce

Create a S3 multi-region access point. I created using aws_s3control_multi_region_access_point resource.
It will generate a new global Amazon Resource Name (ARN).
Upload a file into S3 Bucket using aws_s3_bucket_object resource.
add multi-region access point ARN as bucket name.

References

https://docs.aws.amazon.com/AmazonS3/latest/userguide/MultiRegionAccessPointRequests.html

[justinretzolk]

Hey @JulioMakita 👋 Thank you for taking the time to file this issue. In reviewing, I believe I've come up with a solution that may work for you. A lot of this came from this helpful AWS document.

According to the document:

When you make a request to a Multi-Region Access Point endpoint, Amazon S3 automatically routes the request to the bucket that responds to the request with the lowest latency. It does not consider the contents of the request when making this decision. If you make a request to GET an object, your request might be routed to a bucket that does not have a copy of this object.

What this means is that you'll likely want to set up bucket replication (as discussed in the document) to ensure that all buckets have the same objects. This can be achieved by setting the replication_configuration on the S3 bucket resource definition, or (perhaps more resilient) using the separate aws_s3_bucket_replication_configuration resource.

Once the bucket objects are being replicated, you could then use the aws_s3_bucket_object resource that you're currently using, with the bucket ARN being set to one of the buckets in question. This would satisfy the resource's requirement of the bucket argument being a bucket ARN, while ensuring that the object was replicated across the buckets as necessary.

[JulioMakita]

Hi @justinretzolk Thank you for your reply.
Sorry but, I think my issue was not clear.
I just would like to upload a file to S3 bucket using multi-region access point instead of using a specific bucket region.
I already have a bidirectional replication configured in my buckets.
For sure I can pass one of the bucket ARN when using aws_s3_bucket_object. It will work well as it will have the region name in ARN. but, what happens if the region of this bucket is down? S3 is smart enough to redirect to other regions available?
I think that is the reason to have a multi-region access point. if a region goes down it will redirect to the other available bucket automaticaly.

[justinretzolk]

Hey @JulioMakita 👋 I took another look and realized I'd overlooked an important thing here. Looking at the aws_s3_bucket_object resource documentation I noticed this note:

bucket - (Required) Name of the bucket to put the file in. Alternatively, an S3 access point ARN can be specified.

With that in mind (and the fact that you're able to do this via the AWS CLI, and the documentation online seems to indicate that this is possible), it seems like supplying the multi-region access point ARN should work, but the errors that you're seeing are coming from AWS, so it seems like this may be an upstream bug. I'm going to tag it as such so that we can take a deeper look into this as time permits.

[nickkomen]

This problem still exists, even on the latest provider (4.66.1)