Skip to content

Amazon S3 Multi-Region Access Points support #821

Description

@egummett-aws

From https://docs.aws.amazon.com/AmazonS3/latest/userguide/MultiRegionAccessPoints.html:

Amazon S3 Multi-Region Access Points provides a global endpoint that applications can use to fulfill requests from S3 buckets that are located in multiple AWS Regions. You can use Multi-Region Access Points to build multi-Region applications with the same architecture that's used in a single Region, and then run those applications anywhere in the world. Instead of sending requests over the congested public internet, Multi-Region Access Points provides built-in network resilience with acceleration of internet-based requests to Amazon S3. Application requests made to a Multi-Region Access Point global endpoint uses AWS Global Accelerator to automatically route over the AWS global network to the closest proximity S3 bucket with an active routing status.
https://docs.aws.amazon.com/AmazonS3/latest/userguide/MultiRegionAccessPoints.html

An increasingly common pattern we're seeing at AWS is customers using Multi-Region Access Points to simplify networking when accessing S3 across AWS Regions - often with just a single bucket behind the global access point, and clients across various other regions. More on this in the article Cost effective methods for accessing S3 buckets cross-region.

I have not been able to make s5cmd work with Multi-Region Access Points, and expect this is due to either the Signature Version 4 (SigV4A) requirement, the global nature of the endpoint (no need to specify region), or both. I tried specifying --endpoint-url https://<alias>.mrap.accesspoint.s3-global.amazonaws.com/, and --source-region s3-global.

I don't think s5cmd can use ARN format (arn:aws:s3::account_id:accesspoint/MultiRegionAccessPoint_alias//key), but it should be okay with bucket-style alias format (MultiRegionAccessPoint_alias.accesspoint.s3-global.amazonaws.com for example, s3://mfzwi23gnjvgw.mrap.accesspoint.s3-global.amazonaws.com/key).

For testing S3 Multi-Region Access points, https://catalog.workshops.aws/s3-data-protection/en-US/40-module-4 may be helpful.

Happy to help with testing.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions