--fsx-create fails with 301 MovedPermanently when S3 bucket is in a different region than the instance

[scttfrdmn]

Summary

spawn launch --fsx-create --fsx-s3-bucket <bucket> fails immediately with a 301 redirect error when the S3 bucket exists in a different region than the instance being launched.

Error

Error: failed to create FSx filesystem: failed to create S3 bucket: failed to check if bucket exists: 
operation error S3: HeadBucket, https response error StatusCode: 301, RequestID: 0D2K6Y3E468HH7Q8, 
api error MovedPermanently: Moved Permanently

Reproduction

# Bucket exists in us-west-2
# Instance + FSx launching into us-east-2
spawn launch fem-hpc \
    --instance-type hpc6a.48xlarge \
    --count 2 \
    --region us-east-2 \
    --mpi --efa \
    --fsx-create \
    --fsx-s3-bucket my-bucket-in-us-west-2 \
    --fsx-import-path s3://my-bucket-in-us-west-2/inputs/ \
    --fsx-export-path s3://my-bucket-in-us-west-2/outputs/ \
    --fsx-mount-point /fsx \
    --ttl 4h --on-complete stop

Root cause

The S3 HeadBucket call is being made against the global S3 endpoint (or the wrong region endpoint). When a bucket exists in a different region, S3 returns 301 instead of 200. The fix is to use a region-aware S3 client for the bucket existence check — either:

1. Parse the region from the --fsx-s3-bucket URI and construct a region-specific client, or
2. Use GetBucketLocation to discover the bucket's region and redirect accordingly, or
3. Skip the bucket existence check and let FSx return a meaningful error if the bucket doesn't exist (FSx itself handles cross-region validation)

Note on FSx DRA cross-region

As a related point: FSx PERSISTENT_2 Data Repository Associations require the S3 bucket to be in the same region as the filesystem (AWS enforces this). So even after fixing the HeadBucket check, --fsx-create with a cross-region bucket will fail at the DRA creation step. The error message from spawn could be clearer here — ideally detecting region mismatch early and printing:

Error: --fsx-s3-bucket bucket region (us-west-2) does not match instance region (us-east-2).
FSx data repository associations require same-region S3. 
Either use a bucket in us-east-2 or omit --fsx-create and create FSx manually.

Workaround

Create the FSx filesystem separately and pass --fsx-id instead of --fsx-create.

Version

spawn v0.34.13

---

Migrated from spore-host/spore-host#352 — refiled here as spawn issues belong on this repo.

[scttfrdmn]

Fixed in #111 (merged). spawn now sources instance-type capabilities from truffle (GetCapabilities).

[scttfrdmn]

Verified fixed in spawn v0.43.0 (truffle v0.38.1). Cross-region --fsx-create --fsx-s3-bucket <us-west-2 bucket> with an instance in us-east-2 now passes the bucket-existence check cleanly instead of erroring on the 301. Re-tested against the original Tutorial 04 scenario (BuckAI aws-buckai bucket in us-west-2, hpc6a.48xlarge in us-east-2) — the manual --fsx-id workaround is no longer needed. 🎉