set-cors-policy command

[simonw]

For setting the bucket-level CORS policy on newly created (or maybe even existing?) buckets.

See https://til.simonwillison.net/aws/s3-cors

[simonw]

I need a CORS-enabled bucket today in order to experiment more with micropip - https://pyodide.org/en/stable/usage/api/micropip-api.html

[simonw]

CORS policies can look like this:

[
    {
        "AllowedHeaders": [
            "*"
        ],
        "AllowedMethods": [
            "GET"
        ],
        "AllowedOrigins": [
            "https://simonwillison.net/"
        ],
        "ExposeHeaders": []
    }
]

Or this - from https://grrr.tech/posts/2022/cors-s3-bucket/

{
    "CORSRules": [
        {
            "AllowedOrigins": ["*"],
            "AllowedHeaders": ["*"],
            "AllowedMethods": ["GET", "POST"]
        }
    ]
}

Full documentation here: https://docs.aws.amazon.com/AmazonS3/latest/API/API_CORSRule.html - which lists the following options:

• AllowedHeaders - array of strings
• AllowedMethods - array of strings (required)
• AllowedOrigins - array of strings (required)
• ExposeHeaders - array of strings
• ID - string, optional - "Unique identifier for the rule"
• MaxAgeSeconds - integer, optional, "The time in seconds that your browser is to cache the preflight response for the specified resource"

[simonw]

I could do this as a --cors-policy option to s3-credentials create, or I could have a separate s3-credentials cors-policy command which can be run afterwards against a bucket that has been created.

I'm leaning towards that second option because there are a bunch of different options here and it feels like a command would make it easier to express them.

But I could always do s3-credentials create ... --cors-policy path-to-json.json as well.

[simonw]

Boto documentation: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/s3-example-configuring-buckets.html#set-a-bucket-cors-configuration

# Define the configuration rules
s3 = boto3.client("s3")
s3.put_bucket_cors(Bucket="BUCKET_NAME", CORSConfiguration={
    "CORSRules": [
        {
            "AllowedHeaders": ["Authorization"],
            "AllowedMethods": ["GET", "PUT"],
            "AllowedOrigins": ["*"],
            "ExposeHeaders": ["GET", "PUT"],
            "MaxAgeSeconds": 3000,
        }
    ]
})

Though that example looks misleading to me - "ExposeHeaders": ["GET", "PUT"] seems wrong, that should be a list of header names that can be exposed in the response, not a list of methods.

[simonw]

Filed a PR to fix that documentation example:

• Fix example for ExposeHeaders boto/boto3#3251

[simonw]

Moro boto documentation:

• get_bucket_cors(Bucket=name) - returns {"CORSRules": [...]}
• delete_bucket_cors(Bucket=name) - deletes ALL of the CORS rules for the bucket in one go
• put_bucket_cors(Bucket=name, CORSConfiguration=[...]) - replaces the existing configuration with new configuration, provided as a list of up to 100 rules

Interesting to note that while the list of CORSRules would hint that you can add a rule to an existing configuration that's not how these APIs work - you have to provide the FULL list of rules any time you update any of them.

[simonw]

Design for this command:

s3-credentials set-cors-policy mybucket --policy path-to-full-policy.json
# Or to set just a single policy using options
s3-credentials set-cors-policy mybucket --allowed-method GET --allowed-method PUT --expose-header ETag
# And to delete all of the policies
s3-credentials set-cors-policy mybucket --clear

[simonw]

If you don't provide a --allowed-origin it will default to ["*"].

[simonw]

I ran my prototype:

s3-credentials set-cors-policy simonwillison-cors-allowed

I added a s3-credentials get-cors-policy command too, mainly so I could see if the change had been recorded:

% s3-credentials get-cors-policy simonwillison-cors-allowed
[
    {
        "ID": "set-by-s3-credentials",
        "AllowedMethods": [
            "GET"
        ],
        "AllowedOrigins": [
            "*"
        ]
    }
]

[simonw]

Forgot to make the bucket public. Trying this instead:

% s3-credentials create simonwillison-cors-allowed-public --public --create-bucket
Created bucket: simonwillison-cors-allowed-public
Attached bucket policy allowing public access
Created  user: 's3.read-write.simonwillison-cors-allowed-public' with permissions boundary: 'arn:aws:iam::aws:policy/AmazonS3FullAccess'
Attached policy s3.read-write.simonwillison-cors-allowed-public to user s3.read-write.simonwillison-cors-allowed-public
Created access key for user: s3.read-write.simonwillison-cors-allowed-public
{
    "UserName": "s3.read-write.simonwillison-cors-allowed-public",
    "AccessKeyId": "AKIAWXFXAIOZDHIP6CVN",
    "Status": "Active",
    "SecretAccessKey": "...",
    "CreateDate": "2022-05-01 19:59:09+00:00"
}
% s3-credentials set-cors-policy simonwillison-cors-allowed-public
% s3-credentials get-cors-policy simonwillison-cors-allowed-public
[
    {
        "ID": "set-by-s3-credentials",
        "AllowedMethods": [
            "GET"
        ],
        "AllowedOrigins": [
            "*"
        ]
    }
]
% s3-credentials put-object simonwillison-cors-allowed-public click_default_group-1.2.2-py3-none-any.whl /tmp/click-default-group/dist/click_default_group-1.2.2-py3-none-any.whl

[simonw]

https://s3.amazonaws.com/simonwillison-cors-allowed-public/click_default_group-1.2.2-py3-none-any.whl is now a file I can download.

It looks like that is setting the right CORS headers, provided you send an Origin: header:

~ % curl -i 'https://s3.amazonaws.com/simonwillison-cors-allowed-public/click_default_group-1.2.2-py3-none-any.whl' -H "Origin: http://www.example.com/"
HTTP/1.1 200 OK
x-amz-id-2: rx1LP6nXSxXBw/ZRyJhbzFEPnOziNZEYy1lYGytqjfdxlEzW62cgGzeEQWi4vG8dpTi6I3iRlFc=
x-amz-request-id: EXVJQX06ZF0EX4VW
Date: Sun, 01 May 2022 20:03:30 GMT
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET
Vary: Origin, Access-Control-Request-Headers, Access-Control-Request-Method