list-buckets option to get extra --details about buckets

[simonw]

The S3 security best practices in https://docs.aws.amazon.com/AmazonS3/latest/userguide/security-best-practices.html#security-best-practices-prevent suggest:

Use the ListBuckets API to scan all of your Amazon S3 buckets. Then use GetBucketAcl, GetBucketWebsite, and GetBucketPolicy to determine whether the bucket has compliant access controls and configuration.

list-buckets could do this with an extra --details option (since it adds 3 new API calls per bucket).

[simonw]

Relevant to:

• Use PublicAccessBlockConfiguration for extra defense in depth against accidental public access #20
• Support configuring the bucket as a website #21

[simonw]

Example output from my prototype:

{
    "Name": "dogsheep-photos-simon",
    "CreationDate": "2020-04-18 19:36:25+00:00",
    "bucket_acl": {
        "Owner": {
            "DisplayName": "swillison",
            "ID": "36b2eeee501c5952a8ac119f9e5212277a4c01eccfa8d6a9d670bba1e2d5f441"
        },
        "Grants": [
            {
                "Grantee": {
                    "DisplayName": "swillison",
                    "ID": "36b2eeee501c5952a8ac119f9e5212277a4c01eccfa8d6a9d670bba1e2d5f441",
                    "Type": "CanonicalUser"
                },
                "Permission": "FULL_CONTROL"
            }
        ]
    },
    "public_access_block": {
        "BlockPublicAcls": true,
        "IgnorePublicAcls": true,
        "BlockPublicPolicy": true,
        "RestrictPublicBuckets": true
    },
    "bucket_website": null
}

[simonw]

Since this is an expensive operation per bucket, I'm going to add the ability to pass one or more specific bucket names to the command.

[simonw]

Documentation: https://github.com/simonw/s3-credentials/blob/8e207980a23738e5e9119902d1314d51669431e4/README.md#list-buckets

[simonw]

Here's the updated --help:

% s3-credentials list-buckets --help
Usage: s3-credentials list-buckets [OPTIONS] [BUCKETS]...

  List buckets - defaults to all, or pass one or more bucket names

Options:
  --details             Include extra bucket details (slower)
  --array               Output a valid JSON array
  --nl                  Output newline-delimited JSON
  --access-key TEXT     AWS access key ID
  --secret-key TEXT     AWS secret access key
  --session-token TEXT  AWS session token
  --endpoint-url TEXT   Custom endpoint URL
  --help                Show this message and exit.