s3-credentials put-object and s3-credentials get-object commands

[simonw]

I'd like to be able to use this tool to easily upload and download from the S3 buckets, without having to switch to (and install) a a separate tool.

[simonw]

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Client.put_object

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Client.get_object

[simonw]

Design for put-object:

s3-credentials put-object my-bucket my-pictures/panda.jpg panda.jpg

Or:

echo "Content" | s3-credentials put-object my-bucket my-key.txt -

[simonw]

For get-object:

s3-credentials get-object my-bucket my-key.txt # Outputs to standard output

Or:

s3-credentials get-object my-bucket my-key.txt -o file.txt

[simonw]

Maybe in the future add --acl to put-object - from the boto docs:

ACL='private'|'public-read'|'public-read-write'|'authenticated-read'|'aws-exec-read'|'bucket-owner-read'|'bucket-owner-full-control'

[simonw]

Tried this:

@cli.command()
@click.argument("bucket")
@click.argument("key")
@click.argument("body", type=click.File("rb"))
@common_boto3_options
def put_object(bucket, key, body, **boto_options):
    "Upload an object to an S3 bucket"
    s3 = make_client("s3", **boto_options)
    response = s3.put_object(Body=body, Bucket=bucket, Key=key)
    click.echo(json.dumps(response, indent=4, default=str))

And got this error:

 % echo "Hello world" | s3-credentials put-object static.niche-museums.com hello.txt -
Traceback (most recent call last):
...
  File "/Users/simon/.local/share/virtualenvs/s3-credentials-J8M1ChYK/lib/python3.10/site-packages/botocore/utils.py", line 2434, in _calculate_md5_from_file
    start_position = fileobj.tell()
OSError: [Errno 29] Illegal seek

[simonw]

I should use this module, it handles lots of interesting extra features, including progress bars: https://github.com/boto/boto3/blob/c5487b57499ad354e24ca2c8622f90fe089240b6/boto3/s3/transfer.py

[simonw]

I'm going to test these as part of the existing integration tests.