aws s3 cp from stdin

[bukzor]

I'd like to upload a tarball to s3 without incurring the cost of a temporary file (we're low on disk). With other utilities, I'd pipe the output of the tar command to the uploading command, but it doesn't Just Work with aws-cli:

$ echo ok | aws s3 cp /dev/stdin s3://my-bucket/test
upload failed: /dev/stdin to s3://my-bucket/test [Errno 29] Illegal seek

It seems like this simple curl script works however:

#!/bin/bash
configfile="aws.ini"
file="/test"
bucket="my-bucket"
resource="/${bucket}${file}"
contentType="application/x-compressed-tar"

# Grab the config values
eval $(cat aws.ini  | grep -P "^\w+=[-'/\w]+$")

# Calculate the signature.
dateValue=$(date -R)
stringToSign="PUT\n\n${contentType}\n${dateValue}\n${resource}"
signature=$(echo -en ${stringToSign} | openssl sha1 -hmac ${aws_secret_access_key} -binary | base64)

# PUT!
curl \
    -X PUT \
    --data @- \
    -H "Host: ${bucket}.s3.amazonaws.com" \
    -H "Date: ${dateValue}" \
    -H "Content-Type: ${contentType}" \
    -H "Authorization: AWS ${aws_access_key_id}:${signature}" \
    "https://${bucket}.s3.amazonaws.com${file}"

Can we please get similar capability in aws s3 cp?

[bukzor]

The other direction doesn't have this issue; I can force aws-cli to use stdout with:

aws s3 cp s3://my-bucket/test >(cat)

It would be better if it followed the usual unix "-" convention though.

[kyleknap]

Feature added via #903. Closing the issue.