S3 command fails silently when copying large file to location without permission

[phss]

This only happens when copying files larger than a certain size, but not sure what is the threshold. For smaller files, when copying to a directory I don't have permission, I correctly see the permission denied error and the command exits with status 1. But for large files the command just stops, without printing to the output and the exit status is 0.

Steps to reproduce:

mkdir access-denied
sudo chown root:wheel access-denied/
aws s3 cp s3://some-bucket/large-file.gz access-denied/
# Result: nothing in the output and exit code is 0
# Expected: exit code of 1 and output showing premission denied

Looking at the code, it seems the S3 copy command behaves differently depending on the size of the file being copied. For smaller files it seem to just download the file in a single thread. But for larger files it starts multiple threads, each downloading parts of the file. The single threaded process correctly output the error, but the multi-threaded doesn't.

I suspect this might affect other S3 commands as well.

Pull request to follow.

[phss]

I think this is also related to issues #1069 and #1442.

On #1069, the reported mentioned copying a large file. On #1442, there are mentions of the copy command failing silently and with 0 status code.

[jamesls]

Excellent, taking a look at this now.

[jamesls]

@phss I've confirmed that your PR also fixes #1442:

$ aws s3 cp s3://bucket/larger ~/tmpfs/larger2
download failed: s3://bucket/larger to ../../tmpfs/larger2
[Errno 28] No space left on device: u'/Users/jamessar/tmpfs/larger2'
$ echo $?
1

Thanks!