Summary logging

[curry684]

We use s3cmd to upload virtual machine backups of several gigabytes (up to 45GB) to S3 regularly, through cron. Since we pipe the backups through pigz/gzip we have no clue in the surrounding scripting how much we're uploading, and how fast. As we also use multipart uploads that means we currently have 2 logging options: either we get 5000+ output lines describing how fast each part was uploaded, or none (--quiet). In either case we get no useful statistics, or the total file size, at the end.

I would like to propose a --summary option which suppresses all output like --quiet does, but instead at the end prints a small report like, for put:

Uploaded 39.2GB of data in 3:45:10 hours, average 3.24MB/s

[fviard]

For "sync" commands, you have such a line that is displayed at the end, and with the --stats flag, you can also enable some interesting summary stats. But this doesn't enable --quiet, and will not work if --quiet is enabled.

The same thing that is in sync could probably be also added to the put and get commands. (Patches for that are welcome ;-) )

[curry684]

Would love to help if my python capacities were anywhere near my programming skills elsewhere, but unfortunately they most certainly are not hehe.

[mattpker]

We also have this issue, we want to be able to log the results of a sync but not log every single file that gets transferred. The --stats flag gives us the output we need to log, but if we add --quiet it also suppresses the stats we need.