Add a method to download an archive to a file
This method will also verify the tree hashes for each chunk downloaded.
Adding the params to the uri arg will cause the sigv4 signer to create the wrong canonical string. By explicitly passing the params as a separate arg, the sigv4 signer can do the right thing. Previously this would cause signature errors.
This arg is then forwarded to build_base_http_request, which previously was always passing an empty params dict.
Support for returned NotificationARNs in cloudformation
…is causing problems. Fixes #999.
In order to be consistent with the existing connect_* functions in boto/__init__.py, I've updated the signature to just accept access_key/secret_key, and **kwargs.
…95a3ad8 Add initial Glacier support in root boto object.
Add threaded multipart archive upload for glacier
Use simplejson when json module not available
This will internally create a thread pool that concurrently uploads the various chunks of a file using the multipart API for glacier. When testing upload speeds, this was approximately 13 times faster than the single threaded archive upload. It also is much better at sustaining a constant upload rate, in the single threaded upload, the upload rate fluctuated quite a bit (which makes sense given that it part of the time it's reading from disk and computing hashes instead of sending data). For comparison, for a 1GB file: Operation | Time (secs) | Average Upload rate (MB/s) | --------------------------------------------------------------- s3multiput 216 6.0 glacier-single 1291 2.0 glacier-concurrent 98 11.5
This was for abort_multipart_upload and list_parts, and list_multipart_uploads
If python<2.6 is used (including Jython), the json module isn't yet available in the standard library. Using simplejson as a fallback, if available, would be nice.
The make_request method uses urllib yet it is not imported.
The timestamps for expiration time look like this: "Expiration" : "2012-09-18T21:21:58Z" So we should be using utcnow() when checking for credential expiry.
…med upload already complete