Add support for aws_chunked with s3v4.#996
Conversation
|
I wasn't sure if this is the right approach to this problem and the PR doesn't have any unit tests at the moment. If there is agreement that this make sense or guidance on how to rework it, I'm happy to work further on the PR and add the tests. |
Codecov Report
@@ Coverage Diff @@
## develop #996 +/- ##
===========================================
- Coverage 98.03% 96.68% -1.36%
===========================================
Files 45 44 -1
Lines 7345 7264 -81
===========================================
- Hits 7201 7023 -178
- Misses 144 241 +97
Continue to review full report at Codecov.
|
5eff6a0 to
8cf59e6
Compare
|
Linking issues, boto3 tracking issue: boto/boto3#751 Pending discussion over on the issue. |
8cf59e6 to
b1e1297
Compare
When uploading to S3 from a stream, it would be useful to opt into S3 aws_chunked uploads with v4 signatures. The mechanism is documented here: http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html The feature is implemented through a new auth class -- S3SigV4ChunkedAuth. The class wraps the body of the request with a ChunkedUploadWrapper instance. It takes care of producing fixed size upload chunks and computing the required signatures for each one. To opt into this scheme, the caller is required to set the aws_chunked option for client configuration and it will only be used with the PutObject operations. Fixes boto#995
b1e1297 to
d27e1e6
Compare
|
Greetings! It looks like this issue hasn’t been active in longer than one year. We encourage you to check if this is still an issue in the latest release. Because it has been longer than one year since the last update on this, and in the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment to prevent automatic closure, or if the issue is already closed, please feel free to reopen it. |
|
I don't know if there is anything I could've done to draw more attention to this or generate discussion. In light of lack of desire to discuss this feature, it probably makes sense to close it. |
When uploading to S3 from a stream, it would be useful to opt into S3
aws_chunked uploads with v4 signatures. The mechanism is documented
here:
http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html
The feature is implemented through a new auth class --
S3SigV4ChunkedAuth. The class wraps the body of the request with a
ChunkedUploadWrapper instance. It takes care of producing fixed size
upload chunks and computing the required signatures for each one.
To opt into this scheme, the caller is required to set the aws_chunked
option for client configuration and it will only be used with the
PutObject operations.
Fixes #995