Skip to content

Skip object checksum check on upload if server side AWS KMS encryption is used#1406

Merged
Kami merged 7 commits intoapache:trunkfrom
Kami:fix_s3_hash_checksum_on_encrypted_objects
Jan 14, 2020
Merged

Skip object checksum check on upload if server side AWS KMS encryption is used#1406
Kami merged 7 commits intoapache:trunkfrom
Kami:fix_s3_hash_checksum_on_encrypted_objects

Conversation

@Kami
Copy link
Copy Markdown
Member

@Kami Kami commented Jan 10, 2020

This pull request updates S3 driver code to skip uploaded object checksum check at the end of the upload when server side AWS KMS encryption is used for that object.

In such scenario, ETag header value doesn't contain object data MD5 digest.

As part of that change I also noticed some of the tests were out of date and some of the mock http methods are not needed anymore so I did some cleanup. That likely happened during the requests migration.

Resolves #1401.

Kami added 4 commits January 10, 2020 06:04
is encrypted using AWS KMS encryption.

In such scenarios, ETag header doesn't contain object data MD5 digest.

See https://docs.aws.amazon.com/AmazonS3/latest/API/RESTCommonResponseHeaders.html
and apache#1401 for details.
It appears they become obsolete and unused during requests migration.
@codecov-io
Copy link
Copy Markdown

Codecov Report

Merging #1406 into trunk will increase coverage by 0.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##            trunk    #1406      +/-   ##
==========================================
+ Coverage   86.44%   86.46%   +0.01%     
==========================================
  Files         366      366              
  Lines       76767    76773       +6     
  Branches     7529     7529              
==========================================
+ Hits        66365    66378      +13     
+ Misses       7534     7527       -7     
  Partials     2868     2868
Impacted Files Coverage Δ
libcloud/test/storage/test_s3.py 92.77% <100%> (+1.3%) ⬆️
libcloud/storage/drivers/s3.py 89.34% <100%> (+0.06%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 435d82b...8bf5724. Read the comment docs.

@Kami Kami merged commit 17f9636 into apache:trunk Jan 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Uploading to s3 encrypted with a custom KMS key fails

2 participants