-
-
Notifications
You must be signed in to change notification settings - Fork 372
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Check both MD5 locations for S3 KMS support. #1167
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1167 +/- ##
==========================================
- Coverage 74.63% 74.62% -0.01%
==========================================
Files 144 144
Lines 16299 16312 +13
==========================================
+ Hits 12164 12173 +9
- Misses 3188 3192 +4
Partials 947 947 ☔ View full report in Codecov by Sentry. |
I am getting a build error after rebase on master:
could you try to fix the build on master ? |
Rebased on master, some fixes and updated test: #1272 Could you maybe check if this works for you ? |
If the S3 bucket used to house a repo has KMS encryption enabled then the etag of an object may not match the MD5 of the file. This may cause an incorrect error to be reported stating the file already exists and is different. A mechanism exists to work around this issue by using the MD5 stored in object metadata. This check doesn't always cover the case where KMS is enabled as the fallback is only used if the etag is not 32 characters long. This commit changes the fallback mechanism so that it is used in any case where the object's etag does not match the source MD5. This will incur a performance penalty of an extra head request for each object with a mismatch.
Adds check to see if the S3 bucket is encrypted by default. If so this uses the existing workaround for object etags not matching file MD5s.
1b4c0f7
to
8e3c12f
Compare
abandonned #1272, continuing here |
Fixes #1117
Requirements
All new code should be covered with tests, documentation should be updated. CI should pass.
Description of the Change
If the S3 bucket used to house a repo has KMS encryption enabled then the etag of an object may not match the MD5 of the file. This may cause an incorrect error to be reported stating the file already exists and is different.
A mechanism exists to work around this issue by using the MD5 stored in object metadata. This check doesn't always cover the case where KMS is enabled as the fallback is only used if the etag is not 32 characters long.
This commit changes the fallback mechanism so that it is used in any case where the object's etag is not 32 characters or if the S3 bucket has encryption enabled for new objects by default.
Checklist
AUTHORS