release-26.1: cloud/amazon: fix AWS_SKIP_CHECKSUM not suppressing CRC32 on multipart uploads#166236
Open
blathers-crl[bot] wants to merge 1 commit intorelease-26.1from
Open
release-26.1: cloud/amazon: fix AWS_SKIP_CHECKSUM not suppressing CRC32 on multipart uploads#166236blathers-crl[bot] wants to merge 1 commit intorelease-26.1from
blathers-crl[bot] wants to merge 1 commit intorelease-26.1from
Conversation
…t uploads After the aws-sdk-go-v2 s3 manager upgrade to v1.17.65, the manager's internal initChecksumAlgorithm() unconditionally defaults ChecksumAlgorithm to CRC32 when the field is empty. This means setting ChecksumAlgorithm = "" on the PutObjectInput (which the code did when AWS_SKIP_CHECKSUM=true) no longer suppresses checksums for multipart uploads — the manager overwrites the cleared value with CRC32 before the S3 client sees it, bypassing the RequestChecksumCalculation = WhenRequired configuration. Fix this by adding smithy-go Initialize middleware via the Uploader's ClientOptions that clears ChecksumAlgorithm from CreateMultipartUploadInput, UploadPartInput, CompleteMultipartUploadInput, and PutObjectInput before the S3 client's checksum middleware processes them. This runs after the manager's initChecksumAlgorithm but before the client computes checksums, effectively suppressing all checksum computation on multipart uploads when AWS_SKIP_CHECKSUM=true. Resolves: #166090 Release note (bug fix): Fixed a bug where the AWS_SKIP_CHECKSUM=true storage option did not fully suppress checksums on multipart uploads to S3-compatible storage. After an internal SDK upgrade, multipart uploads included CRC32 checksums despite the skip flag, which could cause errors with S3-compatible services that do not support CRC checksums. Generated by Claude Code Auto-Solver Co-Authored-By: Claude <noreply@anthropic.com>
9823d4e to
2b8a5d3
Compare
Author
|
Thanks for opening a backport. Before merging, please confirm that the change does not break backwards compatibility and otherwise complies with the backport policy. Include a brief release justification in the PR description explaining why the backport is appropriate. All backports must be reviewed by the TL for the owning area. While the stricter LTS policy does not yet apply, please exercise judgment and consider gating non-critical changes behind a disabled-by-default feature flag when appropriate. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport 1/1 commits from #166164 on behalf of @cockroach-teamcity.
After the aws-sdk-go-v2 s3 manager upgrade to v1.17.65, the manager's
internal initChecksumAlgorithm() unconditionally defaults ChecksumAlgorithm
to CRC32 when the field is empty. This means setting ChecksumAlgorithm = ""
on the PutObjectInput (which the code did when AWS_SKIP_CHECKSUM=true) no
longer suppresses checksums for multipart uploads — the manager overwrites
the cleared value with CRC32 before the S3 client sees it, bypassing
the RequestChecksumCalculation = WhenRequired configuration.
Fix this by adding smithy-go Initialize middleware via the Uploader's
ClientOptions that clears ChecksumAlgorithm from CreateMultipartUploadInput,
UploadPartInput, CompleteMultipartUploadInput, and PutObjectInput before
the S3 client's checksum middleware processes them. This runs after the
manager's initChecksumAlgorithm but before the client computes checksums,
effectively suppressing all checksum computation on multipart uploads when
AWS_SKIP_CHECKSUM=true.
Resolves: #166090
Release note (bug fix): Fixed a bug where the AWS_SKIP_CHECKSUM=true
storage option did not fully suppress checksums on multipart uploads
to S3-compatible storage. After an internal SDK upgrade, multipart
uploads included CRC32 checksums despite the skip flag, which could
cause errors with S3-compatible services that do not support CRC
checksums.
Generated by Claude Code Auto-Solver
Co-Authored-By: Claude noreply@anthropic.com
This PR was auto-generated by issue-autosolve using Claude Code.
Please review carefully before approving.
Release justification: