Skip to content

release-26.1: cloud/amazon: fix AWS_SKIP_CHECKSUM not suppressing CRC32 on multipart uploads#166236

Open
blathers-crl[bot] wants to merge 1 commit intorelease-26.1from
blathers/backport-release-26.1-166164
Open

release-26.1: cloud/amazon: fix AWS_SKIP_CHECKSUM not suppressing CRC32 on multipart uploads#166236
blathers-crl[bot] wants to merge 1 commit intorelease-26.1from
blathers/backport-release-26.1-166164

Conversation

@blathers-crl
Copy link

@blathers-crl blathers-crl bot commented Mar 19, 2026

Backport 1/1 commits from #166164 on behalf of @cockroach-teamcity.


After the aws-sdk-go-v2 s3 manager upgrade to v1.17.65, the manager's
internal initChecksumAlgorithm() unconditionally defaults ChecksumAlgorithm
to CRC32 when the field is empty. This means setting ChecksumAlgorithm = ""
on the PutObjectInput (which the code did when AWS_SKIP_CHECKSUM=true) no
longer suppresses checksums for multipart uploads — the manager overwrites
the cleared value with CRC32 before the S3 client sees it, bypassing
the RequestChecksumCalculation = WhenRequired configuration.

Fix this by adding smithy-go Initialize middleware via the Uploader's
ClientOptions that clears ChecksumAlgorithm from CreateMultipartUploadInput,
UploadPartInput, CompleteMultipartUploadInput, and PutObjectInput before
the S3 client's checksum middleware processes them. This runs after the
manager's initChecksumAlgorithm but before the client computes checksums,
effectively suppressing all checksum computation on multipart uploads when
AWS_SKIP_CHECKSUM=true.

Resolves: #166090

Release note (bug fix): Fixed a bug where the AWS_SKIP_CHECKSUM=true
storage option did not fully suppress checksums on multipart uploads
to S3-compatible storage. After an internal SDK upgrade, multipart
uploads included CRC32 checksums despite the skip flag, which could
cause errors with S3-compatible services that do not support CRC
checksums.

Generated by Claude Code Auto-Solver
Co-Authored-By: Claude noreply@anthropic.com


 pkg/cloud/amazon/BUILD.bazel        |  4 +++
 pkg/cloud/amazon/s3_storage.go      | 29 ++++++++++++++++++
 pkg/cloud/amazon/s3_storage_test.go | 59 +++++++++++++++++++++++++++++++++++++
 3 files changed, 92 insertions(+)

This PR was auto-generated by issue-autosolve using Claude Code.
Please review carefully before approving.


Release justification:

…t uploads

After the aws-sdk-go-v2 s3 manager upgrade to v1.17.65, the manager's
internal initChecksumAlgorithm() unconditionally defaults ChecksumAlgorithm
to CRC32 when the field is empty. This means setting ChecksumAlgorithm = ""
on the PutObjectInput (which the code did when AWS_SKIP_CHECKSUM=true) no
longer suppresses checksums for multipart uploads — the manager overwrites
the cleared value with CRC32 before the S3 client sees it, bypassing
the RequestChecksumCalculation = WhenRequired configuration.

Fix this by adding smithy-go Initialize middleware via the Uploader's
ClientOptions that clears ChecksumAlgorithm from CreateMultipartUploadInput,
UploadPartInput, CompleteMultipartUploadInput, and PutObjectInput before
the S3 client's checksum middleware processes them. This runs after the
manager's initChecksumAlgorithm but before the client computes checksums,
effectively suppressing all checksum computation on multipart uploads when
AWS_SKIP_CHECKSUM=true.

Resolves: #166090

Release note (bug fix): Fixed a bug where the AWS_SKIP_CHECKSUM=true
storage option did not fully suppress checksums on multipart uploads
to S3-compatible storage. After an internal SDK upgrade, multipart
uploads included CRC32 checksums despite the skip flag, which could
cause errors with S3-compatible services that do not support CRC
checksums.

Generated by Claude Code Auto-Solver
Co-Authored-By: Claude <noreply@anthropic.com>
@blathers-crl blathers-crl bot force-pushed the blathers/backport-release-26.1-166164 branch from 9823d4e to 2b8a5d3 Compare March 19, 2026 23:15
@blathers-crl blathers-crl bot requested a review from a team as a code owner March 19, 2026 23:15
@blathers-crl blathers-crl bot requested review from kev-cao and removed request for a team March 19, 2026 23:15
@blathers-crl blathers-crl bot added blathers-backport This is a backport that Blathers created automatically. O-robot Originated from a bot. labels Mar 19, 2026
@blathers-crl blathers-crl bot requested review from dt and rafiss March 19, 2026 23:15
@blathers-crl
Copy link
Author

blathers-crl bot commented Mar 19, 2026

Thanks for opening a backport.

Before merging, please confirm that the change does not break backwards compatibility and otherwise complies with the backport policy. Include a brief release justification in the PR description explaining why the backport is appropriate. All backports must be reviewed by the TL for the owning area. While the stricter LTS policy does not yet apply, please exercise judgment and consider gating non-critical changes behind a disabled-by-default feature flag when appropriate.

@blathers-crl blathers-crl bot added backport Label PR's that are backports to older release branches T-disaster-recovery labels Mar 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport Label PR's that are backports to older release branches blathers-backport This is a backport that Blathers created automatically. O-robot Originated from a bot. T-disaster-recovery

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant