Announcement: S3 default integrity change

[aBurmeseDev]

In AWS SDK for Go v2 service/s3 v1.73.0, we released changes to the S3 client that adopts new default integrity protections. For more information on default integrity behavior, please refer to the official SDK documentation. In SDK releases from this version on, clients default to enabling an additional checksum on all Put calls and enabling validation on Get calls.

You can disable default integrity protections for S3. We do not recommend this because checksums are important to S3 integrity posture. Integrity protections can be disabled by setting the config flag to when_required, or by using the related AWS shared config file settings or environment variables.

Compatibility w/ 3rd-party S3 emulators

Disclaimer: the AWS SDKs and CLI are designed for usage with official AWS services. We may introduce and enable new features by default, such as these new default integrity protections, prior to them being supported or otherwise handled by third-party service implementations. You can disable the new behavior with the WHEN_REQUIRED value for the request_checksum_calculation and response_checksum_validation configuration options covered in Data Integrity Protections for Amazon S3.

Reinstating MD5 for checksum-required operations

Prior to this update, S3 operations that required a checksum - such as DeleteObjects (plural, not singular) - would have the SDK compute and send an MD5 checksum of the request payload by default. Sending an alternative checksum, such as CRC32, was always possible, but the SDKs did not exhibit this behavior.

After this update, the Go SDK will instead compute and send a CRC32 checksum. This may result in compatibility issues with 3rd-party S3 emulators whose versions of these APIs were written to expect MD5.

The following code snippet demonstrates a re-usable functional option, withContentMD5, which reinstates the Content-MD5 checksum header behavior. You can apply this option to individual service calls if you experience any compatibility issues with 3rd-party service emulators.

package main

import (
	"context"

	"github.com/aws/aws-sdk-go-v2/aws"
	"github.com/aws/aws-sdk-go-v2/config"
	"github.com/aws/aws-sdk-go-v2/service/s3"
	"github.com/aws/aws-sdk-go-v2/service/s3/types"
	"github.com/aws/smithy-go/middleware"
	smithyhttp "github.com/aws/smithy-go/transport/http"
)

// withContentMD5 removes all flexible checksum procecdures from an operation,
// instead computing an MD5 checksum for the request payload.
func withContentMD5(o *s3.Options) {
	o.APIOptions = append(o.APIOptions, func(stack *middleware.Stack) error {
		stack.Initialize.Remove("AWSChecksum:SetupInputContext")
		stack.Build.Remove("AWSChecksum:RequestMetricsTracking")
		stack.Finalize.Remove("AWSChecksum:ComputeInputPayloadChecksum")
		stack.Finalize.Remove("addInputChecksumTrailer")
		return smithyhttp.AddContentChecksumMiddleware(stack)
	})
}

// example
func main() {
	cfg, err := config.LoadDefaultConfig(context.Background())
	if err != nil {
		panic(err)
	}

	svc := s3.NewFromConfig(cfg)
	_, err = svc.DeleteObjects(context.Background(), &s3.DeleteObjectsInput{
		Bucket: aws.String("bucket"),
		Delete: &types.Delete{
			Objects: []types.ObjectIdentifier{
				{Key: aws.String("0-sdk-upload-md5")},
			},
		},
	}, withContentMD5) // apply the content-md5 override to this particular request
	if err != nil {
		panic(err)

	}
}

[stefansundin]

Hello, I have a question that I haven't seen answered anywhere yet.

It appears that as of right now, this SDK does not support calculating CRC64NVME checksums. I found one comment (#2981 (comment)) saying "since we don't support it natively", but it didn't elaborate more than that.

Is support coming in the future, and if so is there a timeline? Can someone elaborate more on why it isn't supported yet?

It seems like crc64.New(crc64.MakeTable(0x9a6c9329ac4bc9b5)) is working for me, but I won't implement it on my own program unless I really have to. :)

Thank you!

[mdelvecchio-wasabi]

I ran into the same issue, and it's not as simple as that. The polynomial you are specifying does not look right. In addition, the initial value is 0xFFFFFFFFFFFFFFFF, and the final value has to be XORd with that same number.

My colleague figured this out from https://patches.linaro.org/project/linux-crypto/patch/20220303201312.3255347-6-kbusch@kernel.org/. We tested it against the AWS S3 client, and were able to get it right that way.

[mdelvecchio-wasabi]

I have the same issue; I updated all of our SDK modules to the latest. In algorithms.go, I see AlgorithmCRC64NVME defined alongside the original four, but that value does not appear in the supportedAlgorithms values.

Why is this algorithm not supported in the Go SDK? It is supported in the AWS S3 CLI I just installed (version 2.23.5).

[nguyengg]

This feature was not turned on by default in a backwards-compatible way.

In nguyengg/xy3#1, I have my own S3 uploader that uses multipart upload and I found that in s3@v1.75.4 (probably start with v1.73.0 - https://github.com/aws/aws-sdk-go-v2/blob/main/service/s3/CHANGELOG.md#v1730-2025-01-15), the response from each UploadPart call starts returning a ChecksumCRC32 (which tracks with https://docs.aws.amazon.com/sdkref/latest/guide/feature-dataintegrity.html#dataintegrity-sdk-compat saying CRC32 is the default algorithm for SDK Go v2). However, my library wasn't passing any ChecksumCRC32 to the final CompleteMultipartUpload because my customers weren't using this feature. As a result, upload operations started failing with error messages like this:

cause: operation error S3: CompleteMultipartUpload, https response error StatusCode: 400, RequestID: R5B20AV083Y6HD7S, HostID: E6JXPySCHvlrX/ehR/jWufF4HnhcfVKrAfB0FF3Vjvc97d1iS0VaG03yAviRHlybylla+7sNQBE=, api error InvalidPart: One or more of the specified parts could not be found. The part may not have been uploaded, or the specified entity tag may not match the part's entity tag.

I have to disable this feature with:

	c.client = s3.NewFromConfig(cfg, func(options *s3.Options) {
		options.RequestChecksumCalculation = aws.RequestChecksumCalculationWhenRequired
	})

Regardless, it is a pretty painful customer experience to have your dependencies introducing hidden breaking changes.