chainloop artifact upload seems to fail for S3 buckets

[javirln]

UPDATE
This is the issue https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/feature/s3/manager@v1.20.1#hdr-Pre_computed_Checksums-Uploader

We have seen a weird behaviour when trying to upload some artifacts to a S3 bucket. Some artifacts report the following:

$ chainloop artifact upload --file ~/Downloads/chainloop-linux-amd64
INF uploading chainloop-linux-amd64 - sha256:44c7fa3a3545c29c8c990699c3209aeaef1d71f3773a0b53d7a5ca9d3910f56b
 ... done! [89.84MB in 7.713s; 11.65MB/s]
ERR server error

The error on the CAS side is something like:

{"level":"error","ts":1762342756.7378519,"component":"service","msg":"failed to upload to bucket: upload multipart failed, upload id: THE-ID, cause: operation error S3: CompleteMultipartUpload, https response error StatusCode: 400, RequestID: OTHER-ID, HostID: HOST-ID, api error BadDigest: The sha256 you specified did not match the calculated checksum."...

Uploading to OCI seems to work fine:

$ chainloop artifact upload --file ~/Downloads/chainloop-linux-amd64
 ... done! [89.84MB in 10.1s; 8.90MB/s]

[migmartri]

reproduced locally

[migmartri]

reverting to a previous commit to the AWS SDK upgrade makes it work, so it was indeed introduced there

git revert 7b72e14dc1206e1541cc78785dd6fb1b1ac6709f

mkfile -n 10M temp_10GB_file

go run main.go artifact upload -f temp_10GB_file
WRN API contacted in insecure mode
WRN API contacted in insecure mode
INF uploading temp_10GB_file - sha256:e5b844cc57f57094ea4585e235f36c78c1cd222262bb89d53c94dcb4d6b3e55d
 ... done! [10.49MB in 11.896s; 881.40KB/s]

[migmartri]

The problem is related to the checksum check

	if b.checksumVerificationEnabled() {
		// Check that the object is uploaded correctly
		input.ChecksumSHA256 = aws.String(hexSha256ToBinaryB64(resource.Digest))
	}

and it only happens if the file is uploaded as multipart, meaning it's bigger than 5MB

[jiparis]

This seems related aws/aws-sdk-js-v3#6742, using PutObject from client-s3 instead of Upload might be a solution

[migmartri]

The issue explained here https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/feature/s3/manager@v1.20.1#hdr-Pre_computed_Checksums-Uploader

[migmartri]

Another issue aws/aws-sdk-go-v2#3005

[jiparis]

For now, we will turn off the validation, since there is no way to generate FULL_OBJECT digests with SHA256 (only CRC algorithms are supported for multiplart uploads)