x-amz-meta-checksum

[iarvind]

New feature, improvement proposal

Description

Maven Resolver currently supports extracting checksums from HTTP response headers using the following conventions:

x-checksum-sha1
x-checksum-md5
x-goog-meta-checksum-sha1
x-goog-meta-checksum-md5

AWS S3 and S3-compatible object storage services, such as Cloudflare R2, expose user-defined object metadata using the x-amz-meta-* prefix.

For example, an object uploaded with the metadata:

checksum-sha1 = c74edb60ca2a0b57ef88d9a7da28f591e3d4ce7b
checksum-md5 = 9ad0d8e3482767c122e85f83567b8ce6

is returned with the following response headers:

x-amz-meta-checksum-sha1: c74edb60ca2a0b57ef88d9a7da28f591e3d4ce7b
x-amz-meta-checksum-md5: 9ad0d8e3482767c122e85f83567b8ce6

Maven Resolver does not currently recognize these headers. As a result, when an artifact download is redirected to an S3-compatible presigned URL, Resolver downloads the artifact successfully but subsequently makes additional requests for checksum sidecar files. This happens even though the checksum is already available in the final artifact download response.

Proposed enhancement

Extend XChecksumExtractor to recognize the following headers:

x-amz-meta-checksum-sha1
x-amz-meta-checksum-md5

The values should use the standard hexadecimal representation:

SHA-1: 40 hexadecimal characters
MD5: 32 hexadecimal characters

This enhancement would improve support for repositories backed by AWS S3 and S3-compatible object stores, including Cloudflare R2.

This proposal is specifically for S3 user-defined metadata headers:

x-amz-meta-checksum-sha1
x-amz-meta-checksum-md5

Relevant code
maven-resolver-impl/src/main/java/
org/eclipse/aether/internal/impl/transport/http/XChecksumExtractor.java

I would be happy to submit a pull request with the implementation, tests, and documentation updates.

[elharo]

Any specs around this? I'd prefer not to implement this for every provider's idiosyncratic headers.

Does this accomplish anything besides detecting bit corruption in transit? It doesn't seem to be MITM protection.

[cstamas]

Could you provide a PR for this? Thanks

@elharo There is no spec, it is just big ones doing this since ages, really.

[cstamas]

@elharo see here for benefits https://maven.apache.org/resolver/expected-checksums.html
See "Remote Included" ones. Benefit is literally cutting HTTP request count to half (without: one GET for JAR one GET for jar.sha1; with: one GET for both).

[iarvind]

Thanks! I just raised the PR. This should cut the HTTP request count in half, not only for files served through AWS S3 but also for many S3-compatible storage solutions such as Wasabi, Backblaze B2, Cloudflare R2, and Minio.