aws_sigv4: consult x-%s-content-sha256 for payload hash

[cbodley]

Curl_output_aws_sigv4() doesn't always have the whole payload in memory to generate a real payload hash. this commit allows the user to pass in a header like x-amz-content-sha256 to provide their desired payload hash

some services like s3 require this header, and may support other values like s3's UNSIGNED-PAYLOAD and STREAMING-AWS4-HMAC-SHA256-PAYLOAD with special semantics. servers use this header's value as the payload hash during signature validation, so it must match what the client uses to generate the signature

[cbodley]

will follow up with test cases if the approach looks good

[outscale-mgo]

will follow up with test cases if the approach looks good

It look good to me :)

[bagder]

Is this really a method a user would like to use? It seems a rather complicated way to get the checksum set for the content.

This method needs to get documented somewhere.

[bagder]

the length should be an int according to how *printf() works, but I think using the %.*s construct seems unnecessary here. Can't you just rather do %s and payload_hash_len ? payload_hash: "" as argument?

[cbodley]

Can't you just rather do %s and payload_hash_len ? payload_hash: "" as argument?

parse_content_sha_hdr() trims any trailing whitespace from the header value, so may return a shorter length

payload_hash points to memory returned by Curl_checkheaders(), so i used the explicit payload_hash_len here to avoid mutating that header to move the null terminator

i considered copying this header value into a different buffer to change the null. but that required the choice of a buffer size, so added an extra error path when the user provides a header value that's longer

i found this solution to be the simplest, but would welcome your guidance here

the length should be an int according to how *printf() works

ok, thanks

[cbodley]

@bagder thanks for the review!

Is this really a method a user would like to use? It seems a rather complicated way to get the checksum set for the content.

ultimately this is for services that already require this header. i agree that it's not user-friendly, so would like for Curl_output_aws_sigv4() to add this header itself when service=s3. we're still discussing the best way to handle this part in #8935. once we decide how to add that header, we'll still need this logic to a) allow the user to pass in itheir own checksum, and b) to make sure that any header values we generate match what we put in the signature - that's why i chose to break this piece out into its own PR

This method needs to get documented somewhere.

sure, is CURLOPT_AWS_SIGV4 the right place to mention this?

[cbodley]

test cases planned:

• signature without X-Xxx-Content-Sha256 matches signature with X-Xxx-Content-Sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 (hash of empty buffer)
• signature without X-Xxx-Content-Sha256 does not match signature with other values of X-Xxx-Content-Sha256
• leading/trailing whitespace in header value doesn't effect signature
• header value longer than 'char sha_hex[65]'

[cbodley]

updated with new test cases:

• 1956 adds the sha256 value corresponding to an empty buffer
• 1957 adds an arbitrary value and confirms that the signature differs from 1956
• 1958 adds whitespace to 1957 and confirms that the signature matches 1957
• 1959 adds a value longer than 'char sha_hex[65]' in Curl_output_aws_sigv4()

test cases planned:

• signature without X-Xxx-Content-Sha256 matches signature with X-Xxx-Content-Sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 (hash of empty buffer)

this part didn't quite work out, because the presence/absence of x-xxx-content-sha256 in SignedHeaders changes the signature

[bagder]

Thanks!