http_aws_sigv4: Improve payload hashing

[matwey]

Hi,

I think the following could be useful for your PR curl#7966. Unfortunately, AWS SIGv4 implementation in curl is far from ideal currently.

Introduce X-Provider-Content-Sha256 header as it is required by the specs [1]
and we anyway need to know its value to calculate the CanonicalRequest.

Additionally:

1. Allow the user to provide precomputed SHA256 of the payload (when it is known in
   advance)

2. Fallback to "Unsigned payload" when postdata is not available instead of
   providing the hash for empty payload and having the wrong signature
   verification. postdata is only (can be) available for POST method and may be
   missed for non-empty payload.

   For instance, the following valid use-case will lead to PUT request with
   empty postdata:

   curl ... --aws-sigv4 "aws:amz" -u accesskey:secretkey -T file_to_upload.dat http://endpoint
   

[1] https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-header-based-auth.html

Signed-off-by: Matwey V. Kornilov matwey.kornilov@gmail.com

[outscale-mgo]

First thanks you for tackling X-Provider-Content-Sha256 problem, which for now still need to be manually computed.

Introduce X-Provider-Content-Sha256 header as it is required by the specs [1]

kind of, X-Provider-Content-Sha256 is require by s3 v4 signature spec, but not for other API sure as EC2 for instance.
So adding this header by force, seems dangerous, as you force to add this header on some API that doesn't require it.
So I guess we need an option to optionally add X-Provider-Content-Sha256 header.

[matwey]

Thanks. Now I see, you are right.

[matwey]

Put here for reference: minio/minio#5340

[matwey]

So I guess we need an option to optionally add X-Provider-Content-Sha256 header.

Actually, there is already such an option: when service is set to s3.

[matwey]

I've just double-checked. S3 rejects responses without X-Provider-Content-Sha256:

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>InvalidRequest</Code><Message>Missing required header for this request: x-amz-content-sha256</Message><RequestId>JK9VRX9RB76M58CR</RequestId><HostId>BtP3ov6WCpGYYf4772w9XEqW7TWGusRFn34r+MXD0/VjD/P8/4LsV50Lka7T1U4lU3PbhA+RaQI=</HostId></Error>⏎

Also, S3 seems to be not validating payload when the payload is empty. In this case X-Provider-Content-Sha256 should be consistent with the canonical request and may be arbitrary.