aws-sigv4 not correctly signing requests with empty URL query params

[hjmallon]

I did this

URL="https://${BUCKET}.s3.${REGION}.amazonaws.com/?versioning"
curl -v \
     --aws-sigv4 "aws:amz:${REGION}:s3" \
     --get "${URL}" \
     --user "${aws_access_key_id}:${aws_secret_access_key}" \
     --header "X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" \
     --header "X-Amz-Security-Token: ${aws_session_token}"

I expected the following

With the proper setup of BUCKET, REGION, aws_access_key_id, aws_secret_access_key and aws_session_token I expect 200 OK and data. See https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketVersioning.html

However I got 403 Forbidden and an error returned SignatureDoesNotMatch.

I have tracked it down a bit further and it it related to the ?versioning parameter. If I set it to ?versioning= then it works. AWS CLI seems to be doing that automatically, expanding versioning into versioning=.

So this works

URL="https://${BUCKET}.s3.${REGION}.amazonaws.com/?versioning="
curl -v \
     --aws-sigv4 "aws:amz:${REGION}:s3" \
     --get "${URL}" \
     --user "${aws_access_key_id}:${aws_secret_access_key}" \
     --header "X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" \
     --header "X-Amz-Security-Token: ${aws_session_token}"

curl/libcurl version

curl 7.87.0-DEV (x86_64-apple-darwin22.2.0) libcurl/7.87.0-DEV (SecureTransport) OpenSSL/1.1.1s zlib/1.2.11 brotli/1.0.9 zstd/1.5.2 libidn2/2.3.4 libssh2/1.10.0 nghttp2/1.51.0 librtmp/2.3
Release-Date: [unreleased]
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz MultiSSL NTLM NTLM_WB SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd

operating system

Darwin hostname 22.2.0 Darwin Kernel Version 22.2.0: Fri Nov 11 02:08:47 PST 2022; root:xnu-8792.61.2~4/RELEASE_X86_64 x86_64

[bagder]

curl uses the query string you use in the URL. Isn't this then just you using the wrong URL?

[hjmallon]

I’m not an expert on this by any means and I would be happy to close this as ‘just how it is’. Using the ‘?versioning’ URL without the = sign should work, but seems to require that key=value normalisation in the authentication. Using the query string without the equals appears to match the AWS docs too as far as I can tell.

Since the work around is so simple this doesn’t stop me doing anything I want to do with curl.

[bagder]

I don't think curl should correct or modify the URL. It is meant to work with exactly what is provided.

But if the sigv4-code can act is if there is a trailing = (even if there isn't) when it generates the signature and that works, then I think that is probably the right direction.

[matchbn]

Hello, Same issue for me, i'm able to get by adding the "=", but i cannot PUT versioning SignatureDoesNotMatch .
@hjmallon are you able to PUT something?