You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When using sigv4, HTTP requests to S3 are incorrectly signed when non-ascii characters are present in the query string in Python 2. When sigv4 auth is added to a HTTP request for S3, the auth path and parameters are mangled. The auth path's query string is removed, parsed, and placed into the request parameters. The updated requests parameters are then used for part of the signature calculations. Parsing of the query string is done using urllib.parse.parse_qs. In Python 2, parse_qs does not decode URL-encoded %HH escapes. Thus, it outputs wrongly if given a Unicode query string with URL-encoded non-ascii characters.
>>>urllib.parse.parse_qs(u'prefix=El%20Ni%C3%B1o')
{u'prefix': [u'El Ni\xc3\xb1o']}
>>>u'El Ni\xc3\xb1o'=='El Ni\xc3\xb1o'.decode('utf-8')
False# parse_qs did not decode to utf-8 in Python 2>>>'El Ni\xc3\xb1o'.decode('utf-8') ==u'El Ni\xf1o'True
In the below log snippet, you can see this in action when the request path does not agree with the CanonicalRequest parameters:
File "/usr/lib/python2.6/site-packages/boto-2.34.0-py2.6.egg/boto/s3/bucket.py", line 475, in get_all_keys
'', headers, **params)
File "/usr/lib/python2.6/site-packages/boto-2.34.0-py2.6.egg/boto/s3/bucket.py", line 413, in _get_all
response.status, response.reason, body)
S3ResponseError: S3ResponseError: 403 Forbidden
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>
...
boto: DEBUG: Method: GET
boto: DEBUG: Path: /?max-keys=1&prefix=El%20Ni%C3%B1o/
boto: DEBUG: Data:
boto: DEBUG: Headers: {}
...
boto: DEBUG: CanonicalRequest:
GET
/
max-keys=1&prefix=El%20Ni%C3%83%C2%B1o%2F
...
The text was updated successfully, but these errors were encountered:
When using sigv4, HTTP requests to S3 are incorrectly signed when non-ascii characters are present in the query string in Python 2. When sigv4 auth is added to a HTTP request for S3, the auth path and parameters are mangled. The auth path's query string is removed, parsed, and placed into the request parameters. The updated requests parameters are then used for part of the signature calculations. Parsing of the query string is done using
urllib.parse.parse_qs
. In Python 2, parse_qs does not decode URL-encoded %HH escapes. Thus, it outputs wrongly if given a Unicode query string with URL-encoded non-ascii characters.In the below log snippet, you can see this in action when the request path does not agree with the CanonicalRequest parameters:
HTTP request query string: max-keys=1&prefix=El%20Ni%C3%B1o/
CanonicalRequest Params: max-keys=1&prefix=El%20Ni%C3%83%C2%B1o%2F
The text was updated successfully, but these errors were encountered: