Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix sign v4 breakage in Python 3 #427

Merged
merged 1 commit into from
Nov 28, 2016

Conversation

donatello
Copy link
Member

It was introduced in commit id 31e8a35 - once this was found (with git bisect), the fix was easy to figure out.

_get_bucket_region() was failing in Python 3 with signature mismatch
error.

Short program that demonstrates the issue:

from minio import Minio
from minio.error import ResponseError
from sys import stdout
s3Client = Minio(
    'localhost:9000',
    access_key='minio',
    secret_key='minio123',
    secure=False,
)

s3Client.trace_on(stdout)

objects = s3Client.list_objects('aditya', recursive=True)

for c, obj in enumerate(objects):
    print(obj)
    if c > 10:
        break

Generates:

Traceback (most recent call last):
  File "../minio-test.py", line 21, in <module>
    for c, obj in enumerate(objects):
  File "/home/aditya/Code/minio-code/python/venv/lib/python3.5/site-packages/minio-2.0.3-py3.5.egg/minio/api.py", line 912, in list_objects
  File "/home/aditya/Code/minio-code/python/venv/lib/python3.5/site-packages/minio-2.0.3-py3.5.egg/minio/api.py", line 1699, in _url_open
  File "/home/aditya/Code/minio-code/python/venv/lib/python3.5/site-packages/minio-2.0.3-py3.5.egg/minio/api.py", line 1645, in _get_bucket_region
  File "/home/aditya/Code/minio-code/python/venv/lib/python3.5/site-packages/minio-2.0.3-py3.5.egg/minio/api.py", line 1678, in _get_bucket_location
minio.error.ResponseError: ResponseError: code: SignatureDoesNotMatch, message: The request signature we calculated does not match the signature you provided. Check your key and signing method., bucket_name: None, object_name: None, request_id: RUSVQ0114N14O0ZG, host_id: 3L137, region: 

Copy link
Member

@krisis krisis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -198,7 +198,7 @@ def sign_v4(method, url, region, headers=None, access_key=None,

date = datetime.utcnow()
headers['X-Amz-Date'] = date.strftime("%Y%m%dT%H%M%SZ")
headers['X-Amz-Content-Sha256'] = content_sha256
headers['X-Amz-Content-Sha256'] = content_sha256.decode('ascii')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

content_sha256 can be "UNSIGNED_PAYLOAD" ".decode will fail in that scenario. Do it like this

if content_sha256 == _UNSIGNED_PAYLOAD: 
     headers['X-Amz-Content-Sha256'] = content_sha256
else:
     headers['X-Amz-Content-Sha256'] = content_sha256.decode('ascii')

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems pretty clumsy to keep content_sha256 as bytes in one case and as str in the other. I am making a slightly better change to handle this case.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is but it should be handled in a way UNSIGNED_PAYLOAD should still be a string, keeping it as byte causes presigned signature requests to fail.

@@ -209,7 +209,7 @@ def sign_v4(method, url, region, headers=None, access_key=None,
canonical_req = generate_canonical_request(method,
parsed_url,
headers_to_sign,
content_sha256)
content_sha256.decode('ascii'))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as above.

It was introduced in commit id
31e8a35

_get_bucket_region() was failing in Python 3 with signature mismatch
error.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants