Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Boto Compatibility for gsutil #214

Closed
danielgtaylor opened this issue Jul 22, 2014 · 4 comments
Closed

Boto Compatibility for gsutil #214

danielgtaylor opened this issue Jul 22, 2014 · 4 comments

Comments

@danielgtaylor
Copy link

I am one of the maintainers of Boto. A few weeks ago we began to merge backward-compatible Python 3.3 and 3.4 support into our develop branch. Most modules now support Python 3 with passing unit and integration tests, and I am hoping to do a release soon. I am reaching out to you to ask if you would test our develop branch with gsutil.

Testing against Python 2.x will help to ensure that we don't accidentally break third party modules like gsutil. If you would like, you may also try out the Python 3 support and let us know any feedback. More information about the Python 3 support can be found in this boto-users post:

https://groups.google.com/forum/#!topic/boto-users/vcvoy8zrfP8

I appreciate any time that you can put into helping the Boto project with this.

This request is part of boto/boto#2437

@mfschwartz
Copy link
Contributor

Hi Daniel,

I tried running the gsutil tests with boto updated to the most recent
develop branch commit. I found 1 test failure, which fails when computing
the etag for binary data - the end of the stack trace is:

...

File
"/usr/local/google/home/mfschwartz/gsutil_boto_py3/third_party/boto/tests/integration/s3/mock_storage_service.py",
line 199, in set_etag
m.update(self.data.encode('utf-8'))
UnicodeDecodeError: 'ascii' codec can't decode byte 0x8b in position 1:
ordinal not in range(128)

The breaking change to mock_storage_service.py" came in commit
boto/boto@96cd280,
with this change:
198c199

< m.update(self.data)

    m.update(self.data.encode('utf-8'))

We think this should be changed to:

if not isinstance(self.data, bytes):
m.update(self.data.encode('utf-8'))
else:
m.update(self.data)

This is needed because we should always be passing bytes to the update
function, and it only makes sense to encode the input data if it's string,
not a byte string.

I can commit this change if you like.

Thanks,

Mike

On Tue, Jul 22, 2014 at 3:09 PM, Daniel G. Taylor notifications@github.com
wrote:

I am one of the maintainers of Boto. A few weeks ago we began to merge
backward-compatible Python 3.3 and 3.4 support into our develop branch.
Most modules now support Python 3 with passing unit and integration tests,
and I am hoping to do a release soon. I am reaching out to you to ask if
you would test our develop branch with gsutil.

Testing against Python 2.x will help to ensure that we don't accidentally
break third party modules like gsutil. If you would like, you may also try
out the Python 3 support and let us know any feedback. More information
about the Python 3 support can be found in this boto-users post:

https://groups.google.com/forum/#!topic/boto-users/vcvoy8zrfP8

I appreciate any time that you can put into helping the Boto project with
this.

This request is part of boto/boto#2437
boto/boto#2437


Reply to this email directly or view it on GitHub
#214.

@danielgtaylor
Copy link
Author

Thanks for pointing this out! I'll go ahead and add a test for it and then update the code. Sounds like everything else is working well. Thanks for taking the time to test! 👍

@mfschwartz
Copy link
Contributor

Great - thanks for bringing boto forward to support Python 3.

Mike

On Wed, Jul 23, 2014 at 1:46 PM, Daniel G. Taylor notifications@github.com
wrote:

Thanks for pointing this out! I'll go ahead and add a unit test for it and
then update the code. Sounds like everything else is working well. Thanks
for taking the time to test! [image: 👍]


Reply to this email directly or view it on GitHub
#214 (comment)
.

@danielgtaylor
Copy link
Author

The fix has been merged. Since everything else seems to pass, I think we can close this now. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants