Boto Compatibility for gsutil

[danielgtaylor]

I am one of the maintainers of Boto. A few weeks ago we began to merge backward-compatible Python 3.3 and 3.4 support into our develop branch. Most modules now support Python 3 with passing unit and integration tests, and I am hoping to do a release soon. I am reaching out to you to ask if you would test our develop branch with gsutil.

Testing against Python 2.x will help to ensure that we don't accidentally break third party modules like gsutil. If you would like, you may also try out the Python 3 support and let us know any feedback. More information about the Python 3 support can be found in this boto-users post:

https://groups.google.com/forum/#!topic/boto-users/vcvoy8zrfP8

I appreciate any time that you can put into helping the Boto project with this.

This request is part of boto/boto#2437

[mfschwartz]

Hi Daniel,

I tried running the gsutil tests with boto updated to the most recent
develop branch commit. I found 1 test failure, which fails when computing
the etag for binary data - the end of the stack trace is:

...

File
"/usr/local/google/home/mfschwartz/gsutil_boto_py3/third_party/boto/tests/integration/s3/mock_storage_service.py",
line 199, in set_etag
m.update(self.data.encode('utf-8'))
UnicodeDecodeError: 'ascii' codec can't decode byte 0x8b in position 1:
ordinal not in range(128)

The breaking change to mock_storage_service.py" came in commit
boto/boto@96cd280,
with this change:
198c199

< m.update(self.data)

    m.update(self.data.encode('utf-8'))

We think this should be changed to:

if not isinstance(self.data, bytes):
m.update(self.data.encode('utf-8'))
else:
m.update(self.data)

This is needed because we should always be passing bytes to the update
function, and it only makes sense to encode the input data if it's string,
not a byte string.

I can commit this change if you like.

Thanks,

Mike

On Tue, Jul 22, 2014 at 3:09 PM, Daniel G. Taylor notifications@github.com
wrote:

I am one of the maintainers of Boto. A few weeks ago we began to merge
backward-compatible Python 3.3 and 3.4 support into our develop branch.
Most modules now support Python 3 with passing unit and integration tests,
and I am hoping to do a release soon. I am reaching out to you to ask if
you would test our develop branch with gsutil.

Testing against Python 2.x will help to ensure that we don't accidentally
break third party modules like gsutil. If you would like, you may also try
out the Python 3 support and let us know any feedback. More information
about the Python 3 support can be found in this boto-users post:

https://groups.google.com/forum/#!topic/boto-users/vcvoy8zrfP8

I appreciate any time that you can put into helping the Boto project with
this.

This request is part of boto/boto#2437
boto/boto#2437

—
Reply to this email directly or view it on GitHub
#214.

[danielgtaylor]

Thanks for pointing this out! I'll go ahead and add a test for it and then update the code. Sounds like everything else is working well. Thanks for taking the time to test! 👍

[mfschwartz]

Great - thanks for bringing boto forward to support Python 3.

Mike

On Wed, Jul 23, 2014 at 1:46 PM, Daniel G. Taylor notifications@github.com
wrote:

Thanks for pointing this out! I'll go ahead and add a unit test for it and
then update the code. Sounds like everything else is working well. Thanks
for taking the time to test! [image: 👍]

—
Reply to this email directly or view it on GitHub
#214 (comment)
.

[danielgtaylor]

The fix has been merged. Since everything else seems to pass, I think we can close this now. Thanks!