Prefer io.BytesIO over six; available on all supported Pythons

[jdufresne]

On all supported Pythons, the io.BytesIO is always a stream implementation using an in-memory bytes buffer. Makes code slightly more forward compatible by reducing use of the six module.

[ansibot]

@jdufresne Greetings! Thanks for taking the time to open this pullrequest. In order for the community to handle your pullrequest effectively, we need a bit more information.

Here are the items we could not find in your description:

• issue type

Please set the description of this pullrequest with this template:
https://raw.githubusercontent.com/ansible/ansible/devel/.github/PULL_REQUEST_TEMPLATE.md

click here for bot help

[ansibot]

cc @Qalthos @ganeshrn @gdpak @justjais @rcarrillocruz @trishnaguha
click here for bot help

[webknjaz]

LGTM

[webknjaz]

@abadger https://docs.python.org/2.6/library/io.html#io.BytesIO

[abadger]

Note: io.BytesIO and six.BytesIO are different. Not sure yet if that's going to make a difference. Looking into it.

[abadger]

>>> import io
>>> io.BytesIO
<class 'io.BytesIO'>
>>> import six
>>> six.BytesIO
<class StringIO.StringIO at 0x7f4811b87d70>
>>> io.BytesIO(u'test').read()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: expecting a bytes object, got unicode
>>> six.BytesIO(u'test').read()
u'test'

However, on Python3, they should be the same (and both traceback when given a non-bytes string). We'll have to check that there's nothing here that is only used on python2 before switching.

[abadger]

Networking team needs to review this for my above question and then can merge it.

A quick look and I think:

• httpapi.py is okay to apply this patch
• test/units/plugins/httpapi/test_ftd.py is okay but probably should be rewritten:

- response_data = BytesIO(response_text.encode() if response_text else ''.encode())
+ response_data = BytesIO(to_bytes(response_text, errors='strict') if response_text else b'')

• network_cli.py: I'm not sure about this one. I think it hinges on what self._ssh_shell.recv(256) returns on both Python2 and Python3. If it always returns bytes (str type on python2 and bytes type on python3) then this is fine. if it sometimes returns bytes and sometimes returns text (unicode type on python2 or str type on python3) then it will fail sometimes and should be changed so that to_bytes() is called on data.

Note that in any case, this change is not wrong.... it's just that the code which uses it may be incorrect and would need to be fixed so that it doesn't cause an error with this new code.

[jdufresne]

>>> io.BytesIO(u'test').read()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: expecting a bytes object, got unicode

If code is passing Unicode strings, then I think BytesIO (whether from six or io) is the wrong choice. If the incoming argument is always Unicode, then io.StringIO should be used. If the incoming argument is always the native str type (a byte string on Python 2, a Unicode string on Python 3), then six.StringIO should be used.

[abadger]

@jdufresne As I said:

Note that in any case, this change is not wrong.... it's just that the code which uses it may be incorrect and would need to be fixed so that it doesn't cause an error with this new code.

[abadger]

Also, in our code base, six.StringIO should not be used with native string type, it should be used with the text type only (just as io.StringIO would be)

[Qalthos]

I took a trip through paramiko's source just to be sure, but I feel confident that _ssh_shell.recv() should only return bytes. Thanks for the catch!