binascii.a2b_* functions could accept unicode strings

[pitrou]

BPO	13637
Nosy	@pitrou, @vstinner, @bitdancer, @berkerpeksag, @vadmium, @vajrasky
Files	binasciistr.patch fix_doc_binascii_unhexlify.patch better_error_message_binascii.patch fix_doc_binascii_unhexlify.v2.patch fix_doc_binascii_unhexlify.v3.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2015-02-14.22:33:10.215>
created_at = <Date 2011-12-19.16:30:44.388>
labels = ['type-bug', 'library', 'docs']
title = 'binascii.a2b_* functions could accept unicode strings'
updated_at = <Date 2015-02-22.00:17:47.705>
user = 'https://github.com/pitrou'

bugs.python.org fields:

activity = <Date 2015-02-22.00:17:47.705>
actor = 'r.david.murray'
assignee = 'docs@python'
closed = True
closed_date = <Date 2015-02-14.22:33:10.215>
closer = 'berker.peksag'
components = ['Documentation', 'Library (Lib)']
creation = <Date 2011-12-19.16:30:44.388>
creator = 'pitrou'
dependencies = []
files = ['24060', '32461', '32463', '38038', '38132']
hgrepos = []
issue_num = 13637
keywords = ['patch']
message_count = 20.0
messages = ['149876', '149905', '149911', '149913', '153829', '154265', '154964', '155001', '201963', '201972', '202181', '235545', '235931', '235934', '235996', '236000', '236001', '236373', '236390', '236395']
nosy_count = 10.0
nosy_names = ['pitrou', 'vstinner', 'luke-jr', 'Arfrever', 'r.david.murray', 'docs@python', 'python-dev', 'berker.peksag', 'martin.panter', 'vajrasky']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue13637'
versions = ['Python 3.3']

[pitrou]

a2b_hex and friends accept only byte strings:

>>> binascii.a2b_hex(b'00')
b'\x00'
>>> binascii.a2b_hex('00')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'str' does not support the buffer interface

But they could just as well accept ASCII-only unicode strings. Also, with PEP-393, accessing the 8-bit ASCII data doesn't even need a conversion.

[pitrou]

Here is a patch.

[python-dev]

New changeset eb8d62706d5f by Antoine Pitrou in branch 'default':
Issue bpo-13637: "a2b" functions in the binascii module now accept ASCII-only unicode strings.
http://hg.python.org/cpython/rev/eb8d62706d5f

[pitrou]

Committed now.

[bitdancer]

I disagree with this feature. Reopening pending discussion on python-dev.

[bitdancer]

Discussion resolved in favor of patch.

[luke-jr]

Has this been fixed in 3.2 yet? Somehow it seems to have been "reclassified" as an enhancement when it's really a regression. str worked fine in these functions in 3.1.

[Arfrever]

I confirm that it works in Python 3.1 and doesn't work in Python 3.2.

[vajrasky]

Antoine, I think you forgot to update the doc.

http://docs.python.org/3.4/library/binascii.html#binascii.unhexlify

Changed in version 3.2: Accept only bytestring or bytearray objects as input.

Attached the patch to update the doc to reflect the changes in this ticket.

[vajrasky]

Here is another patch to better the error message. Right now, the error message when wrong input sent to unhexlify is a little bit funny.

>>> import binascii
>>> binascii.unhexlify(3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: argument should be bytes, buffer or ASCII string, not <class 'int'>

After patch:
>>> import binascii
>>> binascii.unhexlify(3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: argument should be bytes, buffer or ASCII string, not int

[vadmium]

The a2b_qp() function also documents a byte string restriction for 3.2, and now 3.3 also seems to support ASCII-compatible text strings. Maybe the documentation should reflect this also?

[vadmium]

The error fix patch looks good.

I updated the doc patch as fix_doc_binascii_unhexlify.v2.patch, which also clarifies a2b_qp(), and restores the description of Python 3.2 not allowing text strings.

[berkerpeksag]

The binascii documentation already says

a2b_* functions accept Unicode strings containing only ASCII characters.
[...]

.. versionchanged:: 3.3
ASCII-only unicode strings are now accepted by the a2b_* functions.

I think we can just remove versionchanged directives in fix_doc_binascii_unhexlify.v2.patch.

[vadmium]

That’s another option. Someone might wonder why a2b_hex() and a2b_qp() actually accept text strings in Python 3.1 when the documentation says you need 3.3, but I guess that’s not such a big deal. Posting patch v3 which removes the 3.2 notices.

[python-dev]

New changeset 8d32453dd0f7 by Berker Peksag in branch '3.4':
Issue bpo-13637: Remove outdated versionchanged directives.
https://hg.python.org/cpython/rev/8d32453dd0f7

New changeset d3ca674cf716 by Berker Peksag in branch 'default':
Issue bpo-13637: Remove outdated versionchanged directives.
https://hg.python.org/cpython/rev/d3ca674cf716

[python-dev]

New changeset ad4a8176a71a by Berker Peksag in branch '3.4':
Issue bpo-13637: Improve exception message of a2b_* functions.
https://hg.python.org/cpython/rev/ad4a8176a71a

New changeset 55f5e960cc40 by Berker Peksag in branch 'default':
Issue bpo-13637: Improve exception message of a2b_* functions.
https://hg.python.org/cpython/rev/55f5e960cc40

[berkerpeksag]

Thank you to both Vajrasky and Martin.

[bitdancer]

I think this doc change was incorrect. The current document is supposed to provide the correct historical information. So "changed in 3.3: accept ASCII input" (and just ignore the fact that 3.1 also accepted ascii, since we pretty much prefer to ignore the existence of 3.0 and 3.1 :) would be the better choice, IMO.

[vadmium]

If I understand what you are saying, you don’t mind not mentioning 3.1 behaviour, but want to mention 3.2 behaviour. Perhaps the existing “Note” box at the top of the page is good enough for you, as pointed out in <http://bugs.python.org/issue13637#msg235931\>.

[bitdancer]

Ah, yes, I see. Sorry for the confusion, I misread that part of the discussion and did not look at that part of the docs.

(I find that note confusing...it seems to imply that a2b only accepts ascii. But that's a different issue and I don't feel strongly enough about it to open one :)