Add Base32 support for RFC4648 "Extended Hex" alphabet (patch attached)

[matthaeuswander]

BPO	16995
Nosy	@pitrou, @ezio-melotti, @vadmium, @serhiy-storchaka, @pganssle, @csabella, @miss-islington, @FFY00
PRs	bpo-16995: add support for base32 extended hex (base32hex) #20441
Files	base32hex.tar.gz: Patches for Python 2.7 and 3 py3_base32hex.patch: Patch for Python 3 py3_base32hex.patch: Patch for 3.5 or 3.6 py36_base32hex.patch: fixed Patch for 3.6 py36_base32hex.patch: Regenerated for review

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-08-10.19:00:47.787>
created_at = <Date 2013-01-18.18:21:19.913>
labels = ['type-feature', 'library', '3.10']
title = 'Add Base32 support for RFC4648 "Extended Hex" alphabet (patch attached)'
updated_at = <Date 2020-08-10.19:00:47.785>
user = 'https://bugs.python.org/matthaeuswander'

bugs.python.org fields:

activity = <Date 2020-08-10.19:00:47.785>
actor = 'p-ganssle'
assignee = 'none'
closed = True
closed_date = <Date 2020-08-10.19:00:47.787>
closer = 'p-ganssle'
components = ['Library (Lib)']
creation = <Date 2013-01-18.18:21:19.913>
creator = 'matthaeus.wander'
dependencies = []
files = ['28771', '28772', '39981', '39995', '39996']
hgrepos = []
issue_num = 16995
keywords = ['patch']
message_count = 12.0
messages = ['180206', '180207', '180210', '243591', '247134', '247158', '247205', '247216', '338962', '375123', '375137', '375138']
nosy_count = 9.0
nosy_names = ['pitrou', 'ezio.melotti', 'martin.panter', 'serhiy.storchaka', 'matthaeus.wander', 'p-ganssle', 'cheryl.sabella', 'miss-islington', 'FFY00']
pr_nums = ['20441']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue16995'
versions = ['Python 3.10']

Linked PRs

• gh-61199: Remove superfluous global statements from base64._b32{en,de}code()* #111785

[matthaeuswander]

RFC4648 specifies two different alphabets for Base32: 1) the common one already used in RFC3548 and implemented in Python, and 2) an "Extended Hex" alphabet which is used by e.g. DNSSEC NSEC3 but not yet implemented in Python.

The attached patches add the "Extended Hex" alphabet to Python 2.7 and 3 via an optional parameter 'base32hex=False'. Default behavior is not changed.

patch Lib/base64.py < py27_base64.patch
patch Lib/test/test_base64.py < py27_test_base64.patch

[ezio-melotti]

Since this is a new feature it can go on 3.4 only.
It would also be better to include both the changes for base64.py and test_base64.py in a single patch and attach that, rather than uploading an archive that contains both.

FTR the links to the relevant RFCs are:
http://tools.ietf.org/html/rfc3548
http://tools.ietf.org/html/rfc4648

[matthaeuswander]

Added single patch as requested.

[serhiy-storchaka]

The patch can't be longer applied cleanly. It should be synchronized with sources. It is the last chance for this feature to get to 3.5 if somebody will update the patch and add the documentation.

The feature itself LGTM. The only my doubt is about the name of boolean parameter. "base32hex" is recommended as encoding name. Parameter can be named as "exthex" or "extended_hex". Or may be add separate functions b32hexencode/b32hexdecode. I'm not sure.

[matthaeuswander]

I've created a new patch that works against the current 3.5 sources. Should be fine for 3.6, I guess.

Separate functions b32hexencode and b32hexdecode are used now. There is no optional parameter "base32hex" anymore.

[vadmium]

Matthäus, I think you uploaded the old patch again by accident. Also, see Berker’s old comments about documentation, and my new suggestions, on the Rietveld code review, if you haven’t already.

[matthaeuswander]

*facepalm* Yes, I uploaded the old patch twice. Sorry for that.

• Added doc update.
• Added test case that includes all 32 characters codes.

I'm reusing the existing Base32 table generation logic without changes. It has been changed (in 3.4 or so) since the first patch. If the logic is still to be changed as suggested by Martin in Rietveld review, I'd suggest to open a separate issue because it affects the existing Base32 implementation, too.

[serhiy-storchaka]

Added comments on Rietveld.

[csabella]

Is there interest in having this patch converted to a pull request?

[miss-islington]

New changeset 4ce6faa by Filipe Laíns in branch 'master':
bpo-16995: add support for base32 extended hex (base32hex) (GH-20441)
4ce6faa

[FFY00]

Paul, the PR is now merged :). Can you close the bug?

[pganssle]

Thanks Filipe!