Deprecate the binhex module, binhex4 and hexbin4 standards

[vstinner]

BPO	39353
Nosy	@ronaldoussoren, @vstinner, @ned-deily, @vadmium, @serhiy-storchaka
PRs	bpo-39353: Deprecate the binhex module #18025 bpo-39353: binascii.crc_hqx() is no longer deprecated #18276

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-01-30.08:57:13.601>
created_at = <Date 2020-01-16.09:14:45.630>
labels = ['OS-mac', 'library', '3.9']
title = 'Deprecate the binhex module, binhex4 and hexbin4 standards'
updated_at = <Date 2021-09-01.22:32:32.050>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2021-09-01.22:32:32.050>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2020-01-30.08:57:13.601>
closer = 'vstinner'
components = ['Library (Lib)', 'macOS']
creation = <Date 2020-01-16.09:14:45.630>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 39353
keywords = ['patch']
message_count = 19.0
messages = ['360100', '360105', '360109', '360113', '360201', '360202', '360203', '360429', '360501', '360502', '360517', '360518', '360519', '360605', '360669', '361020', '361023', '361024', '361078']
nosy_count = 5.0
nosy_names = ['ronaldoussoren', 'vstinner', 'ned.deily', 'martin.panter', 'serhiy.storchaka']
pr_nums = ['18025', '18276']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue39353'
versions = ['Python 3.9']

[vstinner]

The binhex module encodes and decodes Apple Macintosh binhex4 data. It was originally developed for TRS-80. In the 1980s and early 1990s it was used on classic Mac OS 9 to encode binary email attachments.

Mac OS 9 is now heavily outdated, replaced by "macOS" (previously known as "Mac OS X").

I propose to emit a DeprecationWarning in the binhex module and document that it's deprecated.

I don't think that we have to schedule its removal yet, it can be decided later. A first deprecation warning emitted at runtime might help to warn last users, if there is any.

There are two binhex open issues:

• bpo-29566: no activity for almost 2 years (February 2017)
• bpo-34063: no activity for 1 year 1/2 (July 2018)

If we ignore global cleanup (done on the whole Python code base at once, not specific to binhex), the binhex was basically untouched since it has been ported to Python 3 (10 years ago). Maybe it means that it is very stable, which can be seen as a quality ;-)

I looked for "import binhex" in the first 10 pages of GitHub code search (restricted to Python programming language). I mostly found copies of Python test_binhex.py, no real usage of binhex.

On Stackoverflow, the latest questions about binhex has been asked in 2012:
https://stackoverflow.com/questions/12467973/binhex-decoding-using-java-code

I also found an answer suggesting to use binascii.a2b_hex() to decode a string the hexadecimal string "2020202020202020202020205635514d385a5856":
https://stackoverflow.com/questions/9683278/how-to-get-hard-disk-drivers-serial-number-in-python/9683837#9683837

But binascii.unhexlify() does the same than binascii.a2b_hex().

Attached PR deprecates binhex.

[serhiy-storchaka]

binhex provides an encoding different from binascii.a2b_hex() and binascii.unhexlify().

If deprecate the binhex module, we perhaps should deprecate also binascii.b2a_hqx(), binascii.a2b_hqx(), binascii.rlecode_hqx(), binascii.rledecode_hqx() and binascii.crc_hqx().

But it would be nice to create a separate package on PyPI that provides such functionality before removing it from the stdlib.

[vstinner]

But it would be nice to create a separate package on PyPI that provides such functionality before removing it from the stdlib.

This issue is *not* about removing the module :-) Only to mark it as deprecated.

If deprecate the binhex module, we perhaps should deprecate also binascii.b2a_hqx(), binascii.a2b_hqx(), binascii.rlecode_hqx(), binascii.rledecode_hqx() and binascii.crc_hqx().

I have no opinion on that. I mean: in case of doubt, I prefer to not deprecate them. I never used any of these functions.

[vstinner]

I also found an answer suggesting to use binascii.a2b_hex() to decode a string the hexadecimal string "2020202020202020202020205635514d385a5856":
https://stackoverflow.com/questions/9683278/how-to-get-hard-disk-drivers-serial-number-in-python/9683837#9683837

Oh wait, it is just unrelated :-) It's a typo in the StackOverflow answer: "import binhex; binascii.a2b_hex(...)". It should be "import binascii" obviously ;-) I didn't notice at the first read. binhex has no a2b_hex() function :-)

[vstinner]

If deprecate the binhex module, we perhaps should deprecate also binascii.b2a_hqx(), binascii.a2b_hqx(), binascii.rlecode_hqx(), binascii.rledecode_hqx() and binascii.crc_hqx().

Ok, let's do that. I updated the issue title and my PR.

[vstinner]

Serhiy Storchaka asked:
#18025 (comment)

"""
What is a benefit of deprecating it?

This module is a high-level interface to several functions in the binascii module. This is all macOS specific stuff, so you need to ask macOS experts. If this encoding no longer used in macOS, its support can be removed, with deprecating first. If there are no plans to remove, there is no reason to deprecate.
"""

The intent is to reduce the size of the standard library to reduce the maintenance burden of Python in general. The final goal is to remove the module. But that's a first step to check if there is still any user around. We should deprecate it for at least one cycle.

I understood that binhex4 and hexbin4 standards were used on Mac OS 9 but are no longer used on macOS, nor used anywhere else. See my first message.

Mac OS 9 is really outdated in 2020 and will likely be completly dead when Python 3.10 will be released (first Python release when we will be able to remove the module).

About the removal: I would like first write a PEP to propose a process to move old or unmaintained modules from the stdlib to PyPI. The "removal" would mean "move to PyPI" (but yeah, technically it would still be removed from the stdlib).

But I don't want to open a discuss on this hypothetical PEP *here* (I didn't write it yet :-D).

[vstinner]

BinHex format: https://en.wikipedia.org/wiki/BinHex

The question here is not only if binhex remains popular, but more generally if we want to continue to maintain it forever. Each module has a cost on the overall Python maintenance burden.

[ronaldoussoren]

I agree with deprecating binhex.

[vstinner]

New changeset beea26b by Victor Stinner in branch 'master':
bpo-39353: Deprecate the binhex module (GH-18025)
beea26b

[vstinner]

I agree with deprecating binhex.

Good :-) It's now deprecated.

[vadmium]

Is there a recommended replacement for calculating CRC-CCITT? Do it yourself in Python code, or use a particular external module?

[vstinner]

Is there a recommended replacement for calculating CRC-CCITT? Do it yourself in Python code, or use a particular external module?

What is your use case?

[vadmium]

Building and verifying the checksum in "RTA protocol" that uses this: <https://rms.nsw.gov.au/business-industry/partners-suppliers/documents/specifications/tsi-sp-003.pdf\>. But I understand CRC-CCITT is one of the two popular 16-bit CRC polynomials, used in many other places, according to <https://en.wikipedia.org/wiki/Cyclic_redundancy_check#Polynomial_representations_of_cyclic_redundancy_checks\>.

[vstinner]

Building and verifying the checksum in "RTA protocol" that uses this: (...)

Are you simply asking to not deprecate binascii.crc_hqx()?

[vadmium]

Of course I would prefer “crc_hqx” to stay, because we use it at work. But I understand if you think it is not popular enough to justify maintaining it.

But I was more asking if the deprecation notice should point the way forward. This function is no longer recommended, so what should users do instead? Or at least explain why it is deprecated.

In my case, I may eventually write or resurrect a simple Python CRC implementation that shifts one bit at a time. But other people may desire a faster C implementation.

[vstinner]

Of course I would prefer “crc_hqx” to stay, because we use it at work.

I propose to deprecate it because I understood that it was strictly related to binhex4 and hexbin4 protocols. If there is an use case outside these protocols, I'm fine with maintaining it.

The first motivation was to drop the binhex module in the long term, I'm less worried about specific binascii functions.

[vstinner]

New changeset c38fd0d by Victor Stinner in branch 'master':
bpo-39353: binascii.crc_hqx() is no longer deprecated (GH-18276)
c38fd0d

[vstinner]

binascii.crc_hqx() is no longer deprecated. I close again the issue.

[vadmium]

Thanks Victor