Older Python builds are missing a required file on Big Sur

[rhettinger]

BPO	43393
Nosy	@rhettinger, @ronaldoussoren, @ned-deily, @kamilturek

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-03-04.02:16:26.059>
created_at = <Date 2021-03-03.22:17:02.782>
labels = ['OS-mac', 'type-crash']
title = 'Older Python builds are missing a required file on Big Sur'
updated_at = <Date 2021-03-04.02:16:26.058>
user = 'https://github.com/rhettinger'

bugs.python.org fields:

activity = <Date 2021-03-04.02:16:26.058>
actor = 'ned.deily'
assignee = 'none'
closed = True
closed_date = <Date 2021-03-04.02:16:26.059>
closer = 'ned.deily'
components = ['macOS']
creation = <Date 2021-03-03.22:17:02.782>
creator = 'rhettinger'
dependencies = []
files = []
hgrepos = []
issue_num = 43393
keywords = []
message_count = 2.0
messages = ['388062', '388068']
nosy_count = 4.0
nosy_names = ['rhettinger', 'ronaldoussoren', 'ned.deily', 'kamilturek']
pr_nums = []
priority = 'normal'
resolution = 'out of date'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue43393'
versions = []

[rhettinger]

When I upgraded to Big Sur, all my older builds stopped working.

Do we have any known workarounds or can we publish updated builds that work?

Like a lot of consultants, I still have to help clients maintain older code. Having a standard working build is essential.

$ python3.5
dyld: Library not loaded: /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
  Referenced from: /Library/Frameworks/Python.framework/Versions/3.5/Resources/Python.app/Contents/MacOS/Python
  Reason: image not found
Abort trap: 6

[ned-deily]

In summary, the Python binaries provided by python.org macOS installers for already end-of-life versions of Python, like 3.5 and earlier 3.x versions, do not run on macOS 11 Big Sur and attempting to build these end-of-life versions from source will still not produce a fully-functional Python on macOS 11.

For others reading this, per PEP-478, Python 3.5.4 was the last "bugfix" release of 3.5 back in August 2017 and, per long-standing release policy, binary installers for Windows and macOS are no longer manufactured once a version transitions from its "bugfix" to its "security-fix-only" phase. Further, Python 3.5 reached "end-of-life" in September 2020, five years after its initial release. So, technically, Python 3.5 is totally unsupported by the CPython project.

That said, I did take a quick look at this. A web search found a number of other similar reports dating back to the initial release of Big Sur last year. The error message is definitely not very helpful (the CoreFoundation framework remains an essential piece of macOS) and I still have not seen a total explanation of what is going on but I'm 99% sure that the problem is that python.org macOS installers for Python 3.5 were built for macOS 10.6+ and it is likely that there is some old cruft in the macOS CoreFoundation framework that Apple decided it no longer wanted to support as of macOS 11 (Apple did a lot of "housecleaning" in and leading up to Big Sur, one reason for the numbering change from 10.x to 11) and likely bumped a minimum supported version of CoreFoundation thus preventing executables that dynamically link to the older version to fail on launch, as in this case.

Note that the most recent python.org macOS installers for releases since 3.5 (including the final bugfix/binary releases of 3.7.x, 3.6.x, and 2.7.x) were built for at least macOS 10.9+ and do not exhibit this problem on macOS 11 Big Sur. So it's only an issue for 3.5 and earlier 3.x versions of the python.org installers, all of which have reached end-of-life.

I'm not aware of any way practical way to workaround this problem and make those installer binaries load on Big Sur. Clearly, the best longer-term solution is to upgrade to a supported version (like 3.9.x). But possible workarounds:

1. If 3.5.x is absolutely needed, there are third-party distributors that continue to support older versions of Python. In particular, the MacPorts project provides pre-built binaries of Python 3.5 (and other releases) that run on macOS 11 (and for older releases of macOS, too).
2. If the python.org 3.5 is absolutely needed, don't upgrade to Big Sur or run MacOS 10.15 (or earlier) in a virtual machine.
3. Build from source on Big Sur.

But keep in mind that, with any of those workarounds, there still is the fundamental issue that there were a number of changes in Big Sur that affect Python; there are build changes needed and, even when those are overcome, some things just don't work correctly on Big Sur without fixes; distutils and ctypes find_library of a system library (see bpo-41116), come to mind. At the moment, only Python 3.9.1 and later are fully supported on Big Sur.

Sorry I don't have a more positive answer.