find_library on macOS Big Sur

[sumanthratna]

BPO	41179
Nosy	@ronaldoussoren, @ned-deily, @sumanthratna
PRs	[WIP] bpo-41179: Fix ctypes.util.find_library on macOS Big Sur #21250
Superseder	bpo-41100: Support macOS 11 and Apple Silicon Macs

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-07-01.16:27:01.548>
created_at = <Date 2020-07-01.02:10:57.715>
labels = ['OS-mac', 'type-bug', '3.8', '3.9', '3.10', 'ctypes', '3.7']
title = 'find_library on macOS Big Sur'
updated_at = <Date 2020-07-01.16:27:01.547>
user = 'https://github.com/sumanthratna'

bugs.python.org fields:

activity = <Date 2020-07-01.16:27:01.547>
actor = 'ned.deily'
assignee = 'none'
closed = True
closed_date = <Date 2020-07-01.16:27:01.548>
closer = 'ned.deily'
components = ['macOS', 'ctypes']
creation = <Date 2020-07-01.02:10:57.715>
creator = 'sumanthratna'
dependencies = []
files = []
hgrepos = []
issue_num = 41179
keywords = ['patch']
message_count = 3.0
messages = ['372728', '372734', '372754']
nosy_count = 4.0
nosy_names = ['ronaldoussoren', 'ned.deily', 'python-dev', 'sumanthratna']
pr_nums = ['21250']
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '41100'
type = 'behavior'
url = 'https://bugs.python.org/issue41179'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8', 'Python 3.9', 'Python 3.10']

[sumanthratna]

The following all return None on macOS Big Sur, but return a valid path (str) on macOS Catalina:

python3.6 -c "from ctypes import util; print(util.find_library('objc'))"
python3.7 -c "from ctypes import util; print(util.find_library('objc'))"
python3.8 -c "from ctypes import util; print(util.find_library('objc'))"

The solution I'm thinking of is using platform.mac_ver() in ctypes.util.find_library to see if the macOS version is either 10.16 or 11.0; if so, use an alternative method to locate the path of the library somehow—else, use the current functionality. I'm hoping that looking in /System/Library/Frameworks/ is enough.

From the macOS Big Sur release notes (https://developer.apple.com/documentation/macos-release-notes/macos-big-sur-11-beta-release-notes):

New in macOS Big Sur 11 beta, the system ships with a built-in dynamic linker cache of all system-provided libraries. As part of this change, copies of dynamic libraries are no longer present on the filesystem. Code that attempts to check for dynamic library presence by looking for a file at a path or enumerating a directory will fail. Instead, check for library presence by attempting to dlopen() the path, which will correctly check for the library in the cache. (62986286)

Related links:

• https://bugs.python.org/issue41116 (this is a build issue, but is related I think)
• https://stackoverflow.com/questions/62587131/macos-big-sur-python-ctypes-find-library-does-not-find-libraries-ssl-corefou
• https://www.reddit.com/r/MacOSBeta/comments/hfknpa/is_corefoundation_missing_for_everyone_on_big_sur/
• Unable to import on macOS Big Sur vispy/vispy#1885
• OpenGL on macOS Big Sur napari/napari#1393
• MacOS Big Sur dlsym(RTLD_DEFAULT, kIOMasterPortDefault): symbol not found (ESPTOOL-82) espressif/esptool#540

This is my first issue in Python; sorry in advance if I've made a mistake anywhere.

[ronaldoussoren]

We're tracking macOS 11 changes in bpo-41100. That issue has a PRs from Apple for, amongst others, this issue.