ctypes.util.find_library not working on macOS

[ihaveamac]

BPO	33281
Nosy	@amauryfa, @abalkin, @vstinner, @ned-deily, @meadori, @mingwandroid, @aixtools, @ihaveamac
PRs	bpo-33281: Change some ifs to be elifs so the correct find_library is not later overridden. #6625 [3.7] bpo-33281: Fix ctypes.util.find_library regression on macOS (GH-6625) #6680 bpo-33281: NEWS and ACK #6681 [3.7] bpo-33281: NEWS and ACK (GH-6681) #6682

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-05-02.02:56:57.992>
created_at = <Date 2018-04-15.05:24:20.526>
labels = ['3.8', 'ctypes', 'type-bug', '3.7']
title = 'ctypes.util.find_library not working on macOS'
updated_at = <Date 2018-05-02.02:56:57.989>
user = 'https://github.com/ihaveamac'

bugs.python.org fields:

activity = <Date 2018-05-02.02:56:57.989>
actor = 'ned.deily'
assignee = 'none'
closed = True
closed_date = <Date 2018-05-02.02:56:57.992>
closer = 'ned.deily'
components = ['ctypes']
creation = <Date 2018-04-15.05:24:20.526>
creator = 'Ian Burgwin'
dependencies = []
files = []
hgrepos = []
issue_num = 33281
keywords = ['patch']
message_count = 8.0
messages = ['315309', '315310', '315742', '316035', '316037', '316038', '316039', '316040']
nosy_count = 8.0
nosy_names = ['amaury.forgeotdarc', 'belopolsky', 'vstinner', 'ned.deily', 'meador.inge', 'Ray.Donnelly', 'Michael.Felt', 'Ian Burgwin']
pr_nums = ['6625', '6680', '6681', '6682']
priority = None
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue33281'
versions = ['Python 3.7', 'Python 3.8']

[ihaveamac]

On Python 3.7.0a4 and later (including 3.7.0b4), find_library currently always returns None on macOS. It works on 3.7.0a3 and earlier. Tested on macOS 10.11 and 10.13.

Expected result: Tested on 3.6.5, 3.7.0a1 and 3.7.0a3:

>>> from ctypes.util import find_library
>>> find_library('iconv')
'/usr/lib/libiconv.dylib'
>>> find_library('c')
'/usr/lib/libc.dylib'
>>>

Current output on 3.7.0a4 to 3.7.0b3:

>> from ctypes.util import find_library
>> find_library('iconv')
>> find_library('c')
>>

[ned-deily]

Thanks for the report and the good detective work! I see the same results. It appears that the error was introduced by c5ae169 for bpo-26439. It looks we need both a fix and a test for this. This really needs to be fixed for 3.7.0b4.

[aixtools]

On 15/04/2018 07:56, Ned Deily wrote:

Ned Deily <nad@python.org> added the comment:

Thanks for the report and the good detective work! I see the same results. It appears that the error was introduced by c5ae169 for bpo-26439. It looks we need both a fix and a test for this. This really needs to be fixed for 3.7.0b4.

Perhaps he can give a bit more info. I do not understand how this could
break things, as the darwin code is earlier in the queue.

if os.name == "posix" and sys.platform == "darwin":
     from ctypes.macholib.dyld import dyld_find as _dyld_find
     def find_library(name):
         possible = ['lib%s.dylib' % name,
                     '%s.dylib' % name,
                     '%s.framework/%s' % (name, name)]
         for name in possible:
             try:
                 return _dyld_find(name)
             except ValueError:
                 continue
         return None

if sys.platform.startswith("aix"):
     # AIX has two styles of storing shared libraries
     # GNU auto_tools refer to these as svr4 and aix
     # svr4 (System V Release 4) is a regular file, often with .so as suffix
     # AIX style uses an archive (suffix .a) with members (e.g., shr.o, 
libssl.so)
     # see issue#26439 and _aix.py for more details

    from ctypes._aix import find_library

elif os.name == "posix":
    # Andreas Degert's find functions, using gcc, /sbin/ldconfig, objdump
    import re, tempfile

    def _findLib_gcc(name):

As I understand the code above (and maybe I am wrong) - the code should
be calling the unchanged routines in the macholib subdirectory. A simple
test on macOS would be to comment out the two lines

if sys.platform.startswith("aix"):
     from ctypes._aix import find_library

That will "delete" the AIX find_library logic.

In other words - can someone check whether the unchanged
Lib/ctypes/macholib/* files are being used? And if not, then something
surprising is preventing that. If they are - no idea how the macholib
code got effected by this. The goal was to have all changes in the
_aix.py file, rather than in util.py.

----------
nosy: +Michael.Felt, ned.deily, vstinner
priority: normal -> release blocker
stage: -> needs patch

---

Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue33281\>

---

[ned-deily]

New changeset d06d345 by Ned Deily (Ray Donnelly) in branch 'master':
bpo-33281: Fix ctypes.util.find_library regression on macOS (GH-6625)
d06d345

[ned-deily]

New changeset 69a013e by Ned Deily in branch 'master':
bpo-33281: NEWS and ACK (GH-6681)
69a013e

[ned-deily]

New changeset c74ca53 by Ned Deily (Miss Islington (bot)) in branch '3.7':
bpo-33281: Fix ctypes.util.find_library regression on macOS (GH-6625) (GH-6680)
c74ca53

[ned-deily]

New changeset d74f353 by Ned Deily (Miss Islington (bot)) in branch '3.7':
bpo-33281: NEWS and ACK (GH-6681) (GH-6682)
d74f353

[ned-deily]

Thanks for the PR, Ray. Merged for 3.7.0b4 (and 3.8.0).