Compilation errors on macOS

[sandipshah]

BPO	46248
Nosy	@ronaldoussoren, @ned-deily

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2022-01-05.18:05:48.496>
created_at = <Date 2022-01-03.19:36:05.763>
labels = ['OS-mac', 'build', '3.11']
title = 'Compilation errors on macOS'
updated_at = <Date 2022-01-05.18:05:48.495>
user = 'https://bugs.python.org/sandipshah'

bugs.python.org fields:

activity = <Date 2022-01-05.18:05:48.495>
actor = 'ned.deily'
assignee = 'none'
closed = True
closed_date = <Date 2022-01-05.18:05:48.496>
closer = 'ned.deily'
components = ['macOS']
creation = <Date 2022-01-03.19:36:05.763>
creator = 'sandipshah'
dependencies = []
files = []
hgrepos = []
issue_num = 46248
keywords = []
message_count = 5.0
messages = ['409611', '409616', '409683', '409704', '409769']
nosy_count = 3.0
nosy_names = ['ronaldoussoren', 'ned.deily', 'sandipshah']
pr_nums = []
priority = 'normal'
resolution = 'third party'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'compile error'
url = 'https://bugs.python.org/issue46248'
versions = ['Python 3.11']

[sandipshah]

macOS Monterey 12.1
Python 3.10.1

Trying to compile libplist, libimobiledevice, libimobiledevice_glue generates the error
"Could not link test program to Python. Maybe the main Python library has been
installed in some non-standard library path. If so, pass it to configure,
via the LDFLAGS environment variable."

The fix is change the following lines (570-571) in /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/_sysconfigdata__darwin_darwin.py

from:
'LINKFORSHARED': '-Wl,-stack_size,1000000 -framework CoreFoundation '
'Python.framework/Versions/3.10/Python',
to:
'LINKFORSHARED': '-Wl,-stack_size,1000000 -framework CoreFoundation '
'/Library/Frameworks/Python.framework/Versions/3.10/Python',

Also, change the following line (110) in /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/config-3.10-darwin

from:
LINKFORSHARED= -Wl,-stack_size,1000000 -framework CoreFoundation $(PYTHONFRAMEWORKDIR)/Versions/$(VERSION)/$(PYTHONFRAMEWORK)
to:
LINKFORSHARED= -Wl,-stack_size,1000000 -framework CoreFoundation $(PYTHONFRAMEWORKINSTALLDIR)/Versions/$(VERSION)/$(PYTHONFRAMEWORK)

It seems the problem got created from - https://bugs.python.org/issue15298 - and most likely it exists in all the versions of Python (it exists in the native Python 2.7.18 that comes with macOS too.

Thanks.

[ned-deily]

Thanks for the report. Without more details, we can only guess at what you are trying to accomplish. I would assume it involves embedding Python in another library or application. If so, you should not be using LINKFORSHARED at all which is used to build Python itself (see, for example, bpo-36508). Moreover, when embedding on macOS, we do not recommend trying to use the -framework argument to compile with or link to Python: it's too easy to get wrong. Instead, as shown in the use the "Extending and Embedding the Python Interpreter" section of the docset, use the output from the appropriate python3.x-config command, for example:

$ python3.10-config --cflags
-I/Library/Frameworks/Python.framework/Versions/3.10.1_11/include/python3.10 -I/Library/Frameworks/Python.framework/Versions/3.10.1_11/include/python3.10 -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch arm64 -arch x86_64 -g

$ python3.10-config --ldflags
-L/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/config-3.10-darwin -ldl -framework CoreFoundation

https://docs.python.org/3/extending/embedding.html#compiling-and-linking-under-unix-like-systems

[ronaldoussoren]

Part of the problem here is that most keys in the result of sysconfig.get_config_vars() are copied as-is from the CPython Makefile and pyconfig.h header file.

The keys copied from the Makefile are in a lot of cases not useful for users of python because they contain values that are only useful while building CPython itself.

Without knowing your use case it is hard to point you to a better solution, but as Ned writes in general just follow the instructions from the extending and embedding documentation.

[sandipshah]

Hi,

Like I mentioned in the ticket, I am trying to compile libplist,
libimobiledevice , and libimobiledevice-glue-1.0 on my mac. These are
needed to do any development on a mobile device (specifically for testing /
automation). I use Python for a lot of developmental work, and hence would
rather stick to the same framework to work on mobile devices too.

https://github.com/libimobiledevice

The project is not to "embed" Python, but to link to it, so that it can be
used for testing the app on the mobile (the tools rely on Cython).

The three tools have their own "autogen.sh" which fails with the error
mentioned in the ticket.

While researching, I had found this solution on stackexchange -
https://stackoverflow.com/questions/6490513/vim-failing-to-compile-with-python-on-os-x

Which led me to this after looking more in the python distribution files:

---
In file
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/sysconfig.py
On line 410 - fn _generate_posix_vars

439-448

There's a chicken-and-egg situation on OS X with regards to the

_sysconfigdata module after the changes introduced by bpo-15298:

get_config_vars() is called by get_platform() as part of the

make pybuilddir.txt target -- which is a precursor to the

_sysconfigdata.py module being constructed. Unfortunately,

get_config_vars() eventually calls _init_posix(), which attempts

to import _sysconfigdata, which we won't have built yet. In order

for _init_posix() to work, if we're on Darwin, just mock up the

_sysconfigdata module manually and populate it with the build vars.

This is more than sufficient for ensuring the subsequent call to

---

Let me know if you need any other information.

Thanks,

Sandip

On Tue, Jan 4, 2022 at 6:30 AM Ronald Oussoren <report@bugs.python.org>
wrote:

Ronald Oussoren <ronaldoussoren@mac.com> added the comment:

Part of the problem here is that most keys in the result of
sysconfig.get_config_vars() are copied as-is from the CPython Makefile and
pyconfig.h header file.

The keys copied from the Makefile are in a lot of cases not useful for
users of python because they contain values that are only useful while
building CPython itself.

Without knowing your use case it is hard to point you to a better
solution, but as Ned writes in general just follow the instructions from
the extending and embedding documentation.

----------

---

Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue46248\>

---

[ronaldoussoren]

The libraries you mention are 3th party projects. Please check with the project first.

I've look at the GitHub repository for libmobiledevice and have no idea how they determine the compiler flags for Python. As mentioned in an earlier version "python-config" is the documented way to fetch such flags.