Python 2.7.18 install fails on Mac OS X Sonoma (14.5)

[laulaz]

Python 2.7.18 install fails on Mac OS X Sonoma (14.5) with traceback :

Traceback (most recent call last):
  File "get-pip.py", line 23974, in <module>
    main()
  File "get-pip.py", line 199, in main
    bootstrap(tmpdir=tmpdir)
  File "get-pip.py", line 82, in bootstrap
    from pip._internal.cli.main import main as pip_entry_point
  File "/var/folders/x2/49j6tpl10f9bsqh4dsw2hf040000gn/T/tmplOcDu7/pip.zip/pip/_internal/cli/main.py", line 10, in <module>
  File "/var/folders/x2/49j6tpl10f9bsqh4dsw2hf040000gn/T/tmplOcDu7/pip.zip/pip/_internal/cli/autocompletion.py", line 9, in <module>
  File "/var/folders/x2/49j6tpl10f9bsqh4dsw2hf040000gn/T/tmplOcDu7/pip.zip/pip/_internal/cli/main_parser.py", line 7, in <module>
  File "/var/folders/x2/49j6tpl10f9bsqh4dsw2hf040000gn/T/tmplOcDu7/pip.zip/pip/_internal/cli/cmdoptions.py", line 25, in <module>
  File "/var/folders/x2/49j6tpl10f9bsqh4dsw2hf040000gn/T/tmplOcDu7/pip.zip/pip/_internal/cli/progress_bars.py", line 12, in <module>
  File "/var/folders/x2/49j6tpl10f9bsqh4dsw2hf040000gn/T/tmplOcDu7/pip.zip/pip/_internal/utils/logging.py", line 18, in <module>
  File "/var/folders/x2/49j6tpl10f9bsqh4dsw2hf040000gn/T/tmplOcDu7/pip.zip/pip/_internal/utils/misc.py", line 21, in <module>
  File "/var/folders/x2/49j6tpl10f9bsqh4dsw2hf040000gn/T/tmplOcDu7/pip.zip/pip/_vendor/pkg_resources/__init__.py", line 36, in <module>
  File "/Users/laurent/.pyenv/versions/2.7.18/lib/python2.7/email/parser.py", line 12, in <module>
    from email.feedparser import FeedParser
  File "/Users/laurent/.pyenv/versions/2.7.18/lib/python2.7/email/feedparser.py", line 27, in <module>
    from email import message
  File "/Users/laurent/.pyenv/versions/2.7.18/lib/python2.7/email/message.py", line 16, in <module>
    import email.charset
  File "/Users/laurent/.pyenv/versions/2.7.18/lib/python2.7/email/charset.py", line 13, in <module>
    import email.base64mime
  File "/Users/laurent/.pyenv/versions/2.7.18/lib/python2.7/email/base64mime.py", line 40, in <module>
    from email.utils import fix_eols
  File "/Users/laurent/.pyenv/versions/2.7.18/lib/python2.7/email/utils.py", line 29, in <module>
    import urllib
  File "/Users/laurent/.pyenv/versions/2.7.18/lib/python2.7/urllib.py", line 1448, in <module>
    from _scproxy import _get_proxy_settings, _get_proxies
ImportError: No module named _scproxy

trace.log file is here : https://gist.github.com/laulaz/58b3eb618607a92c082f288a37573ce7

Thank you for your help 🙏

Prerequisite

• Make sure your problem is not listed in the common build problems.
• Make sure no duplicated issue has already been reported in the pyenv issues. You should look for closed issues, too.
• Make sure you are not asking us to help solving your specific issue.
  • GitHub issues is opened mainly for development purposes. If you want to ask someone to help solving your problem, go to some community site like Gitter, StackOverflow, etc.
• Make sure your problem is not derived from packaging (e.g. Homebrew).
  • Please refer to the package documentation for the installation issues, etc.
• Make sure your problem is not derived from plugins.
  • This repository is maintaining pyenv and the default python-build plugin only. Please refrain from reporting issues of other plugins here.

Description

• Platform information (e.g. Ubuntu Linux 16.04): Mac OS 14.5 (Sonoma)
• OS architecture (e.g. amd64): ARM
• pyenv version: 2.4.1
• Python version: 2.7.18 (trying to install)
• C Compiler information (e.g. gcc 7.3): GCC 4.2.1
• Please attach the debug trace of the failing command as a gist: https://gist.github.com/laulaz/58b3eb618607a92c082f288a37573ce7
• If you have a problem with installing Python, please also attach config.log from the build directory
  • The build directory is reported after the "BUILD FAILED" message and is usually under /tmp.
• If the build succeeds but the problem is still with the build process (e.g. the resulting Python is missing a feature), please attach
  • the debug trace from reinstalling the faulty version with env PYENV_DEBUG=1 pyenv install -f -k -v <version> 2>&1 | tee trace.log
  • config.log from the build directory. When using pyenv install with -k as per above, the build directory will be under $PYENV_ROOT/sources.

[native-api]

Please also attach config.log

[native-api]

/private/var/folders/x2/49j6tpl10f9bsqh4dsw2hf040000gn/T/python-build.20240515221458.92130/Python-2.7.18/Modules/_ctypes/libffi/src/arm/sysv.S:64:1: error: unexpected token at start of statement
@ This selects the minimum architecture level required.

Bundled libffi doesn't seem to support Apple Silicon.

@ as a line comment is ARM-specific: see https://sourceware.org/binutils/docs/as/ARM_002dChars.html vs https://sourceware.org/binutils/docs/as/AArch64_002dChars.html .

Apple Silicon in ARM64 (AArch64) but reports itself as "arm" -- this may be the reason.

The best option seems to compile for x64 (which would need x64 Homebrew for dependencies).

[native-api]

This seems to be the upstream issue: libffi/libffi#571

[native-api]

Bundled libffi doesn't seem to support Apple Silicon.

But this should be irrelevant since our downstream patches make 2.7.18 use system libffi.

config.log may clarify what's happening

[laulaz]

@native-api thank you for your help 🙏

Here are the files gists :

• config.log
• python-build.log

[native-api]

configure:3928: checking for C compiler version
configure:3937: clang --version >&5
Apple clang version 15.0.0 (clang-1500.3.9.4)
Target: arm64-apple-darwin23.5.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

This toolchain may not have libffi headers. The recommended setup is to use the XCode Command Line Tools toolchain.

Make sure that you have XCode Command line tools installed with xcode-select --install. If you do, select it with xcode-select -r .
In the end, xcode-select -p should be showing command line tools:

% xcode-select -p    
/Library/Developer/CommandLineTools

---

PATH: /opt/homebrew/opt/binutils/bin

this is a big no-no as it conflicts with Apple's toolchain, see https://github.com/pyenv/pyenv/wiki/Common-build-problems#keg-only-homebrew-packages-are-forcibly-linked--added-to-path

I fact, this may be the root cause, as it overrides the toolchain's assembler IIRC.

[laulaz]

Thank you for your research and your patience @native-api !

I removed the /opt/homebrew/opt/binutils/bin exported path.

XCode Command Line Tools were indeed installed, but xcode-select -p was returning : /Applications/Xcode.app/Contents/Developer
I switched it with : sudo xcode-select -switch /Library/Developer/CommandLineTools

Now, when I try to install Python, I get InstalledDir: /Library/Developer/CommandLineTools/usr/bin in the config.log.

But the error is still the same as in my first post 😞

If it can help, here are my installed Brew formulaes :

brew list
==> Formulae
autoconf		gdbm			libssh2			pcre2			readline
autoenv			htop			libuv			pkg-config		sqlite
bash			httpie			lz4			postgresql@14		tcl-tk
bat			icu4c			m4			pyenv			tig
brotli			krb5			mpdecimal		python@3.10		xz
c-ares			libffi			ncurses			python@3.11		zsh-autosuggestions
ca-certificates		libgit2			node			python@3.12
certifi			libnghttp2		oniguruma		python@3.8
direnv			libpq			openssl@3		python@3.9

[native-api]

Could you attach the new logs?

[native-api]

Other suspicious things in config.log were:

CFLAGS='-I/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk/usr/include/ffi '

These should not be needed (and may even hurt). I suspected that you only set this envvar temporarily when trying different things and ultimately removed. But I think I should mention it regardless just in case.

[laulaz]

Could you attach the new logs?

• trace.log
• config.log

I removed all the "**_FLAGS" exports and the Homebrew binutils.

[native-api]

For some reason, the logic in setup.py doesn't use system Libffi despite the fact it should for MacOS >= 10.15 .

You may try to force it to by setting PYTHON_CONFIGURE_OPTS="--with-system-libffi".

To get to the root cause, I suggest to run installation with -k, then go to the build tree under $PYENV_ROOT/sources, add debug printing to this file to see what's happening, and run Make. The invocation line for Make take from the trace log.

As another workaroud, I still suggest trying to install this for x64. I see in setup.py that the logic clearly wasn't designed to support ARM in MacOS, only x64 and PowerPC.