configure fails on macOS with non-Apple clang version 13 which implements --print-multiarch

[debohman]

BPO	45405
Nosy	@ned-deily, @ambv, @indygreg, @debohman, @miss-islington
PRs	bpo-45405: Prevent internal configure error when running configure with recent versions of non-Apple clang. #28845 [3.9] bpo-45405: Prevent internal configure error when running configure with recent versions of non-Apple clang. (GH-28845) #28910 [3.10] bpo-45405: Prevent internal configure error when running configure with recent versions of non-Apple clang. (GH-28845) #28911 [3.8] bpo-45405: Prevent internal configure error when running configure with recent versions of clang. (GH-28845) #31889 [3.7] bpo-45405: Prevent internal configure error when running configure with recent versions of clang. (GH-28845) #31890

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2022-03-15.14:47:54.189>
created_at = <Date 2021-10-07.12:56:37.730>
labels = ['3.8', 'build', '3.7']
title = 'configure fails on macOS with non-Apple clang version 13 which implements --print-multiarch'
updated_at = <Date 2022-03-15.14:47:54.189>
user = 'https://github.com/debohman'

bugs.python.org fields:

activity = <Date 2022-03-15.14:47:54.189>
actor = 'ned.deily'
assignee = 'none'
closed = True
closed_date = <Date 2022-03-15.14:47:54.189>
closer = 'ned.deily'
components = ['Build']
creation = <Date 2021-10-07.12:56:37.730>
creator = 'debohman'
dependencies = []
files = []
hgrepos = []
issue_num = 45405
keywords = ['patch']
message_count = 24.0
messages = ['403401', '403424', '403443', '403454', '403455', '403456', '403515', '403556', '403558', '403561', '403562', '403563', '403682', '403685', '403767', '403769', '403771', '403773', '403839', '403881', '403886', '415220', '415221', '415247']
nosy_count = 5.0
nosy_names = ['ned.deily', 'lukasz.langa', 'indygreg', 'debohman', 'miss-islington']
pr_nums = ['28845', '28910', '28911', '31889', '31890']
priority = None
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'compile error'
url = 'https://bugs.python.org/issue45405'
versions = ['Python 3.7', 'Python 3.8']

[debohman]

% CC=clang CXX=clang++ ./configure --enable-optimizations --with-lto
checking build system type... x86_64-apple-darwin16.7.0
checking host system type... x86_64-apple-darwin16.7.0
checking for python3.10... no
checking for python3... python3
checking for --enable-universalsdk... no
checking for --with-universal-archs... no
checking MACHDEP... "darwin"
checking for gcc... clang
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether clang accepts -g... yes
checking for clang option to accept ISO C89... none needed
checking how to run the C preprocessor... clang -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for a sed that does not truncate output... /usr/local/bin/sed
checking for --with-cxx-main=<compiler>... no
checking for the platform triplet based on compiler characteristics... darwin
configure: error: internal configure error for the platform triplet, please file a bug report

The problem occurs here at line 5382 of configure:

if test x$PLATFORM_TRIPLET != x && test x$MULTIARCH != x; then
if test x$PLATFORM_TRIPLET != x$MULTIARCH; then
as_fn_error $? "internal configure error for the platform triplet, please file a bug report" "$LINENO" 5
fi

The problem is that $PLATFORM_TRIPLET is set to darwin (which appears to be correct) and $MULTIARCH is set thus:

MULTIARCH=$($CC --print-multiarch 2>/dev/null)

which evaluates to x86_64-apple-darwin16.7.0.

This is with the public llvm / clang version 13.0.0.

If you set MULTIARCH=darwin in configure, python configures, builds and runs.

[ned-deily]

On what platform and OS version are you building? When you say "public llvm / clang", do you mean from Apple for macOS or from some other source? Be aware that we only officially test and support building Python for macOS on macOS itself using the Apple build tools (Xcode or Command Line Tools).

[debohman]

By "public llvm / clang", I mean the toolchain version released by the llvm project.

They just released version 13.0.0 last week.

The problem is that version 12 of llvm / clang did not implement --print-multiarch, so this logic in configure was not exercised due to:

if test x$PLATFORM_TRIPLET != x && test x$MULTIARCH != x; then

failing. In that case, $MULTIARCH would be a null string, so test x$MULTIARCH != x would fail. It seems that the logic in configure needs to be updated so that it functions correctly on darwin where --print-multiarch is implemented.

[debohman]

I can put together a pull request to fix this. The fix is in configure.ac.

Should I include the mods from the autoreconf in the PR?

[debohman]

Also, what branch should the PR be on, 3.10 or main?

[debohman]

Okay, I see from the developer guide, that the generated files should be included. Is it permissible to use the current version of autoconf?

[ned-deily]

OK, I've had a chance to look at this issue and I now understand what you were describing. So, yes, it would be good to fix the test in configure so that it does work with a newer clang that does implement the --print-multiarch command line option. FTR, this is not an issue with clang from any Apple tools chains (Xcode or Command Line Tools) released to this point. A PR against configure.ac in main would be welcome, we can auto-backport to other active branches as necessary. We currently use autoconf 2.69 but, if that is not available, a core developer can regenerate configure prior to merging.

[debohman]

Yes, I don't understand why the llvm / clang decided to implement this in version 13. In version 12, the switch caused an error to stderr, and nothing to stdout. GCC 11.1 generates a single \n to stdout.

Do you plan to stick with $PLATFORM_TRIPLET = darwin, even on the arm machines? Early on, I tried setting $PLATFORM_TRIPLET in the script to the actual value returned by the complier, and that blew up later during the actual build. That was when I decided to just fix the logic so that it continued to operate the same on all platforms, even darwin with --print-multiarch generating the triplet.

[debohman]

I have version 2.71 of autoconf and the latest autoconf-archive installed on my machine.

[ned-deily]

Do you plan to stick with $PLATFORM_TRIPLET = darwin, even on the arm machines?

For now, yes. Apple long ago implemented the equivalent of multi-architecture support using "-arch" in the compiler driver and the Python build system (configure, Makefile, setup.py, distutils, et al) supports that for macOS without using PLATFORM_TRIPLET.

[debohman]

Okay. Should the branch for the PR be off the main branch, or 3.10?

[ned-deily]

"A PR against configure.ac in main would be welcome, we can auto-backport to other active branches as necessary."

[ned-deily]

Merged for release in 3.10.1 and 3.9.8. Thanks for the issue and PR, David!

[debohman]

Thank you for your help, Ned.

[indygreg]

Can you be more specific? This particular issue isn't about cross-compiling.

The point I was trying to make is that clang 13 generically implemented --print-multiarch and the triple logic in configure related to MULTIARCH applies to all platforms, not just macOS.

I was able to perturb this same error on Linux with an x86_64 Clang 13 with configure options --build=x86_64-unknown-linux-gnu --host=i686-unknown-linux-gnu. https://github.com/indygreg/python-build-standalone/runs/3853832124?check_suite_focus=true should work if you are logged into GitHub. Not the exact same failure, but related since it involves --print-multiarch suddenly working.

[ned-deily]

The point I was trying to make is that clang 13 generically implemented --print-multiarch and the triple logic in configure related to MULTIARCH applies to all platforms, not just macOS.

If there are still issues on other supported platforms, PRs are welcome.

[ned-deily]

As of Xcode 13.3, released 2022-03-14 to support macOS 12.3 et al, the included Apple Clang (Apple clang version 13.1.6 (clang-1316.0.21.2)) now supports the --print-multiarch option and so Python 3.8 and 3.7 are now vulnerable to this issue when building with that version of Xcode or Apple Command Line Tools on macOS. While both 3.8 and 3.7 are now in the security-fix-only phase of their life cycles and are not fully supported on macOS 12, you should still be able to run ./configure without error using the current Apple Developer Tools. PR 31889 and PR 31890 are backports of this to 3.8 and 3.7. Setting to "release blocker" priority for upcoming 3.8 and 3.7 releases.

[ned-deily]

New changeset 720bb45 by Ned Deily in branch '3.7':
bpo-45405: Prevent internal configure error when running configure with recent versions of clang. (GH-28845) (GH-31890)
720bb45

[ambv]

New changeset dea270a by Ned Deily in branch '3.8':
bpo-45405: Prevent internal configure error when running configure with recent versions of clang. (GH-28845) (GH-31889)
dea270a