clang executable detection

[sdarwin]

Environment and version details

• Operating System+version: Linux Ubuntu 18.04
• Compiler+version: Clang 10
• Shell: Bash
• B2 Version:

+ b2 -v
B2 Version 4.5. OS=LINUX.
  Copyright 1993-2002 Christopher Seiwald and Perforce Software, Inc.
  Copyright 2001 David Turner.
  Copyright 2001-2004 David Abrahams.
  Copyright 2002-2019 Rene Rivera.
  Copyright 2003-2015 Vladimir Prus.

  DEFAULTS: jobs = 2
+ b2 --version
B2 4.5-git

Describe your use case

Hi,

Here's an issue that I've just encounter.

Imagine you run a build with "toolset=clang-10"

For example,

./b2 libs/json/test toolset=clang-10 cxxstd=17,2a variant=release -j3 warnings=extra warnings-as-errors=on

What is the expected result if clang-10 doesn't exist on that machine? You would expect a warning message would appear. Even more likely, that the build would stop and tell you the compiler is not available.

However, what b2 does is proceed with whatever clang++ that it finds on the system.

By extension, if multiple clang's are installed, which one will it use? Not what is specified by toolset=. Rather, the first clang++ it finds on the PATH.

jeking3 already discovered this, mentioned here: boostorg/build#306 .

Describe the solution you'd like

gcc does not have the problem.
"it would seem that the version selection code currently inside the init rule inside gcc.jam should be considered generic to all toolset= processing."
Make the gcc solution more general, and include clang.

[Kojoley]

• B2 Version:

+ b2 -v
B2 Version 4.5. OS=LINUX.

You had tested not the latest version. The #1 should have solved exactly the issue you are describing.

[sdarwin]

@Kojoley , if that's the case, then excellent news!

Next question.. If you build a boost library (with boost and relevant dependencies) on the 'develop' branch instead of the 'master' branch, it usually pulls in the latest 'develop' branch code from dependent libraries. However, the most recent copy of b2 was a month ago:

commit 2ad7c03b1e5669842b6e5106632cd6cff8cffcb8
Author: Automated Commit <automated@calamity.org.uk>
Date:   Tue Apr 6 13:00:03 2021 +0000

    Update build from develop

Can the new version of b2 be commited into boost 'develop', so that it will apply to CI testing?

[Kojoley]

I do not understand why development had moved here and what is the synchronization policy. These questions must be addressed to @grafikrobot.

[sdarwin]

@Kojoley thank you for the feature enhancement!
The easiest way to test and verify will be in automatic CI, so hopefully it can be merged into boost, or the standard boost scripts.
Closing the issue.

[grafikrobot]

I do not understand why development had moved here and what is the synchronization policy. These questions must be addressed to @grafikrobot.

See.. #29

[sdarwin]

Hi,

The new version of clang detection was merged into boostorg/build a few days ago. Running tests, and something unusual seems to be happening.
clang-9, clang-10, clang-11 are succeeding.
clang 3.8, 4.0, 5.0, 6.0, 7, 8 are failing.

++ ./b2 libs/json/test toolset=clang-8 cxxstd=17 -j3 warnings=extra warnings-as-errors=on
/home/runner/work/json/boost-root/tools/build/src/tools/common.jam:1140: in common.find-compiler from module common
error: toolset clang-linux initialization:
error: version '8' requested but 'clang++-8' not found and version '4.2.1' of default 'clang++-8' does not match

To debug, I added a new section "DEBUG 2", as follows:

echo "PATH IS $PATH" || true
ls -al /usr/bin/clang* || true

It shows that clang++-8 exists and is in the PATH.

example jobs: https://github.com/samd2/json/actions/runs/919878566

Did you already test those combinations? (such as clang 5 ?)

[Kojoley]

That's strange, could you please add --debug-configuration to b2 invocation and post the log?

[sdarwin]

Here is --debug-configuration

++ ./b2 libs/json/test toolset=clang-8 cxxstd=17 -j3 warnings=extra warnings-as-errors=on --debug-configuration
notice: found boost-build.jam at /home/runner/work/json/boost-root/boost-build.jam
notice: loading B2 from /home/runner/work/json/boost-root/tools/build/src/bootstrap.jam
notice: Searching '/etc' '/home/runner' '/home/runner/work/json/boost-root/tools/build/src' '/home/runner/work/json/boost-root/tools/build/src/kernel' '/home/runner/work/json/boost-root/tools/build/src/util' '/home/runner/work/json/boost-root/tools/build/src/build' '/home/runner/work/json/boost-root/tools/build/src/tools' '/home/runner/work/json/boost-root/tools/build/src/contrib' '/home/runner/work/json/boost-root/tools/build/src/.' for site-config configuration file 'site-config.jam'.
notice: Configuration file 'site-config.jam' not found in '/etc' '/home/runner' '/home/runner/work/json/boost-root/tools/build/src' '/home/runner/work/json/boost-root/tools/build/src/kernel' '/home/runner/work/json/boost-root/tools/build/src/util' '/home/runner/work/json/boost-root/tools/build/src/build' '/home/runner/work/json/boost-root/tools/build/src/tools' '/home/runner/work/json/boost-root/tools/build/src/contrib' '/home/runner/work/json/boost-root/tools/build/src/.'.
notice: Searching '/home/runner' '/home/runner/work/json/boost-root/tools/build/src' '/home/runner/work/json/boost-root/tools/build/src/kernel' '/home/runner/work/json/boost-root/tools/build/src/util' '/home/runner/work/json/boost-root/tools/build/src/build' '/home/runner/work/json/boost-root/tools/build/src/tools' '/home/runner/work/json/boost-root/tools/build/src/contrib' '/home/runner/work/json/boost-root/tools/build/src/.' for user-config configuration file 'user-config.jam'.
notice: Configuration file 'user-config.jam' not found in '/home/runner' '/home/runner/work/json/boost-root/tools/build/src' '/home/runner/work/json/boost-root/tools/build/src/kernel' '/home/runner/work/json/boost-root/tools/build/src/util' '/home/runner/work/json/boost-root/tools/build/src/build' '/home/runner/work/json/boost-root/tools/build/src/tools' '/home/runner/work/json/boost-root/tools/build/src/contrib' '/home/runner/work/json/boost-root/tools/build/src/.'.
notice: Searching '.' for project-config configuration file 'project-config.jam'.
notice: Loading project-config configuration file 'project-config.jam' from '.'.
notice: will use 'clang++' for clang-linux, condition <toolset>clang-linux-4
notice: [python-cfg] Configuring python...
notice: [python-cfg]   user-specified version: "2.7"
notice: [python-cfg]   user-specified cmd-or-prefix: "/usr"
notice: [python-cfg] Checking interpreter command "/usr/bin/python2.7"...
notice: [python-cfg] running command '/usr/bin/python2.7 -c "from sys import *; print('version=%d.%d\nplatform=%s\nprefix=%s\nexec_prefix=%s\nexecutable=%s' % (version_info[0],version_info[1],platform,prefix,exec_prefix,executable))" 2>&1'
notice: [python-cfg] ...requested configuration matched!
notice: [python-cfg] Details of this Python configuration:
notice: [python-cfg]   interpreter command: "/usr/bin/python2.7"
notice: [python-cfg]   include path: "/usr/include/python2.7"
notice: [python-cfg]   library path: "/usr/lib/python2.7/config" "/usr/lib"
notice: [python-cfg] Checking for NumPy...
notice: [python-cfg] running command '/usr/bin/python2.7 -c "import sys; sys.stderr = sys.stdout; import numpy; print(numpy.get_include())"'
notice: [python-cfg] NumPy disabled. Reason:
notice: [python-cfg]   /usr/bin/python2.7 -c "import sys; sys.stderr = sys.stdout; import numpy; print(numpy.get_include())" aborted with 
notice: [python-cfg]   Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named numpy
notice: [cmdline-cfg] toolset clang-8 not previously configured; attempting to auto-configure now
/home/runner/work/json/boost-root/tools/build/src/tools/common.jam:1140: in common.find-compiler from module common
error: toolset clang-linux initialization:
error: version '8' requested but 'clang++-8' not found and version '4.2.1' of default 'clang++-8' does not match
error: initialized from
/home/runner/work/json/boost-root/tools/build/src/tools/clang-linux.jam:55: in clang-linux.init from module clang-linux
/home/runner/work/json/boost-root/tools/build/src/build/toolset.jam:44: in toolset.using from module toolset
/home/runner/work/json/boost-root/tools/build/src/tools/clang.jam:33: in clang.init from module clang
/home/runner/work/json/boost-root/tools/build/src/build/toolset.jam:44: in toolset.using from module toolset
/home/runner/work/json/boost-root/tools/build/src/build-system.jam:543: in process-explicit-toolset-requests from module build-system
/home/runner/work/json/boost-root/tools/build/src/build-system.jam:610: in load from module build-system
/home/runner/work/json/boost-root/tools/build/src/kernel/modules.jam:295: in import from module modules
/home/runner/work/json/boost-root/tools/build/src/kernel/bootstrap.jam:139: in boost-build from module
/home/runner/work/json/boost-root/boost-build.jam:17: in module scope from module

Error: Process completed with exit code 1.

[Kojoley]

I get it, Clang 8 and below reports its version as 4.2.1

[sdarwin]

Well, you may be right. Certainly, something is reporting 4.2.1.
Querying the version on the command line shows 8.0.1.

clang++-8 --version
clang version 8.0.1- (branches/release_80)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

[pdimov]

Yes, clang 8 reports the GCC emulation version on clang++ -dumpversion (and does not support -dumpfullversion).

C:\Users\Peter Dimov>clang++ --version
clang version 8.0.1 (tags/RELEASE_801/final)
Target: x86_64-unknown-windows-cygnus
Thread model: posix
InstalledDir: /usr/bin

C:\Users\Peter Dimov>clang++ -dumpversion
4.2.1

C:\Users\Peter Dimov>clang++ -dumpfullversion
clang-8: error: no input files

C:\Users\Peter Dimov>clang++ -dumpfullversion -dumpversion
4.2.1