Unable to make Boost include directory a system directory on macOS because /usr/local/include precedes it #8755

nomis · 2021-05-10T16:07:43Z

Describe the bug
The Boost header files contain issues that are reported as compiler warnings (and then promoted to errors, stopping the build).

To avoid this the dependency can be set as include_type: 'system'.

On macOS with Homebrew, Boost is available in both /usr/local/Cellar/boost/1.74.0 and /usr/local (the latter being symlinks to the former). The compiler has -I/usr/local as a built-in include directory and this is used before -isystem/usr/local/Cellar/boost/1.74.0. The result is that compilation fails because it uses the headers in /usr/local and for some inexplicable reason despite being a built-in location this is not considered a system directory by the compiler.

To attempt to resolve that, I tried to use /usr/local as the BOOST_ROOT but Meson ignores the Boost in /usr/local because mesonbuild/dependencies/boost.py explicitly ignores symlinks when checking for the libraries (if not i.is_file() or i.is_symlink():).

Fixing that problem (by removing the symlink check) then results in another problem because Meson is filtering out the -isystem/usr/local/include include somewhere (it doesn't appear on the compiler command line). The compiler then continues to use /usr/local/include as a non-system directory and complains about issues in the header files.

There are multiple issues here:

Meson should be able to use the Boost in /usr/local despite all the libraries being symlinks, as long as the links can be dereferenced to files.
The clang compiler has /usr/local/include as a built-in include but not as a system include.
The clang compiler is putting -I/usr/local/include before -isystem/usr/local/Cellar/boost/1.74.0/include and there is no way around this. Anything in -I (built-in or not) is ordered before -isystem. Anything specified as both -isystem and -I only gets added as -isystem.
Meson should add -isystem/usr/local to the compiler command line if that's specified in CFLAGS/CXXFLAGS or c_args/cpp_args in the meson.build file.
Meson should be able to add -isystem/usr/local to the compiler command line if that's the location of the dependency and include_type: 'system' is set.

Issue 4 should be fixed. Currently I can't override Meson by setting CXXFLAGS or cpp_args. It's removing my '-isystem/usr/local/include'. I have to use '-isystem//usr/local/include/' or '-isystem/usr/local/include/' to trick it (this works because clang is normalising the path before using it).

Fixing issues 1 and 5 would allow the build to succeed, but it would not be possible to override the location of Boost as a system include. To get clang to use /usr/local/Cellar/boost/1.74.0/include as a system directory, /usr/local/include must also be specified as a system directory so that it doesn't take precedence in the search order: -isystem/usr/local/Cellar/boost/1.74.0 -isystem/usr/local/include.

Fixing issue 1 without also fixing issue 5 would make it impossible to use -isystem with Boost in the default /usr/local/include location.

This isn't a problem with gcc or non-Apple clang because -isystem always comes before built-in directories (and they don't have broken built-in search paths).

To Reproduce

With macOS, install Boost using Homebrew
/usr/local/include/boost is a link to ../Cellar/boost/1.74.0/include/boost
/usr/local/lib/libboost* are links to ../Cellar/boost/1.74.0/lib/libboost_*
Try to build https://github.com/nomis/dtee at e93ffcc1ab0a102b39c6ecfd7acfb04605d03c53

c++ -Idtee.p -I. -I../.. -Isrc -Xclang -fcolor-diagnostics -pipe -Wall -Winvalid-pch -Wnon-virtual-dtor -Wextra -Wpedantic -Werror -std=c++14 -O3 -Wshadow -isystem/usr/local/Cellar/boost/1.74.0/include -DBOOST_FILESYSTEM_DYN_LINK=1 -DBOOST_SYSTEM_DYN_LINK=1 -DBOOST_ALL_NO_LIB -D_POSIX_C_SOURCE=200809L -D_DARWIN_C_SOURCE '-DGETTEXT_PACKAGE="dtee"' '-DGETTEXT_LOCALEDIR="/usr/local/share/locale"' -DBOOST_ASIO_DISABLE_THREADS -MD -MQ dtee.p/src_input.cc.o -MF dtee.p/src_input.cc.o.d -o dtee.p/src_input.cc.o -c ../../src/input.cc -v
Apple clang version 12.0.0 (clang-1200.0.32.27)
Target: x86_64-apple-darwin19.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/c++/v1"
 "/Library/Developer/CommandLineTools/usr/bin/clang" -cc1 -triple x86_64-apple-macosx10.15.0 -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -Werror=implicit-function-declaration -emit-obj -disable-free -disable-llvm-verifier -discard-value-names -main-file-name input.cc -mrelocation-model pic -pic-level 2 -mthread-model posix -mframe-pointer=all -fno-strict-return -masm-verbose -munwind-tables -target-sdk-version=10.15.6 -fcompatibility-qualified-id-block-type-checking -target-cpu penryn -dwarf-column-info -debugger-tuning=lldb -target-linker-version 609.6 -v -resource-dir /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0 -dependency-file dtee.p/src_input.cc.o.d -skip-unused-modulemap-deps -MT dtee.p/src_input.cc.o -sys-header-deps -isystem /usr/local/Cellar/boost/1.74.0/include -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -I dtee.p -I . -I ../.. -I src -D BOOST_FILESYSTEM_DYN_LINK=1 -D BOOST_SYSTEM_DYN_LINK=1 -D BOOST_ALL_NO_LIB -D _POSIX_C_SOURCE=200809L -D _DARWIN_C_SOURCE -D "GETTEXT_PACKAGE=\"dtee\"" -D "GETTEXT_LOCALEDIR=\"/usr/local/share/locale\"" -D BOOST_ASIO_DISABLE_THREADS -I/usr/local/include -stdlib=libc++ -internal-isystem /Library/Developer/CommandLineTools/usr/bin/../include/c++/v1 -internal-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/local/include -internal-isystem /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include -internal-externc-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include -internal-externc-isystem /Library/Developer/CommandLineTools/usr/include -O3 -Wall -Winvalid-pch -Wnon-virtual-dtor -Wextra -Wpedantic -Werror -Wshadow -Wno-reorder-init-list -Wno-implicit-int-float-conversion -Wno-c99-designator -Wno-final-dtor-non-final-class -Wno-extra-semi-stmt -Wno-misleading-indentation -Wno-quoted-include-in-framework-header -Wno-implicit-fallthrough -Wno-enum-enum-conversion -Wno-enum-float-conversion -std=c++14 -fdeprecated-macro -fdebug-compilation-dir /Users/simon/dtee/build/release -ferror-limit 19 -fmessage-length 206 -stack-protector 1 -fstack-check -mdarwin-stkchk-strong-link -fblocks -fencode-extended-block-signature -fregister-global-dtors-with-atexit -fgnuc-version=4.2.1 -fobjc-runtime=macosx-10.15.0 -fcxx-exceptions -fexceptions -fmax-type-align=16 -fdiagnostics-show-option -fcolor-diagnostics -vectorize-loops -vectorize-slp -fcolor-diagnostics -o dtee.p/src_input.cc.o -x c++ ../../src/input.cc
clang -cc1 version 12.0.0 (clang-1200.0.32.27) default target x86_64-apple-darwin19.6.0
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/local/include"
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/Library/Frameworks"
#include "..." search starts here:
#include <...> search starts here:
 dtee.p
 .
 ../..
 src
 /usr/local/include
 /usr/local/Cellar/boost/1.74.0/include
 /Library/Developer/CommandLineTools/usr/bin/../include/c++/v1
 /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include
 /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include
 /Library/Developer/CommandLineTools/usr/include
 /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Frameworks (framework directory)
End of search list.
In file included from ../../src/input.cc:18:
In file included from ../../src/input.h:25:
In file included from /usr/local/include/boost/asio.hpp:112:
In file included from /usr/local/include/boost/asio/ip/basic_resolver.hpp:30:
In file included from /usr/local/include/boost/asio/ip/basic_resolver_query.hpp:21:
In file included from /usr/local/include/boost/asio/ip/resolver_query_base.hpp:19:
/usr/local/include/boost/asio/ip/resolver_base.hpp:69:5: error: declaration shadows a variable in namespace 'boost::asio::ip' [-Werror,-Wshadow]
    v4_mapped = BOOST_ASIO_OS_DEF(AI_V4MAPPED),
    ^
/usr/local/include/boost/asio/ip/address_v6.hpp:296:20: note: previous declaration is here
enum v4_mapped_t { v4_mapped };
                   ^
1 error generated.

Expected behavior
The build is successful because the compiler warnings for the Boost headers are ignored. If the Boost headers are not in the compiler built-in include directories then they must be preferred.

system parameters

macOS Catalina 10.15.7
Python 3.9.0
meson 0.56.0
ninja 1.10.1
Apple clang version 12.0.0 (clang-1200.0.32.27)

The text was updated successfully, but these errors were encountered:

eli-schwartz · 2021-05-10T16:21:15Z

More bugs caused by and only affecting people who misuse and abuse -isystem as "not system, but as a side effect please don't emit warnings for this messy dependency"...

nomis · 2021-05-10T16:32:47Z

More bugs caused by and only affecting people who misuse and abuse -isystem as "not system, but as a side effect please don't emit warnings for this messy dependency"...

No, /usr/include is automatically a system directory on Linux and that's where Boost will be installed by most distributions.

nomis · 2021-05-15T16:00:36Z

The bug where Meson removes -isystem/usr/local/include is in compilers/mixins/clike.py where there is an assumption that every compiler built-in include directory is a system include path, which is not true for Apple's clang:

        # Remove system/default include paths added with -isystem
        default_dirs = self.compiler.get_default_include_dirs()
        if default_dirs:
            bad_idx_list = []  # type: T.List[int]
            for i, each in enumerate(new):
                if not each.startswith('-isystem'):
                    continue

The /usr/local/include below is a non-system include path (output lines split with \ to make it readable):

% clang -xc++ -E -v - </dev/null
Apple clang version 12.0.0 (clang-1200.0.32.27)
Target: x86_64-apple-darwin19.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/c++/v1"
 "/Library/Developer/CommandLineTools/usr/bin/clang" -cc1 -triple\
 x86_64-apple-macosx10.15.0 -Wdeprecated-objc-isa-usage\
 -Werror=deprecated-objc-isa-usage -Werror=implicit-function-declaration -E\
 -disable-free -disable-llvm-verifier -discard-value-names -main-file-name -\
 -mrelocation-model pic -pic-level 2 -mthread-model posix -mframe-pointer=all\
 -fno-strict-return -masm-verbose -munwind-tables -target-sdk-version=10.15.6\
 -fcompatibility-qualified-id-block-type-checking -target-cpu penryn\
 -dwarf-column-info -debugger-tuning=lldb -target-linker-version 609.6 -v\
 -resource-dir /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0\
 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk\
 -I/usr/local/include -stdlib=libc++\
 -internal-isystem /Library/Developer/CommandLineTools/usr/bin/../include/c++/v1\
 -internal-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/local/include\
 -internal-isystem /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include\
 -internal-externc-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include\
 -internal-externc-isystem /Library/Developer/CommandLineTools/usr/include\
 -Wno-reorder-init-list -Wno-implicit-int-float-conversion -Wno-c99-designator\
 -Wno-final-dtor-non-final-class -Wno-extra-semi-stmt -Wno-misleading-indentation\
 -Wno-quoted-include-in-framework-header -Wno-implicit-fallthrough\
 -Wno-enum-enum-conversion -Wno-enum-float-conversion -fdeprecated-macro\
 -fdebug-compilation-dir /Users/simon -ferror-limit 19 -fmessage-length 206\
 -stack-protector 1 -fstack-check -mdarwin-stkchk-strong-link -fblocks\
 -fencode-extended-block-signature -fregister-global-dtors-with-atexit\
 -fgnuc-version=4.2.1 -fobjc-runtime=macosx-10.15.0 -fcxx-exceptions\
 -fexceptions -fmax-type-align=16 -fdiagnostics-show-option -fcolor-diagnostics\
 -o - -x c++ -
clang -cc1 version 12.0.0 (clang-1200.0.32.27) default target x86_64-apple-darwin19.6.0
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/local/include"
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/Library/Frameworks"
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/include
 /Library/Developer/CommandLineTools/usr/bin/../include/c++/v1
 /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include
 /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include
 /Library/Developer/CommandLineTools/usr/include
 /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Frameworks (framework directory)
End of search list.
# 1 "<stdin>"
# 1 "<built-in>" 1
# 1 "<built-in>" 3
# 379 "<built-in>" 3
# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "<stdin>" 2

The only way I can see to identify this is to parse the command line that -v outputs, because it specifies -I/usr/local/include whereas it would be -internal-isystem /usr/local/include if it was a system directory.

On Ubuntu it has -internal-isystem /usr/local/include built-in (output lines split with \ to make it readable):

$ clang -xc++ -E -v - </dev/null
Ubuntu clang version 13.0.0-++20210513063813+10de21720989-1~exp1~20210513164557.2411
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/i686-linux-gnu/8
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/10
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8.5
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9.4
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/5
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/5.5.0
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/6
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/6.5.0
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/7
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/7.5.0
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/8
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/9
Selected GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/10
Candidate multilib: .;@m64
Selected multilib: .;@m64
 (in-process)
 "/usr/lib/llvm-13/bin/clang" -cc1 -triple x86_64-pc-linux-gnu -E -disable-free\
 -disable-llvm-verifier -discard-value-names -main-file-name - -mrelocation-model\
 static -mframe-pointer=all -fmath-errno -fno-rounding-math -mconstructor-aliases\
 -munwind-tables -target-cpu x86-64 -tune-cpu generic -debugger-tuning=gdb -v\
 -fcoverage-compilation-dir=/home/simon -resource-dir /usr/lib/llvm-13/lib/clang/13.0.0\
 -internal-isystem /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10\
 -internal-isystem /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/x86_64-linux-gnu/c++/10\
 -internal-isystem /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/backward\
 -internal-isystem /usr/lib/llvm-13/lib/clang/13.0.0/include -internal-isystem /usr/local/include\
 -internal-isystem /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../x86_64-linux-gnu/include\
 -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include\
 -internal-externc-isystem /usr/include -fdeprecated-macro -fdebug-compilation-dir=/home/simon\
 -ferror-limit 19 -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions\
 -fcolor-diagnostics -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o - -x c++ -
clang -cc1 version 13.0.0 based upon LLVM 13.0.0 default target x86_64-pc-linux-gnu
ignoring nonexistent directory "/usr/local/include"
ignoring nonexistent directory "/usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../x86_64-linux-gnu/include"
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10
 /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/x86_64-linux-gnu/c++/10
 /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/backward
 /usr/lib/llvm-13/lib/clang/13.0.0/include
 /usr/include/x86_64-linux-gnu
 /usr/include
End of search list.
# 1 "<stdin>"
# 1 "<built-in>" 1
# 1 "<built-in>" 3
# 386 "<built-in>" 3
# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "<stdin>" 2

henryiii mentioned this issue Feb 9, 2022

meson: add support for testsuite CLIUtils/CLI11#701

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unable to make Boost include directory a system directory on macOS because /usr/local/include precedes it #8755

Unable to make Boost include directory a system directory on macOS because /usr/local/include precedes it #8755

nomis commented May 10, 2021

eli-schwartz commented May 10, 2021

nomis commented May 10, 2021

nomis commented May 15, 2021

Unable to make Boost include directory a system directory on macOS because /usr/local/include precedes it #8755

Unable to make Boost include directory a system directory on macOS because /usr/local/include precedes it #8755

Comments

nomis commented May 10, 2021

eli-schwartz commented May 10, 2021

nomis commented May 10, 2021

nomis commented May 15, 2021