CMake may unintentionally break order of system include directories

[nthirtyone]

Overview

When linking an imported external target that is located in INTERFACE_INCLUDE_DIRECTORIES to ${EMSCRIPTEN_SYSROOT}/include CMake will emit an -isystem that will break the intended order of system include directories.

Also reported/discussed in #15576, #14686. Possibly others that mention -isystem or CMake with imported packages, I just skimmed over.

Steps to Reproduce

1. Create a basic CMakeLists.txt
2. find_package that is installed in EMSCRIPTEN_SYSROOT
3. target_link_libraries the imported target
4. Include selected standard C++ header, e.g., iostream
5. Try building

Example

Version:

emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 3.1.13-git (c00f6af818882f7a11049fa4a2464cca32d42264)
clang version 15.0.0 (/startdir/llvm-project faef447e72a5c63dfb12bb7b02d44c3c916d31cd)
Target: wasm32-unknown-emscripten
Thread model: posix
InstalledDir: /opt/emscripten-llvm/bin

I installed raylib in sysroot. Then created a new CMakeLists.txt:

cmake_minimum_required(VERSION 3.0)
project(include_order)
set(raylib_USE_STATIC_LIBS ON)
find_package(raylib REQUIRED 4)
add_executable(main main.cpp)
target_link_libraries(main PRIVATE raylib)

The main.cpp is:

#include <vector>
int main(int, char*[]){}

First generate:

$ emcmake cmake -B build .
configure: cmake -B build . -DCMAKE_TOOLCHAIN_FILE=/home/aki/src/emscripten/cmake/Modules/Platform/Emscripten.cmake -DCMAKE_CROSSCOMPILING_EMULATOR=/usr/bin/node;--experimental-wasm-threads
-- Found raylib: /home/aki/src/emscripten/cache/sysroot/lib/libraylib.a
-- Configuring done
-- Generating done
-- Build files have been written to: /home/aki/src/emscripten/tests/cmake/include_order/build

Then build:

[ 50%] Building CXX object CMakeFiles/main.dir/main.cpp.o
In file included from /home/aki/src/emscripten/tests/cmake/include_order/main.cpp:1:
In file included from /home/aki/src/emscripten/cache/sysroot/include/c++/v1/vector:274:
In file included from /home/aki/src/emscripten/cache/sysroot/include/c++/v1/__bit_reference:15:
In file included from /home/aki/src/emscripten/cache/sysroot/include/c++/v1/algorithm:667:
In file included from /home/aki/src/emscripten/cache/sysroot/include/c++/v1/functional:499:
In file included from /home/aki/src/emscripten/cache/sysroot/include/c++/v1/__functional/bind.h:17:
In file included from /home/aki/src/emscripten/cache/sysroot/include/c++/v1/tuple:172:
In file included from /home/aki/src/emscripten/cache/sysroot/include/c++/v1/__functional_base:22:
In file included from /home/aki/src/emscripten/cache/sysroot/include/c++/v1/exception:83:
/home/aki/src/emscripten/cache/sysroot/include/c++/v1/cstdlib:130:9: error: target of using declaration conflicts with declaration already in scope
using ::abs _LIBCPP_USING_IF_EXISTS;
        ^
/home/aki/src/emscripten/cache/sysroot/include/stdlib.h:58:5: note: target of using declaration
int abs (int);
    ^
/home/aki/src/emscripten/cache/sysroot/include/c++/v1/cmath:338:1: note: conflicting declaration
using ::abs _LIBCPP_USING_IF_EXISTS;
^
In file included from /home/aki/src/emscripten/tests/cmake/include_order/main.cpp:1:
In file included from /home/aki/src/emscripten/cache/sysroot/include/c++/v1/vector:274:
In file included from /home/aki/src/emscripten/cache/sysroot/include/c++/v1/__bit_reference:15:
In file included from /home/aki/src/emscripten/cache/sysroot/include/c++/v1/algorithm:667:
In file included from /home/aki/src/emscripten/cache/sysroot/include/c++/v1/functional:506:
In file included from /home/aki/src/emscripten/cache/sysroot/include/c++/v1/__functional/function.h:22:
In file included from /home/aki/src/emscripten/cache/sysroot/include/c++/v1/__memory/shared_ptr.h:35:
In file included from /home/aki/src/emscripten/cache/sysroot/include/c++/v1/atomic:524:
In file included from /home/aki/src/emscripten/cache/sysroot/include/c++/v1/__thread/timed_backoff_policy.h:16:
In file included from /home/aki/src/emscripten/cache/sysroot/include/c++/v1/__threading_support:17:
In file included from /home/aki/src/emscripten/cache/sysroot/include/errno.h:10:
In file included from /home/aki/src/emscripten/cache/sysroot/include/bits/errno.h:1:
In file included from /home/aki/src/emscripten/cache/sysroot/include/wasi/api.h:29:
In file included from /home/aki/src/emscripten/cache/sysroot/include/stddef.h:17:
/home/aki/src/emscripten/cache/sysroot/include/bits/alltypes.h:302:54: error: typedef redefinition with different types ('struct max_align_t' vs 'struct max_align_t')
typedef struct { long long __ll; long double __ld; } max_align_t;
                                                     ^
/opt/emscripten-llvm/lib/clang/15.0.0/include/__stddef_max_align_t.h:24:3: note: previous definition is here
} max_align_t;
  ^
2 errors generated.
em++: error: '/opt/emscripten-llvm/bin/clang++ -target wasm32-unknown-emscripten -DEMSCRIPTEN -D__EMSCRIPTEN_major__=3 -D__EMSCRIPTEN_minor__=1 -D__EMSCRIPTEN_tiny__=13 -fignore-exceptions -fvisibility=default -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj
-mllvm -disable-lsr -Werror=implicit-function-declaration -Xclang -iwithsysroot/include/SDL --sysroot=/home/aki/src/emscripten/cache/sysroot -Xclang -iwithsysroot/include/compat -isystem /home/aki/src/emscripten/cache/sysroot/include -I/home/aki/src/emscripten/cache/sysroot/include -MD -MT CMakeFiles/main.dir/main.cpp.o -MF CMakeFiles/main.dir/main.cpp.o.d -c /home/aki/src/emscripten/tests/cmake/include_order/main.cpp -o CMakeFiles/main.dir/main.cpp.o' failed (returned 1)

Verbose Output

Let's start with a normal case, where nothing is imported. I'll trim and insert newlines for the CLI options to increase readability:

"/opt/emscripten-llvm/bin/clang-15"
<...>
-isysroot /home/aki/src/emscripten/cache/sysroot
-internal-isystem /home/aki/src/emscripten/cache/sysroot/include/wasm32-emscripten/c++/v1
-internal-isystem /home/aki/src/emscripten/cache/sysroot/include/c++/v1
-internal-isystem /opt/emscripten-llvm/lib/clang/15.0.0/include
-internal-isystem /home/aki/src/emscripten/cache/sysroot/include/wasm32-emscripten
-internal-isystem /home/aki/src/emscripten/cache/sysroot/include
<...>
-iwithsysroot/include/SDL
-iwithsysroot/include/compat
<...>
-o CMakeFiles/main.dir/main.cpp.o
-x c++
/home/aki/src/emscripten/tests/cmake/include_order/main.cpp
clang -cc1 version 15.0.0 based upon LLVM 15.0.0git default target x86_64-unknown-linux-gnu
ignoring nonexistent directory "/home/aki/src/emscripten/cache/sysroot/include/wasm32-emscripten/c++/v1"
ignoring nonexistent directory "/home/aki/src/emscripten/cache/sysroot/include/wasm32-emscripten"
#include "..." search starts here:
#include <...> search starts here:
 /home/aki/src/emscripten/cache/sysroot/include/SDL
 /home/aki/src/emscripten/cache/sysroot/include/compat
 /home/aki/src/emscripten/cache/sysroot/include/c++/v1
 /opt/emscripten-llvm/lib/clang/15.0.0/include
 /home/aki/src/emscripten/cache/sysroot/include
End of search list.

And this is the same thing for faulty compilation:

"/opt/emscripten-llvm/bin/clang-15"
<...>
-isystem /home/aki/src/emscripten/cache/sysroot/include
<...>
-I /home/aki/src/emscripten/cache/sysroot/include
-isysroot /home/aki/src/emscripten/cache/sysroot
-internal-isystem /home/aki/src/emscripten/cache/sysroot/include/wasm32-emscripten/c++/v1
-internal-isystem /home/aki/src/emscripten/cache/sysroot/include/c++/v1
-internal-isystem /opt/emscripten-llvm/lib/clang/15.0.0/include
-internal-isystem /home/aki/src/emscripten/cache/sysroot/include/wasm32-emscripten
-internal-isystem /home/aki/src/emscripten/cache/sysroot/include
<...>
-iwithsysroot/include/SDL
-iwithsysroot/include/compat
<...>
-o CMakeFiles/main.dir/main.cpp.o
-x c++
/home/aki/src/emscripten/tests/cmake/include_order/main.cpp
clang -cc1 version 15.0.0 based upon LLVM 15.0.0git default target x86_64-unknown-linux-gnu
ignoring nonexistent directory "/home/aki/src/emscripten/cache/sysroot/include/wasm32-emscripten/c++/v1"
ignoring nonexistent directory "/home/aki/src/emscripten/cache/sysroot/include/wasm32-emscripten"
ignoring duplicate directory "/home/aki/src/emscripten/cache/sysroot/include"
  as it is a non-system directory that duplicates a system directory
ignoring duplicate directory "/home/aki/src/emscripten/cache/sysroot/include"
#include "..." search starts here:
#include <...> search starts here:
 /home/aki/src/emscripten/cache/sysroot/include
 /home/aki/src/emscripten/cache/sysroot/include/SDL
 /home/aki/src/emscripten/cache/sysroot/include/compat
 /home/aki/src/emscripten/cache/sysroot/include/c++/v1
 /opt/emscripten-llvm/lib/clang/15.0.0/include
End of search list.

Why?

First off, we notice that there are two new options emitted:

• -I /home/aki/src/emscripten/cache/sysroot/include
• -isystem /home/aki/src/emscripten/cache/sysroot/include.

The former is coming from CFLAGS of raylib and by itself it does not create any issue. That's because -I has its own sorting group and path is skipped as duplicate in favour of system path.

The latter is emitted by CMake based on INTERFACE_INCLUDE_DIRECTORIES of the imported target. This one is the problematic one, because it is in the same sorting group as -internal-isystem and disturbs the order by appearing first in effective CLI options.

Please note, that this is based on observation and GCC documentation - I went through related clang documentation and wasn't lucky enough to confirm it there.

Solution

Of course, removing the property is not the way to go. Instead we can instruct CMake to ignore it when emitting all of the include directories for the target. This can be accomplished by using CMAKE_<LANG>_IMPLICIT_INCLUDE_DIRECTORIES (I dug way too much to find it, and when I finally found it, my next google search yielded it in results - I'm still mad).

A naive fix would be adding something like this to cmake/Modules/Platform/Emscripten.cmake:

unset(CMAKE_CXX_IMPLICIT_INCLUDE_DIRECTORIES)
list(APPEND CMAKE_CXX_IMPLICIT_INCLUDE_DIRECTORIES "${EMSCRIPTEN_SYSROOT}/include")

But that would require manual maintenance of the paths in question, so I'm currently looking for a way to automatically resolve them and populate in toolchain file. Any suggestions are welcome.

[sbc100]

What do you mean by "But that would require manual maintenance of the paths in question". Isn't there exactly one include path we need to worry about here. i.e. the top level one (since that is the one that cannot every come before the libc++ headers)

[sbc100]

I'm curious why other platforms/toolchain such as android don't suffer from this issue.

Is it perhaps because they use <sysroot>/usr/include rather than <sysroot>/include and then path is already handling internally by cmake: https://github.com/Kitware/CMake/blob/e6c4af42934da54c3bab44778c796af1a45e0032/Modules/Platform/UnixPaths.cmake#L77-L90

Perhaps we should switch to usr/include too?

[nthirtyone]

Yeah, to fix the issue we just need that one particular path, but for the sake of completeness I'm looking to populate the variable with all of the internal paths.

And yes, I didn't try it (not really sure how to manipulate internal paths in clang now), but switching to usr/include would most likely yield good results (at least that's my expectation based on my current knowledge).

[sbc100]

Perhaps you could upload PR with a failing test and a patch to cmake/Modules/Platform/Emscripten.cmake to address it?

You could add TODO there saying that switching to usr/include under the sysroot would might make it redundnat?

That we get a fix today, and possible cleanup plan in the future.

I don't think it important to any other include paths other than that one that is problematic.

[nthirtyone]

Sure, let's go with that; that's too reasonable for me to ignore it just for the sake of an interesting programming exercise.

I don't think it important to any other include paths other than that one that is problematic.

I agree with that.

There is already a test that would actually catch the issue if it used C++: other.test_cmake_install. I'll do something similar, just with C++.

[sbc100]

Converting the test_cmake_install to C++ seems reasonable.