clang-scan-deps unable to find system headers in p1689 format

[mordante]

While testing modules with CMake I ran into an issue that some system headers are not found.

I tested with Debian clang version 17.0.0 (++20230226071743+b8df886c1e2a-1~exp1~20230226071846.576)

And this code

module;
#include <vector>
export module test;
void test() {}

Compilation with Clang succeeds

/usr/bin/clang++-17 -stdlib=libc++ -std=c++20 -x c++ test.cppm -c -o /tmp/test.cppm.o

But testing with the dependency scanner

/usr/bin/clang-scan-deps-17 -format=p1689 -- /usr/bin/clang++-17 -stdlib=libc++ -std=c++20 -x c++ test.cppm -c -o /tmp/test.cppm.o

Fails with the following output

Error while scanning dependencies for test.cppm:
In file included from test.cppm:3:
In file included from /usr/bin/../include/c++/v1/vector:281:
In file included from /usr/bin/../include/c++/v1/__algorithm/copy.h:12:
In file included from /usr/bin/../include/c++/v1/__algorithm/copy_move_common.h:12:
In file included from /usr/bin/../include/c++/v1/__algorithm/iterator_operations.h:15:
In file included from /usr/bin/../include/c++/v1/__iterator/advance.h:26:
In file included from /usr/bin/../include/c++/v1/cstdlib:87:
In file included from /usr/bin/../include/c++/v1/stdlib.h:94:
In file included from /usr/include/stdlib.h:31:
/usr/bin/../include/c++/v1/stddef.h:17:15: fatal error: 'stddef.h' file not found
{
  "revision": 0,
  "rules": [],
  "version": 1
}

The line where the header is not found uses #include_next <stddef.h> to include stddef.h.

I get similar errors when I don't specify libc++ and use libstdc++.

[llvmbot]

@llvm/issue-subscribers-clang-modules

[ChuanqiXu9]

What is the path of <stddef.h> in your system? Does it live in /usr/lib/clang/17/include? If it doesn't live in the directory, then I can understand the behavior.

I can reproduce this if I make a symbolic link of the real clang in another directory. Since the clang-scan-deps won't invoke <executable> actually. The clang-scan-deps just use it to get the resource dir. In the case that we can't put builtin headers like <stddef.h> in the default resource dir, we can use -I to specify the missing resource dir explicitly.

If this works for you, I'd like to update the document.

[HailToDodongo]

Not OP, but is was having the same issue exclusively on Ubuntu when installing the latest snapshot package via APT.
Installing it from source on another Ubuntu or Arch machine works fine.

By default the package seems to create a broken link:
/usr/lib/clang/17/include -> ../../llvm-17/lib/clang/17.0.0/include
But only /usr/lib/llvm-17/lib/clang/17/include exists.
Changing this doesn't fix it however.

After adding -I/usr/lib/clang/17/include to the clang++ options clang-scan-deps works,
without the option it still fails.

Is this behavior expected?
Because what is strange to me is that clang itself works fine even without the setting.

[mordante]

I did some more testing, using the good case with --trace-includes it finds stddef.h in the directory /usr/lib/llvm-17/lib/clang/17/include. Adding
-system /usr/lib/llvm-17/lib/clang/17/include fixes clang-scan-deps.

Your suggestion to use the real path instead of a symlink works too. I think it would be nice if clang-scan-deps would resolve the symlink and use the proper directory, but since there is a solution I would consider this low priority. I will update the documentation to mention this limitation.

[mordante]

I've created https://reviews.llvm.org/D144894

[tschuett]

> clang -print-resource-dir

clang knows where stddef.h is.

[mordante]

> clang -print-resource-dir

clang knows where stddef.h is.

Thanks for the info, this seems to work when using the symlink too. Maybe then it's trivial to fix. @ChuanqiXu9 if you want I can take a look at it.

[ChuanqiXu9]

> clang -print-resource-dir

clang knows where stddef.h is.

Thanks for the info, this seems to work when using the symlink too. Maybe then it's trivial to fix. @ChuanqiXu9 if you want I can take a look at it.

I'll be pretty appreciated if you can look at it :)

[huangqinjin]

I am trying to use clang-scan-deps from LLVM 16 as the scanner for gcc.

Use the same code and replace /usr/bin/clang++-17 with /usr/bin/g++:

$ /opt/llvm-16/bin/clang-scan-deps -format=p1689 -- /usr/bin/g++ -std=c++20 -x c++ test.cppm -c -o /tmp/test.cppm.o
Error while scanning dependencies for test.cppm:
In file included from test.cppm:3:
In file included from /usr/bin/../lib64/gcc/x86_64-suse-linux/13/../../../../include/c++/13/vector:80:
In file included from /usr/bin/../lib64/gcc/x86_64-suse-linux/13/../../../../include/c++/13/bits/memory_resource.h:38:
/usr/bin/../lib64/gcc/x86_64-suse-linux/13/../../../../include/c++/13/cstddef:50:10: fatal error: 'stddef.h' file not found
{
  "revision": 0,
  "rules": [],
  "version": 1
}

On my system, stddef.h is in /usr/lib64/gcc/x86_64-suse-linux/13/include.

Manually adding -isystem /usr/lib64/gcc/x86_64-suse-linux/13/include resolves the issue.

[superg]

Experiencing the same issue with Ubuntu 22.04 clang-17 nightly packages.
Two symlinks point to non existing paths:

/usr/lib/clang/17/include -> ../../llvm-17/lib/clang/17.0.0/include
/usr/lib/clang/17/lib -> ../../llvm-17/lib/clang/17.0.0/lib

My current workaround:
sudo ln -s /usr/lib/llvm-17/lib/clang/17 /usr/lib/llvm-17/lib/clang/17.0.0

[Splines]

Two symlinks point to non existing paths:

Can confirm this for Clang16:

/usr/lib/clang/16/include -> ../../llvm-16/lib/clang/16.0.5/include
/usr/lib/clang/16/lib -> ../../llvm-16/lib/clang/16.0.5/lib

although I only have these folders:

/usr/lib/llvm-16/lib/clang/16/include/
/usr/lib/llvm-16/lib/clang/16/lib/

[namniav]

I have similar issue using clang-scan-deps with Apple Clang and macOS SDK. Leaving information in case it's helpful for fixing or for folks to find this issue.

$ clang-scan-deps --format=p1689 -- \
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ \
-isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk \
-std=c++20 -c test.cpp -o test.cpp.o

Error while scanning dependencies for test.cpp:
In file included from test.cpp:1:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/iostream:37:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/ios:214:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/iosfwd:95:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/wchar.h:119:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/wchar.h:89:10: fatal error: 'stdarg.h' file not found
{
  "revision": 0,
  "rules": [],
  "version": 1
}

test.cpp just includes <iostream> and does nothing else.

The command following -- works along without clang-scan-deps. By adding -MD we can also exam the location of stdarg.h which clang-scan-deps fails to find.

$ /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ \
-isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk \
-std=c++20 -MD -c test.cpp -o test.cpp.o

In test.cpp.d:

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.1.6/include/stdarg.h

A temporary sloppy workaround, at user side of clang-scan-deps, could be adding this directory to -I flag in clang-scan-deps command, but I'm not sure whether it would introduce other problems that I did't encounter yet.