CMake build failure on macOS: __libcheck.c:4:10: fatal error: 'err.h' file not found

[karkhaz]

[Edit] Current status: the workaround to this bug is to set the absolute path to the compiler when configuring CBMC, like this:

> cmake -DCMAKE_C_COMPILER=/usr/bin/clang ...

---

original bug report follows

---

CBMC fails to build with CMake on macOS. For some reason, building using the Makefile in the src directory works.

I've been unable to find a commit that broke this, and I also recently updated my system (both macOS itself and also the command-line tools). So I'm wondering if that was to blame. I haven't been able to find any information online about moved headers, though. err.h still exists:

> find /Library/Developer -name err.h
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/err.h

but I don't have an older system to compare against.

Command:

> rm -rf build/*
> cmake -H. -Bbuild -GNinja -DWITH_JBMC=0 -DCMAKE_CXX_FLAGS="-fcolor-diagnostics"
> cmake --build build
# ...
[18/803] Generating library-check.stamp
FAILED: src/ansi-c/library-check.stamp
cd /Users/karkhaz/cbmc/src/ansi-c && /Users/karkhaz/cbmc/src/ansi-c/library_check.sh /Library/Developer/CommandLineTools/usr/bin/clang /Users/karkhaz/cbmc/src/ansi-c/library/cegis.c /Users/karkhaz/cbmc/src/ansi-c/library/ctype.c /Users/karkhaz/cbmc/src/ansi-c/library/err.c /Users/karkhaz/cbmc/src/ansi-c/library/errno.c /Users/karkhaz/cbmc/src/ansi-c/library/fcntl.c /Users/karkhaz/cbmc/src/ansi-c/library/fenv.c /Users/karkhaz/cbmc/src/ansi-c/library/float.c /Users/karkhaz/cbmc/src/ansi-c/library/gcc.c /Users/karkhaz/cbmc/src/ansi-c/library/getopt.c /Users/karkhaz/cbmc/src/ansi-c/library/inet.c /Users/karkhaz/cbmc/src/ansi-c/library/intrin.c /Users/karkhaz/cbmc/src/ansi-c/library/locale.c /Users/karkhaz/cbmc/src/ansi-c/library/math.c /Users/karkhaz/cbmc/src/ansi-c/library/netdb.c /Users/karkhaz/cbmc/src/ansi-c/library/noop.c /Users/karkhaz/cbmc/src/ansi-c/library/process.c /Users/karkhaz/cbmc/src/ansi-c/library/pthread_lib.c /Users/karkhaz/cbmc/src/ansi-c/library/semaphore.c /Users/karkhaz/cbmc/src/ansi-c/library/setjmp.c /Users/karkhaz/cbmc/src/ansi-c/library/signal.c /Users/karkhaz/cbmc/src/ansi-c/library/stdio.c /Users/karkhaz/cbmc/src/ansi-c/library/stdlib.c /Users/karkhaz/cbmc/src/ansi-c/library/string.c /Users/karkhaz/cbmc/src/ansi-c/library/strings.c /Users/karkhaz/cbmc/src/ansi-c/library/syslog.c /Users/karkhaz/cbmc/src/ansi-c/library/time.c /Users/karkhaz/cbmc/src/ansi-c/library/unistd.c /Users/karkhaz/cbmc/src/ansi-c/library/windows.c /Users/karkhaz/cbmc/src/ansi-c/library/x86_assembler.c && /usr/local/Cellar/cmake/3.15.0/bin/cmake -E touch /Users/karkhaz/cbmc/build/src/ansi-c/library-check.stamp
Checking /Users/karkhaz/cbmc/src/ansi-c/library/cegis.c
Checking /Users/karkhaz/cbmc/src/ansi-c/library/ctype.c
Checking /Users/karkhaz/cbmc/src/ansi-c/library/err.c
__libcheck.c:4:10: fatal error: 'err.h' file not found
#include <err.h>
         ^~~~~~~
1 error generated.
clang: error: no such file or directory: '__libcheck.i'
clang: error: no input files
rm: __libcheck.s: No such file or directory
rm: __libcheck.i: No such file or directory
[21/803] Building CXX object src/util/CMakeFiles/util.dir/nondet.cpp.o
ninja: build stopped: subcommand failed.

My command-line tools:

> pkgutil --pkg-info=com.apple.pkg.CLTools_Executables
package-id: com.apple.pkg.CLTools_Executables
version: 10.3.0.0.1.1562985497
volume: /
location: /
install-time: 1564051272
groups: com.apple.FindSystemFiles.pkg-group

Operating system is macOS 10.14.6. Commit is 306697d6, but I've tried starting a bisect waaaaay in the past (d201bad22) and this still shows up even then, so I'm guessing it's an OS issue.

[tautschnig]

It seems that /Library/Developer/CommandLineTools/usr/bin/clang has different behaviour than using /usr/bin/clang. The latter processes a file containing just #include <err.h> just fine, the former fails in the way that you describe.

[tautschnig]

Running both commands with -v is actually pretty revealing. It seems /Library/Developer/CommandLineTools/usr/bin/clang has basically nothing in its search path. I'm not sure this is actually meant to be invoked directly?

[karkhaz]

Thank you! Confirming that this works:

> cmake -DCMAKE_C_COMPILER=/usr/bin/clang ...

You're right about the search path. I don't know why the /Library clang is being invoked, it's not on my $PATH, and which seems to find the right one...

> which clang
/usr/bin/clang
> which clang++
/usr/bin/clang++

Is the fix here to hardcode the /usr/bin path in CMakeLists.txt if we detect that we're running on macOS? That seems ugly, I can't think of anything else, and I also can't discount that this is a unique problem on my laptop. Perhaps I could close this and re-open if anybody else has a similar problem?

[tautschnig]

Maybe from your CMake logs you can figure out how it came to pick that path?

[karkhaz]

The logs aren't that helpful, they mention what compiler was used but don't really give a clue to why. I'm now wondering if CMake was upgraded together with my command-line tools? My CMakeVersion is 3.15.0, perhaps a new version of CMake introduced a bug where it looks in the wrong place?

The system is: Darwin - 18.7.0 - x86_64
Compiling the C compiler identification source file "CMakeCCompilerId.c" succeeded.
Compiler: /Library/Developer/CommandLineTools/usr/bin/clang
Build flags:
Id flags:

The output was:
0


Compilation of the C compiler identification source "CMakeCCompilerId.c" produced "a.out"

The C compiler identification is AppleClang, found in "/Users/karkhaz/cbmc/build/CMakeFiles/3.15.0/CompilerIdC/a.out"

[karkhaz]

Closing this, as I haven't made progress and it may well be a unique problem with my machine.

[dlohar]

I have the same issue on MacOS 10.14.5 with CMake 3.15.1. I have the "err.h" file in the exact same location as mentioned here.

[karkhaz]

Confirming that this error exhibits on a brand-new machine running macOS 10.14.6. I still don't understand why.

The workaround is still to build using

> cmake -DCMAKE_C_COMPILER=/usr/bin/clang ...

at some point, I'll try to change the CMake files to hardcode this path to clang if the OS version is >= 10.14.5. Unless somebody figures out what the actual problem is, but I'm pretty flummoxed. I'm also not ruling out that it's a recent CMake bug rather than a macOS Clang bug; I haven't yet bisected cmake or clang, so I'll get round to that at some stage.

(if it's a CMake or upstream Clang problem, then I could at least theoretically fix it. Whereas if it's a problem with how Apple are building the Clang that they ship, then it's hopeless anyway. Whether I get time to fix this is a different matter...)

[karkhaz]

I haven't had time to look into this, but I'll point out that I've been ramping up new CBMC users over the past few months, and they've all experienced this issue

[karkhaz]

@chrisr-diffblue this is the macOS build issue that I mentioned.

[chrisr-diffblue]

Thanks @karkhaz - yes, this is the same behaviour I've found. I've not entirely figured out what happens here, but I believe it is an issue with Apple's developer tools, possibly when you have a combination of CommandLineTools and XCode installed, and how Apple handle their system frameworks. I use a similar workaround (-DCMAKE_CXX_COMPILER=clang++ -DCMAKE_C_COMPILER=clang where /usr/bin is on the PATH first). I'll see if I can figure out a proper fix though.