Compilation errors with --system-libclang on macOS Mojave

[pwnall]

To begin, I acknowledge that --system-libclang is unsupported, and I'm grateful for any help you might be able to give.

I've needed --system-libclang on macOS since I started using ycmd (10.11 or 10.12). It's been letting me do my work around Chromium, and I've been having a pretty good experience. After upgrading to Mojave (10.14.0 / 10.14.1), I haven't been able to build ycmd with --system-libclang -- I keep getting the error below.

[100%] Linking CXX shared library /Users/costan/.ycmd/ycm_core.so
Undefined symbols for architecture x86_64:
  "_clang_getCompletionFixIt", referenced from:
      YouCompleteMe::CompletionData::BuildCompletionFixIt(CXCodeCompleteResults*, unsigned long) in CompletionData.cpp.o
  "_clang_getCompletionNumFixIts", referenced from:
      YouCompleteMe::CompletionData::BuildCompletionFixIt(CXCodeCompleteResults*, unsigned long) in CompletionData.cpp.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[3]: *** [/Users/costan/.ycmd/ycm_core.so] Error 1
make[2]: *** [ycm/CMakeFiles/ycm_core.dir/all] Error 2
make[1]: *** [ycm/CMakeFiles/ycm_core.dir/rule] Error 2

I am able to build ycmd without --system-libclang, but when I do that, I can't find C++ system libraries like <memory>, <string> and <vector>. (I've seen a fair number of issues pointing to that problem, and I can't figure out which one is canonical. I hope it's OK to not link to a particular issue, but I'm happy to point one out of it's helpful.)

[puremourning]

Your system libclang is too old. Ycmd requires clang 7.

[pwnall]

@puremourning Thank you for diagnosing the problem!

For anyone else running into the same problem -- my solution was to point ycmd to the LLVM version installed by Homebrew. The key is to prefix the build.py invocation with EXTRA_CMAKE_ARGS="-DEXTERNAL_LIBCLANG_PATH=$(brew --prefix llvm)/lib/libclang.dylib"

For example, I currently use the following.

EXTRA_CMAKE_ARGS="-DEXTERNAL_LIBCLANG_PATH=$(brew --prefix llvm)/lib/libclang.dylib" \
    python3 ./build.py --clang-completer --go-completer --rust-completer

[sittim]

@pwnall thank you, except id you mean install.py instead of build.py above?

[puremourning]

install.py is a wrapper in YouCompleteMe for build.py which is in ycmd

[sittim]

When using @pwnall recommendation, it compiles, but still getting strange errors. As per issue ycm-core/YouCompleteMe#3283, I added custom include directories:

'-isystem','/usr/local/opt/llvm/include/c++/v1',
'-isystem','/usr/local/opt/llvm/include/llvm/Support',
'-isystem','/usr/local/opt/llvm/include/llvm/Config',
'--sysroot'

now the YcmDebugInfo looks good:

Printing YouCompleteMe debug information...
-- Client logfile: /var/folders/1j/fkfrvhf53vz7p2f998b2skdw0000gp/T/ycm_66M_Id.log
-- Server Python interpreter: /usr/local/opt/python/bin/python3.7
-- Server Python version: 3.7.2
-- Server has Clang support compiled in: True
-- Clang version: clang version 7.0.1 (tags/RELEASE_701/final)
-- Extra configuration file found and loaded
-- Extra configuration path: ./ycm_global_ycm_extra_conf.py
-- C-family completer debug information:
--   Compilation database path: None
--   Flags: ['-Wall', '-Wextra', '-Werror', '-Wno-long-long', '-Wno-variadic-macros',
'-fexceptions', '-DNDEBUG', '-DUNITTEST', '-DMODEL_HG2_0', '-D__MSP430_HAS_USCI_Ax__'
, '-DUSE_CLANG_COMPLETER', '-std=c++14', '-xc++',
'-I', '/Users/tsitnikov/HydroGuardFW/hg_os/../googletest/googletest/include',
'-I', '/Users/tsitnikov/HydroGuardFW/hg_os/../googletest/googlemock/include',
'-I', '/Users/tsitnikov/HydroGuardFW/hg_os/.',
'-I', '/Users/tsitnikov/HydroGuardFW/hg_os/../hw_1_5',
'-I', '/Users/tsitnikov/HydroGuardFW/hg_os/../driverlib_msp430',
'-I', '/Users/tsitnikov/HydroGuardFW/hg_os/./include',
'-isystem', '/usr/local/opt/llvm/include/c++/v1',
'-isystem', '/usr/local/opt/llvm/include/llvm/Support',
'-isystem', '/usr/local/opt/llvm/include/llvm/Config',
'--sysroot',
'-resource-dir=/Users/tsitnikov/.vim/plugged/YouCompleteMe/third_party/ycmd/clang_includes',
'-fspell-checking']
-- Translation unit: /Users/tsitnikov/HydroGuardFW/hg_os/sensor/unittest/sample_sensor_test.cpp
-- Server running at: http://127.0.0.1:60580
-- Server process ID: 86385
-- Server logfiles:
--   /var/folders/1j/fkfrvhf53vz7p2f998b2skdw0000gp/T/ycmd_60580_stdout_skSXZd.log
--   /var/folders/1j/fkfrvhf53vz7p2f998b2skdw0000gp/T/ycmd_60580_stderr_OEcLmF.log

It works just fine in Ubuntu 18.04, so I wonder if I should give up and use VirtualBox? Using VBox feels awkward.

[sittim]

Figured it out, with new Mac, you have to:

$ xcode-select --install
$ open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

Brand new Mac requires header installation for C++ development.

[puremourning]

That’s an option. Apple removed the headers from system frameworks in Mojave (for unknowable reasons). Running the above reinstalls them.

A PR is coming in ycmd which makes this not required anymore.

[ain]

In my case the problem was solved by going with sudo rm -rf /opt/local/bin/install_name_tool on the MacPorts version of install_name_tools that broke the ycm_core build.

[rib]

I've been using --system-libclang when working on code being cross-compiled for iOS and after initially trying to update the latest version of ycmd to support --system-libclang I ended up running out of time and just reverting to an older version of YouCompleteMe, which is a bummer. In case it helps anyone else I'm using commit 788c293aee78c6ab60cbc06bbc3339e7b2bff98f.

It looks like 187e836 in third_party/ycmd/ is what causes the compilation error quoted above. It does at least work to #if 0 out the implementation of BuildCompletionFixIt (equivalent behaviour to clang_getCompletionNumFixIts() returning zero).

In my case, even if I fix the compilation issue in master, there were other python changes related to adding -resource-dir arguments that were breaking things and even though it was easy to comment out those things to get to the point where the YcmDebugInfo flags compiled fine (if I ran them manually on the command line) there was some other internal thing I didn't understand that suggested ycm wasn't using exactly the same flags that YcmDebugInfo showed (like it was still using the -resource-dir anyway).

[puremourning]

In my case, even if I fix the compilation issue in master, there were other python changes related to adding -resource-dir arguments that were breaking things

Are you saying that the latest YCM doesn't work for iOS projects ? You no longer need to use --system-libclang to support iOS project builds anymore. I'm sure we tested that.

If it's not working, then could you raise an issue with a full repro case that we can test/confirm? We'd like to fix it.

[puremourning]

For the record, i'm pretty sure we tested it with this: https://github.com/puremourning/YCMTest-iOS. We should probably add some sort of iOS test to our regression suite I suppose.

[rib]

Just double checked but unfortunately it doesn't work for me. (building https://github.com/glimpse-project/glimpse)
Summary instructions for building Glimpse for iOS would be:

pip3 install --user --upgrade git+https://github.com/glimpse-project/meson
sudo xcode-select -switch /Applications/Xcode.app
git clone https://github.com/glimpse-project/glimpse
mkdir glimpse/build-ios-debug
cd glimpse/build-ios-debug
meson --cross-file ../ios-xcode-arm64-cross-file.txt --buildtype=debug --default-library=static ..
ninja

That will also end up creating a compile_commands.json db under build-ios-debug

Here's a minimal snippit from compile_commands.json:

[
  {
    "directory": "/Users/bob/src/glimpse/glimpse/build-ios-debug",
    "command": "clang++ -Iglimpse_viewer@exe -I. -I.. -I../src -I../src/imgui -Isubprojects/dlib -I../subprojects/dlib -Isubprojects/glm-0.9.9-a2 -I../subprojects/glm-0.9.9-a2 -Isubprojects/libpng-1.6.17 -I../subprojects/libpng-1.6.17 -Isubprojects/pcl -I../subprojects/pcl -I../subprojects/pcl/common/include -Isubprojects/eigen-git-mirror-3.3.4 -I../subprojects/eigen-git-mirror-3.3.4 -Isubprojects/boost_1_69_0 -I../subprojects/boost_1_69_0 -I../subprojects/pcl/octree/include -I../subprojects/pcl/kdtree/include -I../subprojects/pcl/segmentation/include -I../subprojects/pcl/geometry/include -I../subprojects/flann/src/cpp -I../subprojects/pcl/search/include -I../subprojects/pcl/sample_consensus/include -I../subprojects/pcl/filters/include -I../subprojects/pcl/features/include -I../subprojects/pcl/2d/include -I../subprojects/pcl/ml/include -Isubprojects/libepoxy/include -I../subprojects/libepoxy/include -Isubprojects/libepoxy/src -I../subprojects/libepoxy/src -I../subprojects/glfm/include -Isubprojects/libepoxy/include/epoxy -Xclang -fcolor-diagnostics -pipe -Wall -Winvalid-pch -Wnon-virtual-dtor -std=c++14 -g -miphoneos-version-min=11.4 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS12.1.sdk -DBOOST_SYSTEM_NO_DEPRECATED -DBOOST_THREAD_USE_LIB -DEIGEN_MPL2_ONLY -DDLIB_NO_GUI_SUPPORT -DDLIB_ENABLE_ASSERTS -DGLM_ENABLE_EXPERIMENTAL -DUSE_AVF -DUSE_GLFM '-DIMGUI_IMPL_OPENGL_LOADER_CUSTOM=<epoxy/gl.h>' -arch arm64 -fembed-bitcode -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS12.1.sdk -MD -MQ 'glimpse_viewer@exe/src_glimpse_viewer.cc.o' -MF 'glimpse_viewer@exe/src_glimpse_viewer.cc.o.d' -o 'glimpse_viewer@exe/src_glimpse_viewer.cc.o' -c ../src/glimpse_viewer.cc",
    "file": "../src/glimpse_viewer.cc"
  }
]

I can open a separate issue.