clangd: Third-party header file not found

[csyJoy]

Version confirmation

• Confirm

Neovim version

NVIM v0.8.2

Operating system/version

macOS 13.2

Terminal name/version

iTerm2 Build 3.4.18

$TERM environment variable

No response

Branch info

main (Default/Latest)

Fetch Preferences

SSH (use_ssh = true)

Affected language servers

clangd

How to reproduce the issue

1. use gD on a third party header file
2. clangd throw an error path/file file not found on a cpp header file

Actual behavior

clangd doesn't find the header file correctly

Expected behavior

clangd finds the header file correctly

Support info

:LspInfo

:Mason

Logs

No response

Additional information

what I did to solve this problem:

1. specify the compiler by -DCMAKE_CXX_COMPILER=clang++
2. specify cland the absolute path of compile_command.json , in the :LspInfo you can see /Users/csy/cpp_test/build which is the correct path.
3. change the homebrew's .gitignore file by adding !/Cellar

I find something that may cause this problem:

1. when I use :pwd in the third party headers file's buffer, It print /opt/homebrew, not the actual path of the file
2. in :LspInfo the root directory is /opt/homebrew
3. originally, when I open nvim-tree in /opt/homebrew, It doesn't show Cellar dir, after I change the .gitignore file of homebrew by add !/Cellar, nvim-tree shows the Cellar dir, but clangd still throws the file not found error

IMO, buffer's path and the root directory and the wired Cellar directory cause this problem, but what happened actually?

PS: my pc is macbook with apple silicon so the homebrew path is /opt/homebrew

[CharlesChiuGit]

1. when I use :pwd in the third party headers file's buffer, It print /opt/homebrew, not the actual path of the file

Above behavior is by-designed, we use a plugin call https://github.com/ahmedkhalf/project.nvim, which set the cwd to project root base on certain rules¹. And that cause :pwd to be /opt/homebrew.

U can set this line:

nvimdots/lua/modules/tools/config.lua

Line 82 in b984ad0

manual_mode = false,

to true to see if that helps.

2. in :LspInfo the root directory is /opt/homebrew

U might want to check if their any file in Cellar/ that meets lsp's root detect pattern.

Footnotes

1. https://github.com/ayamir/nvimdots/blob/b984ad0c6057fc275a2d349add2e2907a56d8744/lua/modules/tools/config.lua#L84 ↩

[csyJoy]

1. when I use :pwd in the third party headers file's buffer, It print /opt/homebrew, not the actual path of the file

Above behavior is by-designed, we use a plugin call https://github.com/ahmedkhalf/project.nvim, which set the cwd to project root base on certain rules1. And that cause :pwd to be /opt/homebrew.

U can set this line:

nvimdots/lua/modules/tools/config.lua

Line 82 in b984ad0

manual_mode = false,

to true to see if that helps.

2. in :LspInfo the root directory is /opt/homebrew

U might want to check if their any file in Cellar/ that meets lsp's root detect pattern.

Footnotes

1. nvimdots/lua/modules/tools/config.lua

   Line 84 in b984ad0

   patterns = { ".git", "_darcs", ".hg", ".bzr", ".svn", "Makefile", "package.json" },

   ↩

thanks for your advice. I tried both of them whereas the :LspInfo was still /opt/homebrew, here is my step:

1. I found out that for any homebrew installed package, there exist a .brew, ie. /opt/homebrew/Cellar/boost/1.81.0_1/.brew, I want /opt/homebrew/Cellar/boost/1.81.0_1/ to be the lsp's root dir.
2. I changed the pattern field in project.setup by adding ".brew" pattern

Is there something wrong with what I did? or anything should be done after changing the config file to make config file work?

[Jint-lzxy]

@csyJoy No, the core issue is how you generate compile_command.json? You may revert all changes made till now, they're unnecessary. Also please do not pass --compile-commands-dir to clangd.

[Jint-lzxy]

Additional information

what I did to solve this problem:

1. specify the compiler by -DCMAKE_CXX_COMPILER=clang++

Do you also have gcc installed? Otherwise, this is unnecessary - all C/C++ compilers on macOS point to clang internally, even /usr/bin/gcc

2. specify cland the absolute path of compile_command.json , in the :LspInfo you can see /Users/csy/cpp_test/build which is the correct path.

How did you generate this json schema? The best way is to specify set(CMAKE_EXPORT_COMPILE_COMMANDS ON) inside your CMakeLists.txt. In addition, it is better not to specify an absolute path for --compile-commands-dir of clangd, which is troublesome (It is still unable to resolve the project root directory). u should put that file in the root directory of the project.

Also on macOS you may specify global clangd options using the config file:

nvim ~/Library/Preferences/clangd/config.yaml

See #193 (comment) for details.

3. change the homebrew's .gitignore file by adding !/Cellar

clangd wouldn't recognize .gitignore, instead, this might lead to a warning issued by Homebrew when you run brew doctor. So better restore this to defaults.

I find something that may cause this problem:

1. when I use :pwd in the third party headers file's buffer, It print /opt/homebrew, not the actual path of the file

As @CharlesChiuGit metioned, this is the actual behavior when using project.nvim. The same issue here, nvim will not alter the working directory of clangd, and the working directory of clangd should be the root directory of your project.

2. in :LspInfo the root directory is /opt/homebrew

This would be expected, as your compile_command.json (or .clangd, compile_flags.txt) is in the wrong place, and this cannot be recognized by clangd thus it fell back to the default location.

3. originally, when I open nvim-tree in /opt/homebrew, It doesn't show Cellar dir, after I change the .gitignore file of homebrew by add !/Cellar, nvim-tree shows the Cellar dir, but clangd still throws the file not found error

IMO, buffer's path and the root directory and the wired Cellar directory cause this problem, but what happened actually?

As explained above. nvim will not alter the working directory of clangd, and the working directory of clangd should be the root directory of your project. They are not the same process with same PID.

So the solution is to put your compile_commands.json in the project root directory and ensure that it correctly passes the file search paths (-I, -L, -isysroot). (Usually, if the project can be built correctly, there should be no issue. But do make sure that this file is auto-generated by CMake)

You would then see (a small CMake project):

 Client: clangd (id: 1, bufnr: [19])
 	filetypes:       c, cpp, objc, objcpp, cuda, proto
 	autostart:       true
 -->    root directory:  /Users/jint/Desktop/test_proj
 	cmd:             clangd -j=12 --enable-config --query-driver=/usr/bin/clang++,/usr/bin/**/clang-*,/bin/clang,/bin/clang++,/usr/bin/gcc,/usr/bin/g++ --all-scopes-completion --background-index --clang-tidy --completion-parse=auto --completion-style=detailed --header-insertion-decorators --header-insertion=iwyu --limit-references=3000 --limit-results=500 --pch-storage=memory

You can also write a script to automate this process:

#!/bin/bash

if [[ -d build && -e CMakeLists.txt ]]; then
	cmake -S . -B ./build >/dev/null 2>&1
	cmake -S . -B ./build && make --directory ./build && ([[ -e ./build/compile_commands.json ]] && mv -f ./build/compile_commands.json ./)
else
	if [[ -e CMakeLists.txt ]]; then
		mkdir build
		cmake -S . -B ./build >/dev/null 2>&1
		cmake -S . -B ./build && make --directory ./build && ([[ -e ./build/compile_commands.json ]] && mv -f ./build/compile_commands.json ./)
	else
		printf "\033[1;31mError: \033[0mNot a valid CMake directory.\n"
	fi
fi

[Jint-lzxy]

A quick note here, Cellar is the place where Homebrew stores its assets 🥳 See https://docs.brew.sh/Formula-Cookbook#homebrew-terminology.

[CharlesChiuGit]

Wow, I learn a lot too. Maybe we should pin this issue.

edit: Oh no, we can only have three pinned issues. Maybe we can add a wiki page for some valuable issues and comments.

[Jint-lzxy]

1. I found out that for any homebrew installed package, there exist a .brew, ie. /opt/homebrew/Cellar/boost/1.81.0_1/.brew, I want /opt/homebrew/Cellar/boost/1.81.0_1/ to be the lsp's root dir.
2. I changed the pattern field in project.setup by adding ".brew" pattern

Is there something wrong with what I did? or anything should be done after changing the config file to make config file work?

What's the point of that? The purpose of root dir is to load user-defined projects quickly. Since you won't modify third-party libraries in most cases, then why would you set that as the project root directory? All you need is to inform clangd that "you need to search for headers and libs in that folder".

But it's true that the relevant documents are quite vague. I also encountered a lot of issues when I first started to use LSPs. If you have any further questions, feel free to point that out!

[csyJoy]

@Jint-lzxy Thank a lot for your detailed answer. after generating compile_command.json, I found I should copy this file to /opt/homebrew since the root directory now is /opt/hombrew Do I understand your answer correctly?

[Jint-lzxy]

@csyJoy Please read #468 (comment) first 👍

[Jint-lzxy]

I would recommend you to have a look at http://www.compsci.hunter.cuny.edu/~sweiss/resources/about_gcc.pdf to better understand why clangd is designed in this way

[Jint-lzxy]

To be short, you need to put this file in the root directory of your project

[csyJoy]

To be short, you need to put this file in the root directory of your project

I know, but if I only do this, the clangd still throw the error, after copying the compile_command.json to /opt/homebrew clangd works, so this made me confused.

[Jint-lzxy]

To be short, you need to put this file in the root directory of your project

I know, but if I only do this, the clangd still throw the error, after copying the compile_command.json to /opt/homebrew clangd works, so this made me confused.

What was the specific error - you mean "clangd doesn't find the header file correctly"?

Is it convenient for you to share your compile_commands.json here? We only need the compile instructions of one of the files.

[csyJoy]

If I don't copy compile_command.json file to /opt/homebrew the clangd throw error when I look up a third-party header file

my compile_commnad.json is:

[
{
  "directory": "/Users/csy/cpp_test/build",
  "command": "/opt/homebrew/opt/llvm/bin/clang++  -isystem /opt/homebrew/include -g -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.1.sdk -std=gnu++20 -o CMakeFiles/Foo.dir/src/foo.cpp.o -c /Users/csy/cpp_test/src/foo.cpp",
  "file": "/Users/csy/cpp_test/src/foo.cpp"
}
]

My CMakelist.txt is:

cmake_minimum_required(VERSION 3.5)

set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
set(CMAKE_CXX_COMPILER clang++)

project(Foo CXX)

set(CMAKE_BUILD_TYPE Debug)
set(CMAKE_CXX_STANDARD 20)
file(GLOB_RECURSE source_code CONFIGURE_DEPENDS src/*.cpp)
find_package(Boost REQUIRED)

add_executable(${PROJECT_NAME} ${source_code})
target_link_libraries(${PROJECT_NAME} PUBLIC Boost::boost)

if it helps.

[Jint-lzxy]

Is this file located at /opt/homebrew/include/boost/dynamic_bitset/dynamic_bitset.hpp?

[csyJoy]

Is this file located at /opt/homebrew/include/boost/dynamic_bitset/dynamic_bitset.hpp?

I checked the /opt/homebrew/include directory there is a soft link file to the actual location, /opt/homebrew/include/boost/dynamic_bitset/dynamic_bitset.hpp exist but soft link to /opt/homebrew/Cellar/boost/1.81.0_1/include/boost/dynamic_bitset/dynamic_bitset.hpp

[Jint-lzxy]

Try

export CPATH="${CPATH}:/opt/homebrew/include"
export BOOST_ROOT="${BOOST_ROOT:-$(brew --prefix boost)}"
export BOOST_INCLUDE_DIRS="${BOOST_ROOT}/include"
export BOOST_LIBRARY_DIRS="${BOOST_ROOT}/lib"

before starting neovim to see if that helps.

If this works, then you may specify the include dir first:

include_directories(<your-include-search-dir>)

[csyJoy]

Try

export CPATH="${CPATH}:/opt/homebrew/include"
export BOOST_ROOT="${BOOST_ROOT:-$(brew --prefix boost)}"
export BOOST_INCLUDE_DIRS="${BOOST_ROOT}/include"
export BOOST_LIBRARY_DIRS="${BOOST_ROOT}/lib"

before starting neovim to see if that helps.

If this works, then you may specify the include dir first:

include_directories(<your-include-search-dir>)

I export the variable and it works, but when I try to use target_include_directories(${PROJECT_NAME} PUBLIC /opt/homebrew/include) clangd still throw this error

[Jint-lzxy]

Then this should have nothing to do with clangd. Is your compile_commands.json in the same directory as clangd's root directory?

[csyJoy]

Then this should have nothing to do with clangd. Is your compile_commands.json in the same directory as clangd's root directory?

Yes, I just use your shell script to generate compile_command.json file

[Jint-lzxy]

If you can build this project, it is recommended that you add -I/opt/homebrew/include to ~/Library/Preferences/clangd/config.yaml's CompileFlags.Add field, as sometimes clangd cannot correctly index all header files

[Jint-lzxy]

See #193 (comment)

[csyJoy]

If you can build this project, it is recommended that you add -I/opt/homebrew/include to ~/Library/Preferences/clangd/config.yaml's CompileFlags.Add field, as sometimes clangd cannot correctly index all header files

Finally It works fine 🎉 Thank you all for the help, I learn a lot from this issue, I think I should close this issue now.