ale does not use compile_commands.json file in C++

[cridemichel]

Information

VIM version

VIM - Vi IMproved 8.2 (2019 Dec 12, compiled Aug 15 2020 20:27:49)
macOS version
Included patches: 1-1456

Operating System:

mac osx Catilina 10.15.6

What went wrong

In my c++ projects ALE used compile_commands.json file but after some recent update it is not using it anymore.
In my c projects compile_commands.json file is still used as expected.

:ALEInfo

 Current Filetype: cpp
Available Linters: ['cc', 'ccls', 'clangcheck', 'clangd', 'clangtidy', 'clazy', 'cppcheck', 'cpplint', 'cquery', 'flawfinder']
   Linter Aliases:
'cc' -> ['gcc', 'clang', 'g++', 'clang++']
  Enabled Linters: ['cc']
 Suggested Fixers:
  'astyle' - Fix C/C++ with astyle.
  'clang-format' - Fix C/C++ and cuda files with clang-format.
  'clangtidy' - Fix C/C++ and ObjectiveC files with clang-tidy.
  'remove_trailing_lines' - Remove all blank lines at the end of a file.
  'trim_whitespace' - Remove all trailing whitespace characters at the end of every line.
  'uncrustify' - Fix C, C++, C#, ObjectiveC, ObjectiveC++, D, Java, Pawn, and VALA files with uncrustify.
 Linter Variables:
let g:ale_cpp_cc_executable = '<auto>'
let g:ale_cpp_cc_options = '-std=c++17 -Wall'
let g:ale_cpp_clang_executable = '/usr/local/opt/llvm/bin/clang'
let g:ale_cpp_clang_options = '-std=c++17 -Wall'
let g:ale_cpp_gcc_executable = 'gcc-10'
let g:ale_cpp_gcc_options = '-std=c++17 -Wall'
 Global Variables:
let g:ale_cache_executable_check_failures = v:null
let g:ale_change_sign_column_color = 0
let g:ale_command_wrapper = ''
let g:ale_completion_delay = v:null
let g:ale_completion_enabled = 0
let g:ale_completion_max_suggestions = v:null
let g:ale_echo_cursor = 1
let g:ale_echo_msg_error_str = 'Error'
let g:ale_echo_msg_format = '%code: %%s'
let g:ale_echo_msg_info_str = 'Info'
let g:ale_echo_msg_warning_str = 'Warning'
let g:ale_enabled = 1
let g:ale_fix_on_save = 0
let g:ale_fixers = {'tex': ['remove_trailing_lines', 'trim_whitespace'], 'python': ['black', 'isort']}
let g:ale_history_enabled = 1
let g:ale_history_log_output = 1
let g:ale_keep_list_window_open = 0
let g:ale_lint_delay = 200
let g:ale_lint_on_enter = 1
let g:ale_lint_on_filetype_changed = 1
let g:ale_lint_on_insert_leave = 1
let g:ale_lint_on_save = 1
let g:ale_lint_on_text_changed = 'normal'
let g:ale_linter_aliases = {}
let g:ale_linters = {'c': ['gcc', 'clang'], 'tex': ['chktex', 'lacheck', 'texlab'], 'cpp': ['g++', 'clang++'], 'python': ['mypy', 'py
lint', 'pyflakes']}
let g:ale_linters_explicit = 0
let g:ale_list_vertical = 0
let g:ale_list_window_size = 10
let g:ale_loclist_msg_format = '%code: %%s'
let g:ale_lsp_root = {'texlab': '/Users/demichel/MDsimul++/'}
let g:ale_max_buffer_history_size = 20
let g:ale_max_signs = -1
let g:ale_maximum_file_size = v:null
let g:ale_open_list = 0
let g:ale_pattern_options = v:null
let g:ale_pattern_options_enabled = v:null
let g:ale_set_balloons = 1
let g:ale_set_highlights = 1
let g:ale_set_loclist = 1
let g:ale_set_quickfix = 0
let g:ale_set_signs = 1
let g:ale_sign_column_always = 0
let g:ale_sign_error = '>>'
let g:ale_sign_info = '--'
let g:ale_sign_offset = 1000000
let g:ale_sign_style_error = '>>'
let g:ale_sign_style_warning = '--'
let g:ale_sign_warning = '--'
let g:ale_sign_highlight_linenrs = 0
let g:ale_statusline_format = v:null
let g:ale_type_map = {}
let g:ale_use_global_executables = v:null
let g:ale_virtualtext_cursor = 0
let g:ale_warn_about_trailing_blank_lines = 1
let g:ale_warn_about_trailing_whitespace = 1
  Command History:
(finished - exit code 0) ['/usr/local/bin/bash', '-i', '-c', '''clang++'' -S -x c++ -o /dev/null -iquote ''/Users/demichel/MDsimul++/
mdlib'' -std=c++17 -Wall - < ''/var/folders/gc/2x7dz9ps7m38rc7ts03c30ch0000gn/T/vawIPmS/10/polyhedron.hpp''']
<<<OUTPUT STARTS>>>
bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell
<<<OUTPUT ENDS>>>

[w0rp]

You may need to use let g:ale_cpp_cc_executable = '/usr/local/opt/llvm/bin/clang'. See #3299.

Where does the compile_commands.json file live in your project?

[cridemichel]

Dear w0rp,
thank you very much for the prompt reply,
setting g:ale_cpp_cc_executable as you suggested does not do the trick...
all files of my c++ projects (including the compile_commands.json file) are in the same folder
best Cristiano

[moshiba]

setting g:ale_c_build_dir_names or g:ale_c_build_dir might help

[cridemichel]

setting g:ale_c_build_dir_names or g:ale_c_build_dir might help

ok but without these settings previous versions of ALE worked with my C++ projects and with my C projects it still works...

[moshiba]

I don't know the root cause but in your case setting g:ale_c_build_dir_names or soft link the compile command file solves the problem.
The default is ['build', 'bin'], and your compile_commands.json lives in the project root, maybe try adding '.' to the list?

[cridemichel]

I don't know the root cause but in your case setting g:ale_c_build_dir_names or soft link the compile command file solves the problem.
The default is ['build', 'bin'], and your compile_commands.json lives in the project root, maybe try adding '.' to the list?

adding '.' to g:ale_c_build_dir_names does not solve my issue...

[Kimplul]

If I may butt in here, I had the same issue. In my case it was because the compile_commands.json file set the 'file'-attribute to a relative path, i.e.

...
"directory": "/usr/src/linux-headers-5.7.0-2-amd64",
"file": "../../../home/user/whatever/code.c"
...

I 'fixed' it by changing line 351 in autoload/ale/c.vim from

bufnr(l:item.file) is a:buffer

to

bufnr(l:full_path) is a:buffer

where l:full_path is

        let l:full_path = l:item.file

        if ( stridx(l:item.file, '/') != 0 )
            let l:full_path = l:item.directory . s:sep . l:full_path
        endif

I don't know if this is a botch or what, but it's worked in my projects.

[w0rp]

@Kimplul Could you report that as an issue? If issues aren't reported, they won't be fixed. I think that issue can be fixed by resolving the path from compile commands relative to the directory key. The ale#path#GetAbsPath function can be used to get an absolute path from a directory and a possibly relative path.

[w0rp]

@cridemichel Can you produce a minimal C++ project that repeats the bug? If you can create a GitHub repository that contains a compile_commands.json file in the same place it's in, in your project, with similar enough flags, and let me know the compiler version you're using, I might be able to repeat the bug myself.

[cridemichel]

Dear w0rp,
I have invited you as collaborator of my C++ repository, you can go to folder semidisks and there you will find
a "compile_commands.json" file for testing. To reproduce the bug you can edit the file "particle_sd.hpp",
which will give you errors, since the "-std=c++17" flag is provided by compile_commands.json,

best C.

[w0rp]

@cridemichel There's no compile_commands.json file in the repository. Do you run some command other other to generate it?

[cridemichel]

yes sorry I generated it ...I have just committed it to the repository

@cridemichel There's no compile_commands.json file in the repository. Do you run some command other other to generate it?

[cridemichel]

Dear w0rp,
did you manage to do some test?
best C.

[w0rp]

I have fixed your issue now, similar to how @Kimplul mentioned. ALE now resolves file relative to directory for each entry for finding flags for files, and ALE now also tries to find absolute path matches for files and directories before trying to find entries based on the basenames of files and directories, in case you have multiple directories or files with the same names in the same compile_commands.json file.

[cridemichel]

it works! thank you very much!!

best C.

[cridemichel]

this issue seems to be back again with latest ALE. In my project I have the following compile_commands.json file:

[
{
  "directory": "/Users/demichel/MDsimul++/patchy_semidisks",
  "command": "c++ -c -Wall -std=c++17 -O3 -I /usr/local/include/ -o expand expand.cpp",
  "file": "expand.cpp"
},

{
  "directory": "/Users/demichel/MDsimul++/patchy_semidisks",
  "command": "c++ -c -Wall -std=c++17 -O3 -I /usr/local/include/ -o mcsim_psd mcsim_psd.cpp",
  "file": "mcsim_psd.cpp"
},

{
  "directory": "/Users/demichel/MDsimul++/patchy_semidisks",
  "command": "c++ -Wall -std=c++17 -O3 -I /usr/local/include/ -c mcsim_psd.hpp",
  "file": "mcsim_psd.hpp"
},

{
  "directory": "/Users/demichel/MDsimul++/patchy_semidisks",
  "command": "c++ -Wall -std=c++17 -O3 -I /usr/local/include/ -c ../mdlib/mc.hpp",
  "file": "../mdlib/mc.hpp"
},

{
  "directory": "/Users/demichel/MDsimul++/patchy_semidisks",
  "command": "c++ -Wall -std=c++17 -O3 -I /usr/local/include/ -c ../mdlib/pvector.hpp",
  "file": "../mdlib/pvector.hpp"
},

{
  "directory": "/Users/demichel/MDsimul++/patchy_semidisks",
  "command": "c++ -Wall -std=c++17 -O3 -I /usr/local/include/ -c ../mdlib/randnumgen.hpp",
  "file": "../mdlib/randnumgen.hpp"
},

{
  "directory": "/Users/demichel/MDsimul++/patchy_semidisks",
  "command": "c++ -Wall -std=c++17 -O3 -I /usr/local/include/ -c ../mdlib/random_extra.hpp",
  "file": "../mdlib/random_extra.hpp"
},

{
  "directory": "/Users/demichel/MDsimul++/patchy_semidisks",
  "command": "c++ -Wall -std=c++17 -O3 -I /usr/local/include/ -c ../mdlib/pcg_random.hpp",
  "file": "../mdlib/pcg_random.hpp"
},

{
  "directory": "/Users/demichel/MDsimul++/patchy_semidisks",
  "command": "c++ -Wall -std=c++17 -O3 -I /usr/local/include/ -c ../mdlib/pcg_extras.hpp",
  "file": "../mdlib/pcg_extras.hpp"
},

{
  "directory": "/Users/demichel/MDsimul++/patchy_semidisks",
  "command": "c++ -Wall -std=c++17 -O3 -I /usr/local/include/ -c ../mdlib/pcg_uint128.hpp",
  "file": "../mdlib/pcg_uint128.hpp"
},

{
  "directory": "/Users/demichel/MDsimul++/patchy_semidisks",
  "command": "c++ -Wall -std=c++17 -O3 -I /usr/local/include/ -c ../mdlib/pmatrix.hpp",
  "file": "../mdlib/pmatrix.hpp"
},

{
  "directory": "/Users/demichel/MDsimul++/patchy_semidisks",
  "command": "c++ -Wall -std=c++17 -O3 -I /usr/local/include/ -c ../mdlib/linked_cell_lists_3d.hpp",
  "file": "../mdlib/linked_cell_lists_3d.hpp"
},

{
  "directory": "/Users/demichel/MDsimul++/patchy_semidisks",
  "command": "c++ -Wall -std=c++17 -O3 -I /usr/local/include/ -c ../mdlib/particle.hpp",
  "file": "../mdlib/particle.hpp"
},

{
  "directory": "/Users/demichel/MDsimul++/patchy_semidisks",
  "command": "c++ -Wall -std=c++17 -O3 -I /usr/local/include/ -c ../mdlib/params.hpp",
  "file": "../mdlib/params.hpp"
},

{
  "directory": "/Users/demichel/MDsimul++/patchy_semidisks",
  "command": "c++ -Wall -std=c++17 -O3 -I /usr/local/include/ -c ../mdlib/parser.hpp",
  "file": "../mdlib/parser.hpp"
},

{
  "directory": "/Users/demichel/MDsimul++/patchy_semidisks",
  "command": "c++ -Wall -std=c++17 -O3 -I /usr/local/include/ -c ../mdlib/mcsim.hpp",
  "file": "../mdlib/mcsim.hpp"
},

{
  "directory": "/Users/demichel/MDsimul++/patchy_semidisks",
  "command": "c++ -Wall -std=c++17 -O3 -I /usr/local/include/ -c ../mdlib/nnl_3d.hpp",
  "file": "../mdlib/nnl_3d.hpp"
},

{
  "directory": "/Users/demichel/MDsimul++/patchy_semidisks",
  "command": "c++ -Wall -std=c++17 -O3 -I /usr/local/include/ -c ../mdlib/patchy.hpp",
  "file": "../mdlib/patchy.hpp"
},

{
  "directory": "/Users/demichel/MDsimul++/patchy_semidisks",
  "command": "c++ -Wall -std=c++17 -O3 -I /usr/local/include/ -c ../mdlib/particleset.hpp",
  "file": "../mdlib/particleset.hpp"
},

{
  "directory": "/Users/demichel/MDsimul++/patchy_semidisks",
  "command": "c++ -Wall -std=c++17 -O3 -I /usr/local/include/ -c ../mdlib/mcsim_mol.hpp",
  "file": "../mdlib/mcsim_mol.hpp"
},

{
  "directory": "/Users/demichel/MDsimul++/patchy_semidisks",
  "command": "c++ -Wall -std=c++17 -O3 -I /usr/local/include/ -c ../mdlib/mcsim_molp_ps.hpp",
  "file": "../mdlib/mcsim_molp_ps.hpp"
},

{
  "directory": "/Users/demichel/MDsimul++/patchy_semidisks",
  "command": "c++ -Wall -std=c++17 -O3 -I /usr/local/include/ -c particle_psd.hpp",
  "file": "particle_psd.hpp"
},

{
  "directory": "/Users/demichel/MDsimul++/patchy_semidisks",
  "command": "c++ -Wall -std=c++17 -O3 -I /usr/local/include/ -c ../semidisks/particle_sd.hpp",
  "file": "../semidisks/particle_sd.hpp"
},

{
  "directory": "/Users/demichel/MDsimul++/patchy_semidisks",
  "command": "c++ -Wall -std=c++17 -O3 -I /usr/local/include/ -c ../hard_boxes/particle_boxes.hpp",
  "file": "../hard_boxes/particle_boxes.hpp"
},

{
  "directory": "/Users/demichel/MDsimul++/patchy_semidisks",
  "command": "c++ -Wall -std=c++17 -O3 -I /usr/local/include/ -c ../hard_boxes/params_boxes.hpp",
  "file": "../hard_boxes/params_boxes.hpp"
},

{
  "directory": "/Users/demichel/MDsimul++/patchy_semidisks",
  "command": "c++ -Wall -std=c++17 -O3 -I /usr/local/include/ -c ../mdlib/polyhedron.hpp",
  "file": "../mdlib/polyhedron.hpp"
},

{
  "directory": "/Users/demichel/MDsimul++/patchy_semidisks",
  "command": "c++ -Wall -std=c++17 -O3 -I /usr/local/include/ -c ../semidisks/params_sd.hpp",
  "file": "../semidisks/params_sd.hpp"
},

{
  "directory": "/Users/demichel/MDsimul++/patchy_semidisks",
  "command": "c++ -Wall -std=c++17 -O3 -I /usr/local/include/ -c params_psd.hpp",
  "file": "params_psd.hpp"
}
]

and all entries with a relative path to current dir (/Users/demichel/MDsimul++/patchy_semidisks) are ignored by ALE.
For example, if I open the file "mcsim.hpp" which is in the directory "../mdlib" the compilation flags are ignore by ALE,

best Cristiano

[laoshaw]

same here, ALE just uses my .vimrc settings and totally ignores compile_commands.json in the same directory(top of the project)

[themightyoarfish]

Still an issue with current version It seems. How can we help debug?

[themightyoarfish]

My compile_commands.json has only absolute paths, so those cannot be at fault.