Add support for compile commands json

[agauniyal]

Feature Request - Most of the build systems nowadays have support for generating compilation database file, vscode cpp plugin should also support it. For more information - http://clang.llvm.org/docs/JSONCompilationDatabase.html

[d-xo]

Would love to see this, setting up include paths / compiler options (especially on large projects) is one of the (very few) pain points for c++ development in vs code.

[imikejackson]

I gave up setting up our CMake based project because I would have to manually enter a large number of include paths into the settings file. If there were support for the compile_commands file that CMake can generate automatically the setup would be super simple. Here is hoping this could be integrated into VSCode in the near future.

[MathiasMagnus]

Indeed, just came across the exact same issue. Playing along with cmake-server mode, this type of info isn't hard to obtain. Its only a matter of coming up with the responsibility, of which extension should take care of populating the list of include files.

[imikejackson]

So I wrote a short c++ program to parse the compile_commands.json file and rewrite the c_cpp_properties.json file. This still did not allow any completions beyond the basic "Ctags" style completions. My guess (and reading around a bit) is that the clang based code completion is not a full blown code completion like in other IDE's such as Xcode, QtCreator and Visual Studio. If anyone wants the c++ code they are welcome to the hack.

[MathiasMagnus]

The C++ Docs page still suggests that

Q: IntelliSense isn't working.
A: In this release, IntelliSense isn't supported. We plan to enable this and other features in future releases.

Which to me implies that full-blown IntelliSense is known not to work. I might be wrong about this, but AFAIK IntelliSense in big brother VS is done through Roslyn which would have to be ported over to Linux/OSX as well as hooking it into the editor of Code.

Correct me if I'm wrong.

[sean-mcmanus]

Yes, full-blown IntelliSense isn't finished yet. Roslyn is a .NET C#/VB compiler, and not a C++ compiler.

[d-xo]

Just as a note, the ycmd plugin from vscode offers this functionality (as well as proper autocomplete via clang):

https://github.com/richard1122/vscode-youcompleteme

[MathiasMagnus]

I know, hope never dies, but could you give us a heads up where this feature is inside the pipeline? There have been no words from it from the CMake side (as far as @vector-of-bool is concerned).

[sean-mcmanus]

Yes, it's still in the pipeline, near the front.

[agauniyal]

@xwvvvvwx but it has some irritating bugs, check out the issues.

[vector-of-bool]

I'd really like there to be some tighter integration with the build system. With the capabilities brought by cmake-server, I think that support for C and C++ in VSCode can be brought to parity with more heavy-duty IDEs. I also work in an environment with many custom include paths and compile flags, and when those are not used when processing the source files the resulting editing experience is next to useless. The build parameters are available for the tools to use, it's just a matter of making use of them in the tools that already exist.

I've been working on exposing an extension API from my own CMake extension specifically to help support native tooling, and I'd be really happy to see tools use the information provided by the build system rather than some other source. In the end, the project's build system (like CMake) is the ultimate source of knowledge on how to process the project and using any other source of information will result in a sub-par tooling experience.

[sean-mcmanus]

Yes, I agree. Visual Studio 2017 supports CMake in the open folder scenario with IntelliSense, but that only runs on Windows. Support for CMake (in the C++ extension) is further down the pipeline for VS Code.

[defunctzombie]

Is there a way to hook into setting the cpp include paths via an extension? This would allow temporary work-arounds to create simple extensions that would parse compile_commands files for include paths to extend the built-in c++ intellisense system.

[sean-mcmanus]

@defunctzombie You just read/parse/write the c_cpp_properties.json. You can look in the out/src/LanguageServer/C_Cpp_ConfigurtionProperties.js file to see how we do it.

[defunctzombie]

@sean-mcmanus out/src/LanguageServer/C_Cpp_ConfigurtionProperties.js would this be in the installed extension on my computer or is this referencing a file in some repo?

[sean-mcmanus]

@defunctzombie In the installed extensions folder, i.e. in the /Users/<user>/.vscode/extensions/ms-vscode.cpptools-0.11.0 folder (might vary somewhat by OS).

[defunctzombie]

@sean-mcmanus Found the file and looked through it. I see how and where it sets up some features on the language server but would appreciate a pointer on which API call it makes to add includes. I found the vscode extension API docs but did not find a mention of adding include paths.

[sean-mcmanus]

@defunctzombie Do you see the parsePropertiesFile() method? That file calls fs.readFileSync and JSON.parse to read and parse our c_cpp_properties.json file. So you just use configurations[i].includePath or configurations[i].browse.path to access the list of includePaths (the browse.path is for our symbol engine, which is expected to match the locations of all cpp/header files for which symbols are desired, and the other one is for our IntelliSense engine, which is expected to match your compiler settings for the particular C/C++ files you want accurate error squiggles, hover, etc. for (when the IntelliSenseEngine setting is Default).

[defunctzombie]

@sean-mcmanus Thank you! Just to clarify so I don't travel down a bad path, vscode will scan any paths I add to configurations[i].includePath and configurations[i].browse.path. There is no other API call I need to make to inform vscode of the configuration.

[sean-mcmanus]

@defunctzombie No other API needs to be called. We have a file watcher on the c_cpp_properties.json that will run whenever the file is changed, so the changes should be picked up.

[MathiasMagnus]

@sean-mcmanus

Yes, it's still in the pipeline, near the front.

This statement was quite a while back (Feb 17). Also, the one month radio silence doesn't do much good for hopes on finally being able to use VS Code as a serious alternative to anything else. (That is for C++ development)

[sean-mcmanus]

@MathiasMagnus What feature are you talking about? Better autocomplete? I think that's what I was referring to. We're close to releasing it.

[MathiasMagnus]

@sean-mcmanus I was talking about the feature this issue refers to: supporting compile_commands.json Manually adding include paths is tedious, as is adding targets for debugging. @vector-of-bool has been pushing as well for the MS extension to support parsing compile_commands.json (which is ought to be the canonical way of detecting external build systems). CMake Tools would take on the role of populating compile_commands.json, but there's no use if the C++ extension does not care.

[jhasse]

Btw: CMake isn't the only build system with support for generating compile_commands.json, Waf can also do this: https://github.com/waf-project/waf/blob/master/waflib/extras/clang_compilation_database.py

[bobbrow]

We have begun designing how support for compile_commands.json should work in our extension. The current plan is to add a new property to the configuration objects in c_cpp_properties.json that tells the extension where to find the compilation database. If it is missing or corrupt, we will use the includePath and defines instead.

What would you expect the behavior to be if you open a source file not described by the compilation database (or a header file not included by a source file in the database)?

• Use the same includePath/defines as another source/header file in the same folder?
• Use the includePath/defines in the c_cpp_properties.json file?
• Use just the default system include path (similar to what we do for opening files without opening a workspace)

[vector-of-bool]

When I do look up for a file in the compdb, I do a fairly simple path normalization so that headers and sources resolve the same. Like this

[db4]

This feature as does not work for me for version 0.14.2:

"...\compile_commands.json" could not be parsed. Using 'includePath' settings in c_cpp_properties.json.

compile_commands.json is generated by cmake 3.9.4. It exists and at first glance is correct.

Any idea what could be wrong? Can I see/enable some debugging output from cpptools to figure out a problem?

[MathiasMagnus]

It seems Code 1.18 introduced workspaceFolder instead of workspaceRoot, and thus the extension looks for this file through a non-existing helper var. Problem is, the extension might not even find c_cpp_properties.json, so you might not be able to instruct it otherwise either. You can try though.

[db4]

No, that's what I checked first. If the file does not exist, the error message would be

"...\compile_commands.json" could not be found. Using 'includePath' settings in c_cpp_properties.json"

[bobbrow]

Can you please open a new issue for this and provide some information about your workspace and c_cpp_properties.json file? You can try trimming out all but one entry in the compile commands file to see if a simple case still works for you (which is easier to validate).

${workspaceRoot} is still a valid variable. The language server continues to respect it.

[g-arjones]

Is there a way to set a default "compileCommands" in the user's settings file? All of my projects have the same relative path (e.g ${workspaceRoot}/build/compile_commands.json) and I feel that I should not have to set the same thing for all of them.. Thanks!

[bobbrow]

We are considering moving c_cpp_properties.json into settings.json at which point this will be possible.

[g-arjones]

Great! Do you suggest any workaround in the meantime?

[bobbrow]

Unfortunately there is no workaround at this time.

[g-arjones]

No problem, thank you :)

[loaden]

It's seems broken now. VSCode:1.21.1, CppTools:0.15.0

[sean-mcmanus]

@loaden https://github.com/Microsoft/vscode-cpptools/releases/tag/v0.16.0-insiders2 has a workaround for some issues, such as querying the system defines and enabling a separate compilerPath to be set if the first arg in the compile_commands.json command isn't a compiler.

[loaden]

@sean-mcmanus thans for the tip. I've upgraded to 0.16-insiders2 now, but still can not get any help.
I thought paste my json part in here and hope the solved way.

{
            "name": "Win32",
            "includePath": [
                "${workspaceFolder}"
            ],
            "defines": [
                "_DEBUG",
                "UNICODE",
                "_UNICODE"
            ],
            "compileCommands": "${workspaceFolder}/build/compile_commands.json",
            "intelliSenseMode": "clang-x64",
            "browse": {
                "path": [
                    "${workspaceFolder}"
                ],
                "limitSymbolsToIncludedHeaders": true,
                "databaseFilename": ""
            },
            "compilerPath": "",
            "cStandard": "c11",
            "cppStandard": "c++17"
        }

any the compile_commands.json

[
{
  "directory": "d:/qpsoft/projects/test/build",
  "command": "D:\\Office2016\\VSCode\\VSCode_64-bit\\MinGW\\bin\\g++.exe    -g   -o CMakeFiles\\Test.dir\\test.cpp.obj -c D:\\qpSOFT\\Projects\\Test\\test.cpp",
  "file": "D:/qpSOFT/Projects/Test/test.cpp"
}
]

[sean-mcmanus]

@loaden 0.16.0 is released, but it looks like we have a bug and compilerPath needs to be set on Windows or it'll use the Windows includes/defines instead of the one in the compiler_commands.json. Does setting the compilerPath fix the issue for you?

[loaden]

@sean-mcmanus yes, it's fix the issue after set compilerPath like this, and must be absolute path. only 'g++.exe' does not work.
"compilerPath": "D:/qpSOFT/VSCode/VSCode_64-bit/MinGW/bin/g++.exe",

[defunctzombie]

@sean-mcmanus A feature that would be immensely useful as a user of the compile commands feature is a way to inspect what the intellisense system "thinks" of the compile_commands.json file. For starters this would surface whether the file has any syntax violations (or other blockers to parsing), it could also include ways to query or discover the learned include paths and settings. Right now; when this feature does not work I find it a bit of a black box to figure out why.

[sean-mcmanus]

@defunctzombie Setting loggingLevel to "6" will show the end result of the settings we pass to our IntelliSense processes, but on Linux/Mac there's a race condition deadlock that can occur if a child process is forked while another thread is logging, so it's recommended to reduce the logging after you're done with it. Is that sufficient to diagnose what is going wrong? We'll consider adding a better diagnostics system later and adding more logging when errors occur. We expect compile_commands.json to be generated by a tool and to be syntax error free...are you hand generating it?

[defunctzombie]

We expect compile_commands.json to be generated by a tool and to be syntax error free...are you hand generating it?

No, autogenerated, but sometimes tools have errors too :)

[defunctzombie]

@sean-mcmanus Thanks; Ill try that next time I encounter an issue. What would be helpful is to see the information (include paths, etc) on a per file basis (maybe the logging level will do that).

[juxeii]

Sorry to say, but for us VSC is still a no go when include paths are not found via cmake. We will eventually come back to this otherwise nice editor, but in the meantime go with other competitors.

[sean-mcmanus]

@juxeii Do you mean you're having problems when files are not found in compile_commands.json and it falls back to using the includePath? Do you believe we have a bug in that scenario? We might need more repro details.

[MathiasMagnus]

@juexii I have been using VSC as my primary C++ dev platform for a few months now and it does resolve include paths when used with CMake Tools. What's not working?

[juxeii]

First, we have a huge code and complex code base.
With a competitor tool, we can just open the folder with CMakeLists.txt and after a while everything is indexed and found.
With VSC its not so easy.
You have to install CMake Tools and CMake Tools Helper at first glance.
But for us it was not clear if CMake Tools Helper is even working a promised.
So we took the route with setting the path to compile_commands.json.
After some waiting time for Intellisense, almost no errors were reported on the #include statements.
Everything under the projects working directory is found and we can jump to these files.
The problems start with files which are outside of the actual project and are all over the place on the server file system.
The include paths for these are all listed in the compile_commands.json file, so they should be indexed too by VSC?
Problem is we cannot jump/open the files with F12, VSC keeps saying "no definition found"(or similar).
As an artificial example:

#include <Blah.h>
SomeType var;

Here, Blah.h is a file which is not under the project root folder; it resides somewhere on the file system.
VSC does not underline this include statement, yet when trying to jump to the file with F12 it says no definition found.
Strangely enough, the type SomeType is defined only in Blah.h and when hovering over it VSC shows a little window with the actual typedef definition.
So clearly, VSC has somehow parsed Blah.h, but it just won't jump to it.
Is it because it is an *.h file?
Is it because it is an include with "<" bracket type?
Is it because it is not under the project root path?

With our complex project setup I cannot provide a simple example.
Is there some kind of logging mechanism where we can see whats really going on?
We also noticed that each time we start VSC the indexing takes the same amount of time.
With our huge codebase this gets really in the way of productivity(competitor tool builds some kind of cache here).
These are all pain points and we agreed that we should give VSC another try next year.
Personally, I think VSC is a great editor :)

[bobbrow]

Got it. Yes, we know this scenario doesn't currently work right now. The code navigation stuff is controlled by the "browse.path" setting in your c_cpp_properties.json file. There's an issue open to automatically add the include paths from the compile_commands.json file to this setting for you, but we haven't implemented that yet (#1715). I recommend you go upvote that one.

Are there any other blockers for you?

[sean-mcmanus]

@juxeii Setting the browse.path is expected to be relatively easy to set manually because it's recursive, so you should only need to supply the roots of your projects. What do you mean by "indexing takes the same amount of time"? We have a database that stores the parsed symbol info, and we just scan the directories for newer files on startup, which should be fast. What does the database icon in the bottom right say when you hover? You can enable more logging using the hidden values of "6" or "7" (but 7 is for symbol parsing info, which may spew a lot and slow things down).

[juxeii]

@sean-mcmanus , @bobbrow
Thanks for the information on #1715, I will upvote that one.
I havent really looked into it, but colleages complainted about that indexing takes always the same amount of time, no matter if the project was already indexed before.
We would expect that after a restart of VSC(after indexing) the experienced workflow is much faster.
Anyway, we will wait and see what's the situation after #1715 has been resolved.

[sean-mcmanus]

@juxeii Yes, "indexing" is supposed to be faster after the database been built, unless something is causing the database to be corrupted/reset. With the large open source chromium repo with 500k files, the scanning time on reopen is around 20 seconds.

It's possible you're experiencing a delay from the red flame on file open (IntelliSense updating), which is not cached.

[juxeii]

As a remark: for VSC to be a good editor for CMake projects it would be helpful to have a look at how CLion does this. You just open a folder with CMakeLists.txt and the rest is done automatically. There is no need to fiddle around with compile_commands.json.
I would expect a similar approach for VSC, where new users just open the project and experienced ones can do all kind of complicated settings at their will.

[jhasse]

Not everyone is using VS Code with CMake though.

[MathiasMagnus]

@juxeii CLion is a nice experience, but it has it's downsides. The fact that it can author the scripts for you is OK, the fact that it will write into your scripts (or that you are allowed to write into generated stuff, depends on the POV) is not a smart choice. Also, it still does not have CMake server support AFAIK. (Correct me if I'm wrong, I checked it out ~1 year ago.)

That being said, big brother Visual Studio provides you with just what you wrote and even more. It's by far the best CMake experience out there. (xplat squigglies, build locally inside WSL, etc.) If only I were able to template the default CMakeSettings.json, I'd never look back if it comes to solo C++ development. VS Code is useful when there are many languages involved. C++, PowerShell, Python, LaTeX… unfortunately, these do not integrate well. The promoted LaTeX extension is not CMake aware and compile_commands.json does not export custom commands, so… the fact that VS Code allows me to setup the default values to some of the c_cpp_properties.json saves me a lot of time. Setup and forget.

Until WG15 doesn't come up with something better, most of C++ developers are stuck with CMake. Hand authoring tasks.json for every homework handed in to me is not an option for me. It's sure good for people who work on one project for 2 months, but I open 5-10 projects every day, and they all have CMakeLists.txt. I want want to do exactly what @juexii said. Setup once and forget. VS Code is close, but some stuff are still missing. The lack of code navigation in user includes is the biggest pain.