Extension doesn't handle compile_commands where the "command" doesn't begin with the compiler

[jwatt]

It seems that when compile_commands.json is used then VSCode will only use include paths specified in that file, and will fail to use the clang compilers built in include paths. My understanding is that compile_commands.json should only contain the flags that are passed to the compiler though, and there should be no need for it to contain compiler built-in include paths, defines, etc.

This issues causes includes such as:

#include <new>

to fail. That header file should be found under:

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1

I'm on macOS 10.13.1 using VSCode 1.19.0-insider with C/C++ Tools 0.14.2.

[jwatt]

(I should note that includePath is still set alongside compileCommands in my c_cpp_properties.json and contains /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1. So clearly the includes in includePath are not being used in addition to those in compile_commands.json.)

[bobbrow]

We actually do query clang (or whatever compiler is in the "command" property) to determine the built-in includes and we pass those in alongside the paths listed in the "command" property, but something must be going wrong. We should add some logging to figure out what paths are being chosen for you.

[jwatt]

Ah, in that case I bet that you assume that the first thing in the value of the "command" property is the compiler. In my case, that's not true since I'm speeding up recompiles using sccache (similar to ccache). So my "command" properties look like this:

"command": "/path/to/sccache /usr/bin/clang++ ..."

I'll try removing the /path/to/sccache from the "command" property values and see if that fixes things.

[bobbrow]

Ok. Didn't see that coming! 😅

[jwatt]

That does indeed seem to fix things. It's unclear to me whether tools that process compile_commands.json files should be able to handle this, or whether tools that generate compile_commands.json files should avoid inserting ccache etc. into the "commands" properties. From my point of view it would be nice if VSCode could handle this though.

[jwatt]

That said, I'd much, much rather see other issues like #16 fixed than this! :)

[dapirian]

I (and a bunch of people I work with) have the same issue, with ccache. If I modify compile_commands.json to use /usr/bin/clang++ instead of ccache it maybe works? I can't tell if it just takes forever to process my 100+mb compile_commands.json. Some log message or feedback of some kind would be very useful.

If I don't modify my compile_commands.json I get Failed to query default include paths for use with compile_commands.json in the output panel. Ironically, the command doesn't even use the default include paths (they're added explicitly), but it appears if this failure happens the extension just stops attempting to process compile_commands? For me, if it just ignored that 'failure' and kept going, all would be well in the world.

Also, it would be nice if it auto-found (and used) compile_commands if it's in the workspace root :)

[bobbrow]

@dapirian, can you share an example entry from your compile_commands.json file? If you are working on a closed-source project an anonymized entry would be fine.

[bobbrow]

(oops, wrong button)

[dapirian]

Sorry, can't really paste an example :( But basically something like:

    {
        "directory": "/home/somewhere",
        "command": "somewhere/a-script-that-forwards-to-ccache-clang++.sh -nostdinc -nostdinc++ -nobuiltininc -c somewhere/something.cpp -o somehere/something.pic.o",
        "file": "somewhere/something.cpp"
    }

[gigony]

I see the similar issue that @dapirian mentioned when use compile_commands.json for editing Android Native code (cpp) -- My Android (AOSP) build uses ccache so it starts with ccache path.
Hope intelli sense uses some heuristic and parse only compilation-related options.

The following is a real example of compile_commands.json with some modifications.

[
  {
    "command": "prebuilts/misc/linux-x86/ccache/ccache prebuilts/clang/host/linux-x86/clang-2690385/bin/clang++ -I /build/gbae/dev/vendor/aaa/bbb/display/dc/include -I /build/gbae/dev/vendor/aaa/bbb/core/include -I /build/gbae/dev/vendor/aaa/bbb/hwinc -I /build/gbae/dev/vendor/aaa/bbb/hwinc/f222x -I /build/gbae/dev/vendor/aaa/bbb/hwinc-qqq -I /build/gbae/dev/vendor/aaa/bbb/camera/core/include -I /build/gbae/dev/vendor/aaa/bbb/multimedia/codecs/include -I /build/gbae/dev/vendor/aaa/bbb/multimedia/audio/include -I /build/gbae/dev/vendor/aaa/bbb/multimedia/tvmr/include -I /build/gbae/dev/vendor/aaa/bbb/multimedia-partner/utils/include -I /build/gbae/dev/vendor/aaa/bbb/multimedia-partner/mm/include -I /build/gbae/dev/vendor/aaa/bbb/multimedia-partner/openmax/include -I /build/gbae/dev/vendor/aaa/bbb/camera-partner/imager/include -I /build/gbae/dev/vendor/aaa/bbb/camera/core_v3/hwinc/qqq -I /build/gbae/dev/vendor/aaa/bbb/camera/core_v3/hwinc -I /build/gbae/dev/vendor/aaa/bbb/camera/core/camera -I /build/gbae/dev/vendor/aaa/bbb/core-private/include -I /build/gbae/dev/vendor/aaa/bbb/multimedia/codecs/include -I /build/gbae/dev/vendor/aaa/bbb/multimedia/tvmr/include -I /build/gbae/dev/vendor/aaa/bbb/multimedia-partner/utils/include -I /build/gbae/dev/vendor/aaa/bbb/multimedia-partner/mm/include -I /build/gbae/dev/vendor/aaa/bbb/camera/utils/include -I frameworks/base/include -I /build/gbae/dev/vendor/aaa/bbb/camera/core_v3/include -I /build/gbae/dev/vendor/aaa/bbb/camera/core_scf/inc -I /build/gbae/dev/vendor/aaa/bbb/graphics-partner/android/include -I /build/gbae/dev/vendor/aaa/bbb/graphics/2d/include -I /build/gbae/dev/vendor/aaa/bbb/../../../external/jpeg -I frameworks/native/include/media/hardware -I /build/gbae/dev/vendor/aaa/bbb/camera-partner/image_enc/include -I /build/gbae/dev/vendor/aaa/bbb/multimedia-partner/openmax/include/openmax/il -I frameworks/av/services/camera/libcameraservice/camera2 -I system/media/camera/include -I system/core/include/system -I device/generic/goldfish/opengl/system/OpenglSystemCommon -I /build/gbae/dev/vendor/aaa/bbb/camera-partner/imager/include -I frameworks/av/include/camera -I /build/gbae/dev/vendor/aaa/bbb/services/cpl/libcpl/include -I frameworks/native/include/powerservice -I vendor/aaa/bbb/camera-partner/android/halv3/src -I out/target/product/p777/obj_arm/SHARED_LIBRARIES/camera.ppp_intermediates -I out/target/product/p777/gen/SHARED_LIBRARIES/camera.ppp_intermediates -I libnativehelper/include/nativehelper -I hardware/libhardware_legacy/include -I external/libjpeg-turbo -I /build/gbae/dev/vendor/aaa/bbb/graphics-partner/android/include -I frameworks/native/services/powermanager/../../include -I system/media/camera/src/../include -I frameworks/av/services/camera/libcameraservice -I system/media/camera/include -I frameworks/av/include/camera -I out/target/product/p777/obj_arm/SHARED_LIBRARIES/libcamera_client_intermediates/aidl-generated/include -I /build/gbae/dev/vendor/aaa/bbb/camera/utils/include -I external/libcxx/include -I external/libcxxabi/include -isystem system/core/include -isystem system/media/audio/include -isystem hardware/libhardware/include -isystem hardware/libhardware_legacy/include -isystem hardware/ril/include -isystem libnativehelper/include -isystem frameworks/native/include -isystem frameworks/native/opengl/include -isystem frameworks/av/include -isystem out/target/product/p777/obj/include -isystem bionic/libc/arch-arm/include -isystem bionic/libc/include -isystem bionic/libc/kernel/uapi -isystem bionic/libc/kernel/common -isystem bionic/libc/kernel/uapi/asm-arm -isystem bionic/libm/include -isystem bionic/libm/include/arm -c -fno-exceptions -Wno-multichar -msoft-float -ffunction-sections -fdata-sections -funwind-tables -fstack-protector-strong -Wa,--noexecstack -Werror=format-security -D_FORTIFY_SOURCE=2 -fno-short-enums -no-canonical-prefixes -mcpu=cortex-a15 -mfpu=neon-vfpv4 -D__ARM_FEATURE_LPAE=1 -mfloat-abi=softfp -DANDROID -fmessage-length=0 -W -Wall -Wno-unused -Winit-self -Wpointer-arith -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Werror=date-time -DNDEBUG -g -Wstrict-aliasing=2 -DNDEBUG -UDEBUG -D__compiler_offsetof=__builtin_offsetof -Werror=int-conversion -Wno-reserved-id-macro -Wno-format-pedantic -Wno-unused-command-line-argument -fcolor-diagnostics -nostdlibinc -target arm-linux-androideabi -target arm-linux-androideabi -Bprebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/arm-linux-androideabi/bin -fvisibility-inlines-hidden -Wsign-promo -Wno-inconsistent-missing-override -nostdlibinc -target arm-linux-androideabi -mthumb -Os -fomit-frame-pointer -fno-strict-aliasing -fno-rtti -UDEBUG -DDEBUG=0 -DIS_AVP=0 -DBUILD_STUBS=1 -DBUILD_CONFIGURATION_EXPOSING_RRR=1 -DBUILD_CONFIGURATION_EXPOSING_QQQ=1 -DPLATFORM_IS_JELLYBEAN=1 -DPLATFORM_IS_JELLYBEAN_MR1=1 -DPLATFORM_IS_JELLYBEAN_MR2=1 -DPLATFORM_IS_KITKAT=1 -DPLATFORM_IS_AFTER_KITKAT=1 -DPLATFORM_IS_AFTER_LOLLIPOP=1 -DPLATFORM_IS_AFTER_M=1 -DCONFIG_PLLP_BASE_AS_408MHZ=1 -DCONFIG_NONTZ_BL -DNON_SECURE_PROF=0 -DUSE_FD=1 -DCAMERA_PPP -DENABLE_HDR=1 -DENABLE_TRIDENT=0 -DHAL_APP_PROFILE_ENABLE=1 -Werror -Wno-error=sign-compare -DENABLE_BLOCKLINEAR_SUPPORT=1 -DENABLE_ARGUS_SERVER=0 -DENABLE_CAMTRACE=0 -DCAMERA_ENHANCEMENT=1 -Wmissing-declarations -Wno-error=narrowing -Wno-error=missing-field-initializers -Wno-unused-parameter -fPIC -mno-unaligned-access -D_USING_LIBCXX -std=gnu++14 -Werror=int-to-pointer-cast -Werror=pointer-to-int-cast -Werror=address-of-temporary -Werror=null-dereference -Werror=return-type -MD -MF out/target/product/p777/obj_arm/SHARED_LIBRARIES/camera.ppp_intermediates/camerahal3router.d -o out/target/product/p777/obj_arm/SHARED_LIBRARIES/camera.ppp_intermediates/camerahal3router.o vendor/aaa/bbb/camera-partner/android/halv3/src/camerahal3router.cpp",
    "directory": "/build/gbae/dev",
    "file": "/build/gbae/dev/vendor/aaa/bbb/camera-partner/android/halv3/src/camerahal3router.cpp"
  }
]

[sean-mcmanus]

I think this will be fixed via using the new compiler setting, which went set and found, doesn't try to find the compiler in the compile_command.json command.

[tksuoran]

I have hard time getting compileCommands working. Questions:

• Will absolute path work as directory?
• Will absolute paths work after -I?
• Does "-Ipath/somewhere" work (command/arguments)?
• Does "-I", "path/somewhere" work in arguments? "-I./"
• Do I need both command and arguments, or one of them?
• Does command need to begin with compiler?
• Does arguments need to start with compiler?
• Does VSCode actually invoke the compiler?
• What if my command begins with environment variables: "FOO=bar /path/somewhere/cc ..."; Will VSCode try to use FOO=bar as compiler, and fai)? How do I make it work?

I've tried absolute paths and paths relative to workspaceRoot, and I have tried replacing FOO=bar cc with /usr/bin/clang, but I just don't get include paths to work at all with compileCommands. I do have "C_Cpp.intelliSenseEngine": "Default" in my preferences. Any insight would be helpful. Thank you.

[sean-mcmanus]

@tksuoran With 0.15.0 the command or aguments needs to begin with the compiler. With 0.16.0 (https://github.com/Microsoft/vscode-cpptools/releases/ ) you should be able to set the compilerPath to get the correct compiler defaults (and we've also enabled a hidden loggingLevel "6" which will show what includePaths we successfully pick up). We invoke the compiler for getting the compiler's includePath and defines. The environment variable might break stuff, not sure. Everything else you mention is supposed to work, so let us know if you think we have a bug.

[sean-mcmanus]

Does setting the compilerPath work with 0.16.1 fix this issue for you, assuming you're not using WSL? If that is set, then it'll be used instead of the compile_commands.json command (or it'll use the guessed compiler if no compilerPath is set).

[bobbrow]

Closing this issue. If the "compilerPath" property doesn't resolve this for you, please let us know.