JSON Compilation Database exported from Qt Creator 4.9-beta1 is unsupported

[aportale]

Since Version 4.8, Qt Creator supports exporting and importing of JSON Compilation Databases. Theoretically, this is the missing link to pass a project from Qt Creator to Sourcetrail.

However, there are two flavors of compilation databases: those with "arguments" lists and those with "command" values. Qt Creator exports the prior format and Sourcetrail seems to import the latter. That means that Qt Creator and Sourcetrail can currently not interoperate.

Certainly, Qt Creator can/will allow the user to select one of the two formats for compilation database export in a future version. But I believe, it would also make sense to let Sourcetrail import the "arguments" variant.

[mlangkabel]

Hi @aportale, thanks for posting this issue. I'm just setting up a testing environment for QT Creator 4.8 to get to a state where we can reproduce your issue. We'll keep you posted on our progress.

BTW: Congrats to your issue ID ;)

[mlangkabel]

We just tested this on the code of our qtc-sourcetrail extension for the qt creator and everything is working fine. So we have not been able to reproduce your issue.

Can you please try to

• check out that project
• open it with your version of the QT creator
• export the compilation database
• create a Sourcetrail SourceGroup based on the exported compilation database
• try to run the indexer and
• tell us what happens there?

[mlangkabel]

However, it is nice to know that the QT Creator now provides this option. I added #668 to keep our users informed.

[aportale]

I must admit that I generated the compile_commands.json in Qt Creator 4.9-beta1, which then caused the reported issue. But genertaing it now with Qt Creator 4.8, I can confirm that loading that .json file in Sourcetrail it works! Sorry for being unspecific. Good to know that there is such a difference in the exported json files between Qt Creator versions. I will dig around in Qt Creator to find out what the differences are and if there is something to be done to generate files that Sourcetrail accepts.

Anyways, here the details of what goes wrong with the Qt Creator 4.9-beta1 generated compile_commands.json: I opened qtc-sourcetrail and generated compile_commands.json.txt. In Sourcetrail, I got then these error messages after importing the .json:

PS: Usually I wait to catch that ID, but until now never managed to :)

[aportale]

I created a Qt Creator task for this issue

[mlangkabel]

Thank you for taking a deeper look. I just tried to manually add the --driver-mode=cl to the compilation database generated by Qt Creator 4.8 and I get a lot of clang errors in Sourcetrail's console output.

To make sure that it is not just an issue with Sourcetrail I also tried to feed the same json file to clang-tidy and I get the same issues. This is what the console output looks like:

$C:\sourcetrail\plugin\qtc-sourcetrail>"C:/Program Files/LLVM/bin/clang-tidy.exe" "C:\sourcetrail\plugin\qtc-sourcetrail\src\sourcetrailpluginsettingspage.cpp"
7 warnings and 1 error generated.
Error while processing C:\sourcetrail\plugin\qtc-sourcetrail\src\sourcetrailpluginsettingspage.cpp.
warning: c++: 'linker' input unused [clang-diagnostic-unused-command-line-argument]
warning: i686-pc-windows-msvc: 'linker' input unused [clang-diagnostic-unused-command-line-argument]
warning: unknown argument ignored in clang-cl: '-fcxx-exceptions' [clang-diagnostic-unknown-argument]
warning: unknown argument ignored in clang-cl: '-fexceptions' [clang-diagnostic-unknown-argument]
warning: unknown argument ignored in clang-cl: '-std=c++14' [clang-diagnostic-unknown-argument]
warning: unknown argument ignored in clang-cl: '-target' [clang-diagnostic-unknown-argument]
warning: unknown argument ignored in clang-cl: '-x' [clang-diagnostic-unknown-argument]
C:\dev\Qt\Qt5.9.1\5.9.1\msvc2015\include\QtCore/qglobal.h:45:12: error: 'type_traits' file not found [clang-diagnostic-error]
#  include <type_traits>
           ^

In Sourcetrail I get some additional errors that look like this:

warning: unknown argument ignored in clang-cl: '-isystem' [-Wunknown-argument]

This would also explain the 'type_traits' file not found error from clang tidy. To me it looks like setting the driver mode to cl invalidates a lot of the other arguments.

[yvvan]

--driver-mode=cl is only used in combination with msvc-style flags. So it's not a surprise that it generates errors with gcc-style command line. I think you can parse Qt Creator 4.9 command line with the current clang 8 or master because it uses "/clang:" prefix to pass some gcc-style options while the cl driver mode is activated.

This option was introduced in https://reviews.llvm.org/D53457 (commit https://reviews.llvm.org/rC346393)

[mlangkabel]

@yvvan, thanks for joining in. So do I understand it correctly that whenever you use the --driver-mode=cl argument in a compilation database you need to prefix all other gcc-style flags with /clang:?
So you can either specify your arguments with

-std=c++14

or

--driver-mode=cl
/clang:-std=c++14

? I just tested the second style with an older version of clang-tidy (5.0.0) and the

warning: unknown argument ignored in clang-cl: '-std=c++14' [clang-diagnostic-unknown-argument]

is gone from the console output.

[yvvan]

@mlangkabel
Yes, exactly. Unless these flags already work with both modes (-I, -D, -U and some more work in both modes). Older versions might just ignore the option if you don't see any error or warning because they were definitely unable to handle '/clang:' prefix before. Sorry for the delay in my answer.

[aportale]

FYI, @yvvan modified the JSON Compilation Database export in Qt Creator 4.9 to match the traditional style. https://codereview.qt-project.org/255604/

I am happy :) Shall we close this?

[yvvan]

Well, supporting such flags could anyways be an improvement for Sourcetrail :)

[mlangkabel]

Nice to hear that Qt Creator 4.9 is fixed!

Ok, just tested this again and it is looking quite good:

• Sourcetrail supports both of the compilation database styles.

• I took a "regular" compilation database and converted it to clang-cl style in the following way:

  • add --driver-mode=cl flag
  • replace --isystem with /I
  • replace -D with /D
  • replace -std=c++14 with /clang:-std=c++14
  • add /EHa to enable exceptions, because they were used in code indexed by Sourcetrail

-> If I use this converted compilation database with Sourcetrail, it works without an error :)

Thanks for closing this issue and also thank you both for your explanation regarding this whole issue!

[yvvan]

Good, some notes:

1. -D is a valid option for both modes
2. replacing --isystem with /I is not the best way because they put include search paths on a different position in the list of all include search paths.
3. -std=c++14 can be also replaced with -std:c++14
   But these checks are not really necessary in the context of this bugreport because all was needed is a new version of clang (clang-8) used to parse the command line.

[Waleed-Azam]

any implementation for the import-export database Android?