Add a bxl for generating a unified compile_commands. #510
Conversation
This adds a simple bxl for generating a global compilation database for all targets in a build. It is very fast, able to generate a database for around 4,000 files in under 4 seconds. This opens the door for external users being able to integrate with traditional LSPs like clangd.
facebook-github-bot
added
the
CLA Signed
|
I'm not sure why this failed, but it doesn't appear to be related to anything in my PR. @ndmitchell do you have any suggestions for how I can fix this? |
|
A fix for the CI issue landed on master, you should pick it up if you rebase. I have to say I don't quite understand the need for aquery here. The obvious thing to me would have been to run analysis on the whole world to gather all the compilation DBs, and then just merge them. How does that approach go wrong? |
When I tried that (maybe I did it wrong) it was extremely slow. It was an early attempt, so my memory is rusty, but I recall that generation of all the compilation dbs was very slow, because the sub target to generate the compilation db in the first place had dependencies on other targets (didn’t investigate what) and it would build things before it was able to generate the compilation db. I don’t know what actions that “build things” step ran (didn’t investigate), I just remember that it was slow. This method generates a universe compilation db very fast. |
The only action that's run by compilation DB generation does argsfile expansion. I could maybe imagine this forcing targets in the |
|
So the context here is that when you generate a compilation database, you basically have two choices:
I believe our @JakobDegen is suggesting we have |
| # can do until buck2 upstream actually stores a Starlark list as the value of this attribute. | ||
| cmd = action.attrs.cmd.value() | ||
| cmd = cmd.strip("[]").split(',') | ||
| cmd = list(map(lambda x: x.strip(), cmd)) |
There was a problem hiding this comment.
Can we get this chunk of code pulled out into a separate function? We probably will want to do something in future to clean this up, and having it separate makes it easier to see what to do.
CC @wendy728 for how we might want to unpack cmd_args values in future.
| filename = entry["file"] | ||
| if not files.contains(filename): | ||
| entries.append(entry) | ||
| files.add(filename) |
There was a problem hiding this comment.
Indent this line one level?
Summary: 1. I made the default platform in the prelude public so it can be inherited from. I don't know why it wasn't already like that. Maybe I'm misunderstanding how to use it 2. Not sure if I'm using the config_setting and constraint_setting properly since it seems duplicated. 3. The script to generate the compile_commands.json was more or less copied from facebook/buck2#510. There are still a lot of improvements to be made but I think this is good enough to be merged: 1. With MSVC the compilation should probably be done with /Z7 by default and then, if the "stripped" sub-target is asked for, that flag is removed. 2. The cxx toolchain needs the compilation options looked at, not sure if Linux's defaults work properly. 3. The Install.ps1 and Copy.ps1 need to be ported to Linux. X-link: facebook/buck2#517 Reviewed By: KapJI Differential Revision: D53692733 Pulled By: JakobDegen fbshipit-source-id: 8d5962084831a3cccde128f0aa0360a1aaf72004
Summary: 1. I made the default platform in the prelude public so it can be inherited from. I don't know why it wasn't already like that. Maybe I'm misunderstanding how to use it 2. Not sure if I'm using the config_setting and constraint_setting properly since it seems duplicated. 3. The script to generate the compile_commands.json was more or less copied from #510. There are still a lot of improvements to be made but I think this is good enough to be merged: 1. With MSVC the compilation should probably be done with /Z7 by default and then, if the "stripped" sub-target is asked for, that flag is removed. 2. The cxx toolchain needs the compilation options looked at, not sure if Linux's defaults work properly. 3. The Install.ps1 and Copy.ps1 need to be ported to Linux. Pull Request resolved: #517 Reviewed By: KapJI Differential Revision: D53692733 Pulled By: JakobDegen fbshipit-source-id: 8d5962084831a3cccde128f0aa0360a1aaf72004
Summary: 1. I made the default platform in the prelude public so it can be inherited from. I don't know why it wasn't already like that. Maybe I'm misunderstanding how to use it 2. Not sure if I'm using the config_setting and constraint_setting properly since it seems duplicated. 3. The script to generate the compile_commands.json was more or less copied from facebook/buck2#510. There are still a lot of improvements to be made but I think this is good enough to be merged: 1. With MSVC the compilation should probably be done with /Z7 by default and then, if the "stripped" sub-target is asked for, that flag is removed. 2. The cxx toolchain needs the compilation options looked at, not sure if Linux's defaults work properly. 3. The Install.ps1 and Copy.ps1 need to be ported to Linux. X-link: facebook/buck2#517 Reviewed By: KapJI Differential Revision: D53692733 Pulled By: JakobDegen fbshipit-source-id: 8d5962084831a3cccde128f0aa0360a1aaf72004
This adds a simple bxl for generating a global compilation database for all targets in a build. It is very fast, able to generate a database for around 4,000 files in under 4 seconds.
This opens the door for external users being able to integrate with traditional LSPs like clangd.
This addresses several outstanding issues for external users, including #194, #307, and #452.