-
Notifications
You must be signed in to change notification settings - Fork 59
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support non-self-contained files #45
Comments
For posterity, ycmd/libclang allows this by allowing users to return something like That makes the TU main file be the value of
|
Really waiting for this feature to release! |
For the record, YCM has just heard from another users who needed [
{
"directory": "/home/bstaletic/test/",
"command": [ "/usr/bin/c++", "-c", "bar.cpp" ],
"file": "/home/bstaletic/test/bar.cpp",
"override_filename": "/home/bstaletic/test/foo.cpp"
}
] In the above example:
This is just an idea that, frankly, hasn't been given much thought and it doesn't address the problem of wiring this throughout the clangd code. Considering that the latest cmake has implemented support for unity builds and that the comp db generation is broken when used with unity builds, |
+1000 to this. I think unity builds are gaining more and more traction, and this would massively help a lot of people. |
Documentation to llvm 8 said that to achieve what people ask in this bug, one needs to use clangd-indexer. These lines disappeared from llvm 10 documentation, it just says that clangd-indexer is for developers. Why this feature is so complicated? |
AFAIK, clangd-indexer has never supported the use cases described in this bug. The "project-wide view" mentioned in the linked doc just refers to e.g. being able to perform Find References and get results in other translation units. Mention of clangd-indexer has been removed because that functionality is now built into clangd via the background indexing feature. |
Yes, what Nathan said.
A few main reasons:
|
Maybe, but I can't rewrite all C++ libraries. Jumping to definitions is most useful when exploring unfamiliar external code. I have no influence to force all the authors to implement good practices. |
Wait, Nathan? Eclipse CDT supporter, who suggested to use clangd instead of CDT's cpp-indexer? You actually work on clangd? :) |
They kinda do. If your entire codebase is "shoved" into a single TU, your I've heard that people work around this problem by running cmake twice (now that cmake actually has built-in support for unity builds). Once with unity off, to get a working |
I contribute to it, yeah :) I switched from Eclipse CDT about a year ago. |
Clangd does not currently work for unity builds (see e.g. #147 (comment)). Perhaps you mean support for them could be added more easily than by solving the challenging issues you described?
To be fair, LLVM itself does use non-self contained files at times. I'm thinking of things like |
Right, but if you have self-contained files, you don't have to always do a unity build with them. Wouldn't you only want to do the unity build as a final heavily-optimized, slow-building, non-incremental release build, but use a regular build for development?
LLVM does lots of questionable things :-) Tablegen .def format falls more into the "stupid pp tricks" bucket than sensible C++ code we're just having trouble parsing, though. |
one of the main purposes of jumbo TU is to reduce compile time where it's dominated by IO (particularly includes, etc.). At work i benchmarked speedup comparable to the preamble trick on compiling a libarary as a single jumbo vs |
No, some projects use unity builds for build performance (since it means common includes only have to be parsed once per unified-TU rather than once per individual-TU). Some projects take this a step further and only support unity builds, since non-unity builds can easily break due to include-what-you-use violations if devs are doing unity builds locally, and the project may not have the CI resources to test both configurations. Such projects don't even have the option of doing a non-unity build specifically for clangd's consumption. |
What is unity build? |
I am switching now. Especially after C++20 CDT became unbearable. |
It's a technique where groups of source files are combined (by e.g. |
Wow. Never've heard of this technique. |
@sam-mccall so just to understand this correctly, for the moment this is very hard to implement because of #45 (comment). Does what you said earlier also apply in an |
Vote for https://gitlab.kitware.com/cmake/cmake/-/issues/19826 |
To confirm: the ycmd implementation to support this took me less than an hour. (apropos nothing; ycmd libclang engine is an invisible fraction of complexity compared with clangd) |
Also added includes in *.ipp files to enable jump-to-declaration in IDEs. This sadly results in clangd "main file cannot be included recursively" error annotations in Qt Creator, probably due to clangd/clangd#45 . The definitions in inline-only ipp files will be moved to their corresponding headers so that at least the clangd error annotations can go away in CPPWAMP_COMPILED_LIB mode. Fixes #109
I just ran into this problem while trying to look at things like wake_up_var() in today's Linux kernel. This symbol is in a file (wait_bit.c) that is included from within another file (build_utility.c), and so VS Code + clangd cannot browse any of the symbols within wait_bit.c. This is a serious limitation. If you look at the top of build_utility.c (and also look through "git blame build_utility.c"), you'll notice that this is an intentional, recent effort to reduce build times. And that means that my chances of changing this are basically zero--people want it this way. So now I'm reduced to maintaining local hacks in order to browse a big chunk of the sched unit in the Linux kernel, with a fair chance that the damage will continue to spread, because this is apparently a popular way to speed up builds. This is an extremely bad situation that will drive all of us back to horrible older techniques, unless clangd loses the "this is unholy and impure and bad software engineering, so we are not interested" attitude. Please reconsider that, because Real Projects are messy and impure! |
I believe it's been clarified in later comments (particularly this one) that the project is open to contributions to improve support for non-self-contained files. It's just that there are non-trivial technical challenges to overcome (as detailed in this comment and this one), and someone invested in improving support for these scenarios will need to put in the legwork to solve them. |
Very good. In that case, I'll tell my compiler engineering colleagues about this, and see if it sparks any interest. |
nearly 4 years past, is there any body work on it? we have lots of legacy code write as non-self-contained style, looking forward this feature, so we can migrate to vscode from VS or CLion as our main IDE. Vscode + Clangd is so good for daily deploping, is three any plan? |
Since January I have successfully worked around this with a method I have not seen proposed anywhere else, as outlined in this blog post which I wrote at the time. https://www.frogtoss.com/labs/clangd-with-unity-builds.html |
another workaround is -include="xxx.h" in compile_commands or clangd file |
Just to add a data point to this that is not about unity builds: I have seen and used this style for DSP and embedded projects, where
I mention DSP and embedded as a context because it relates to point #2. It seems like one solution to this would be to have the helper functions have external linkage, and have separate private header files for them (or just put them in a separate module). Unit tests could then use that private module/header's API. But this means that the functions have to appear in the symbol table, and can't be inlined, or can be inlined but need to be duplicated as non-inlined versions, even though they're not meant to be public. It's bloat and pessimisation we want to avoid for DSP/embedded applications. Another solution that avoids this is to consider that the unit tests for that module form a kind of single-TU project (or subproject), where only the main function (or top-level test runner function) is exposed, and it has the same set of static "helper" functions as the real project. And so your unit tests include the implementation file ( // module_a/test/test_module_a.c
#include "unit_test_harness_whatever.h"
#include "test_module_a.h"
// Note include of implementation file!
#include "module_a/src/module_a.c"
static void test_module_a_helper_x_but_not_y(void) { /* ... */ }
static void test_module_a_helper_annoying_corner_case(void) { /* ... */ }
void test_module_a_all(void) {
RUN_TEST(test_module_a_helper_x_but_not_y);
RUN_TEST(test_module_a_helper_annoying_corner_case);
} Yes it's niche, yes it looks jarring and unidiomatic at first, but it's valid, it makes sense and works in these contexts, and I've seen it in real, professional, high-quality C code bases. (It also tests that the implementation file is appropriately self-contained!) There are other solutions to this as well, eg. |
Well yes. This fundamentally applies to any TU which is composed of multiple files (including headers, c, cpp, inc, etc). Clangd models TU=file and as the others have said it's a big refactor to fix it. |
You're not wrong, I just wanted to provide another data point to help avoid some kind of unity-build-specific solution (I have no idea how that would arise, but you never know). |
Ok, I'm trying to understand the problem a bit more deeply. Is this a fair restatement of what you said above?: Let's say we have CPython, which is a bit of a mess of non-self-contained headers, but basically the external interface is that anyone using it is supposed to While it would be possible to devise a mechanism to allow the user to say "do not even try to parse this header as a TU, parse Or would it even affect performance for people just using
Is this more of a problem of "where do we get that information"? I noticed that there's support for IWYU pragmas for specifying things like "this header is private; if you want its symbols, always include header X instead". That seems very much the same kind of configuration data that would be needed by this, but I think it's currently only used for flagging non-direct includes and automatically adding includes. Or is it something more nuanced?
No arguments here. I'm a big enough fan of self-contained headers and even C++ modules that I'd generally be willing to spend a non-trivial amount of time to clean up things. With projects like CPython, I suspect selling a major header file refactoring to CPython upstream might be an uphill battle, even if they see the value in good IDE support... |
I now see that some of my previous comment's questions were addressed in this comment, which I somehow missed on the first skim... Sorry about the noise! |
Currently clangd doesn't work with non-self-contained files, even if people have a way to build their projects with those files, for example by merging some/all of non-self-contained files into a single TU and building that file.
The text was updated successfully, but these errors were encountered: