nim doesn't recompile dependencies when it should, can be fixed via clang -M
#510
Comments
timotheecour
changed the title
clang -Mclang -M
|
It's not Nim's business, Nim is not a build tool for C code. If the header file changed you use |
that's the thing, user may not know it the header changed. He may not even know there's a header involved, it could be coming from a third party library (say, if using nimterop or any other thing that depends on C that's pretty much guaranteed to happen, and happened many times to me and @genotrance ). using a blanket defensive strategy via The fix isn't even hard; it's just a bug, and it's a PR away. |
If you have A more general "should this be recompiled?" hook would be fine with me, but not this clang-specific "fix". |
gcc works exact same,
it's not clang-specific as mentioned above, it's a general fix that is robust how about I hide it behind a flag |
I dunno, ideally the implementation is clean enough that it can always be enabled by default. |
|
How does this relate to Nim's own incremental compilation? Can some of that effort be leveraged here? You'd have to run this Looks like we'd have to do something similar in nimterop as well since we cache the Nim output only based on the header changing or not. |
The Nim manual clearly states: I have a similar statement in nimterop as well. We could leave it as a user's problem but now that I know I can detect dependencies, I will be using it to make the Nim code caching more robust. No doubt it is gcc/clang specific and Nim is much larger in its purview but I can bet we have superior support even within Nim for these compilers compared to tcc, vcc and others.
Well, a pragma like Meanwhile, this isn't nimterop specific - we are not talking about Nim code that is dynamically generated and should be recompiled if it changes. All in all, the goal is to make interop better. If that is valuable then the compiler can assist with it. If not, the community will have to work around the status quo. |
It never happened to me though and it can only happen when you use a feature without reading how it really works. Ok, you cannot read about every feature that you end up using, but at the same time, we cannot put ever more effort into rewarding ignorance. |
|
We are talking about improving an existing feature since it has affected us first hand and will inevitably impact our users. Further, we are willing to contribute and improve it with consensus. We can debate the merits of the design or whether it is worth the effort or added complexity but it isn't an RTFM issue. Finishing touches are boring, tedious and never ending but they do make a difference. |
|
There are C/C++ build tools. For example, CMake, Autotools, scons, ninja, etc. I used CMake before I use Nim but I never used it with Nim. Instead of using {.compile.} pragma, use # Configure project and generate Makefiles
# Execute following command on build directory
cmake "path-to-parent-directory-of-CMakeLists.txt"
# Or following command might work with CMake 3.13
cmake -S "path-to-parent-directory-of-CMakeLists.txt" -B "path-to-build-directory"
# Build project
cmake --build "path-to-build-directory" |
can you provide a link with the list? it works with all major compilers AFAIK, eg:
Maybe there's a niche compiler that doesn't have such option but it's irrelevant; functionality shouldn't be dumbed down the the lowest common denominator at the expense of platforms/environments that do expose such functionality /cc @Araq
as always, it's tempting to dismiss a bug by saying: "I was never affected by it, so this must not be a real issue" but reality is different workflows lead to different bugs encountered. as often, I'm spending more time explaining the rationale than it would take me to code up the PR |
Ok, go for it. |
It is. @krux02 I think you misunderstood the test. The test illustrates what happens when a (implicity, maybe recursive) include depdency changes in between 2 compilations. When reading the test, just mentally treat "D20190123T230907.h" as if it were "/home/user/homebrew/include/opencv2/opencv.hpp" ie, in real life, it's not generated by the nim application but exists in your filesystem. The fact it's auto-generated in this test is because I want the test to be self contained.
hopefully above explanation clears that point too.
yes, I thought about that and a proper solution will have to do some benchmarking to measure added cost in compile time to check whether it's acceptable as default or whether it needs to be hidden by a flag. if performance dicates it, It's easy to remove system files from the list of dependencies: we can run and subtract out dependencies starting w a default include.
file size should have no (amortized) impact:
|
This issue affects any nim code that interfaces with C, and is a common annoyance with nimterop, causing sometimes hard to diagnose compile time errors, or worse, silently giving wrong outputs at runtime (as shown in test case)
The issue is that nim doesn't recompile the C files if they depend on a header file that was modified since last compilation.
nimble testfails due to dirty nimcache, -f needed sometimes nimterop/nimterop#29 :nimble testfails due to dirty nimcache, -f needed sometimes /cc @genotranceproposal for a fix
for every compiler invocation, eg:
generate a call to
clang -M, eg:this tells you the (recursive!) include dependencies, hence the things that should get tracked to validate/invalidate the cache
This should be robust.
[EDIT] note: this approach should work w most/(all?) major C compilers
these all return recursive includes:
clang -Mgcc -M/showIncludes(see link )The text was updated successfully, but these errors were encountered: