Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Add IR-to-Objectcode caching with `-ir2obj-cache=<cache dir>`. #1572
This adds a caching mechanism such that compiling the same source twice should be faster as it will use the previously generated object file.
Machine codegen takes a lot of time, so this caching results in a significant speed up (~65sec -> ~39sec on testcase). Generating the hash is done by hashing the raw bitcode output from LLVM. This is not for free and takes quite some time: it takes time to generate a 200MB bitcode file, I think the hash time itself is not noticable. Future work could try to get a (unique) hash quicker through some other means.
I suppose there is little harm in adding this as an experimental feature (maybe add "(experimental)" in place of "for faster recompilation of unchanged IR code."?).
The reason why I'm a bit cautious is that it seems like we might want to do something more fully-featured in the near future like what ThinLTO in Clang/LLVM are slowly moving towards. It would be a pity to have to remove/rework the command line flag then, breaking people's stuff. Although, thinking of it, this should be reasonably painless to keep maintained on the side due to its relatively small size. And if the worst comes to the worst, we could still just ignore the flag, which would only make builds slower, not break them.
So, just ignore the last stream-of-consciousness paragraph of doubts – feel free to merge, although I would make sure that the feature is advertised as experimental sufficiently clearly for the time being.
ThinLTO is definitely something interesting to look into for speeding up builds too.
I'll add "(experimental)" and merge. Perhaps someone can work-out a faster way of obtaining a unique ID for the IR, which would then break existing hashes. If people like to store them for a long time and have them be reusable, we can tell them: "hey, it says 'experimental'!" ;-).
Jun 23, 2016
The nice thing about caches is that you can just throw existing contents without compromising correctness – just performance! ;) (Assuming no collisions, of course.)
Ah, nice, I hadn't noticed that they have finally written up their progress (and that they are
We might need to invest some work into optimising it for the debug build use case, though – at least from the blog post, it sounds like they are exclusively working on release builds so far.