Add ir2obj cache pruning support. #1753
Conversation
| #ifndef LDC_DRIVER_IR2OBJ_CACHE_PRUNING_H | ||
| #define LDC_DRIVER_IR2OBJ_CACHE_PRUNING_H | ||
|
|
||
| void DpruneCache(const char *cacheDirectoryPtr, size_t cacheDirectoryLen, |
There was a problem hiding this comment.
Sorry for nit-picking, but DpruneCache is not a great name ("D"?), especially since it is hardcoded to "ircache_…".
There was a problem hiding this comment.
wip !
(it was to set it apart from the previous C++ implementation)
JohanEngelen
force-pushed
the
prunecache_d
branch
4 times, most recently
from
1a29e8b
to
1bc289a
Compare
JohanEngelen
changed the title
JohanEngelen
force-pushed
the
prunecache_d
branch
from
1bc289a
to
4f284fa
Compare
|
All green :) |
| #define LDC_DRIVER_IR2OBJ_CACHE_PRUNING_H | ||
|
|
||
| #if __LP64__ | ||
| using d_ulong = unsigned long; |
There was a problem hiding this comment.
#include <cstdint> and use uint64_t.
There was a problem hiding this comment.
I had tried it, and it doesn't work :(
There was a problem hiding this comment.
Mac's compiler produces:
pruneCache(char const*, unsigned long, unsigned int, unsigned int, unsigned long long, unsigned int)
DMD and LDC produce:
pruneCache(char const*, unsigned long, unsigned int, unsigned int, unsigned long, unsigned int)
Even the table here https://dlang.org/spec/interfaceToC.html is ambiguous for ulong...
There was a problem hiding this comment.
Sorry, I didn't notice this was actually an extern(C++) interface.
|
I suppose you'll move |
Yep! |
JohanEngelen
force-pushed
the
prunecache_d
branch
from
4f284fa
to
fa42d6e
Compare
|
(CircleCI was green. Currently broken because LLVM 4.0 API change) |
|
We could think about backing out ldc-prune-cache and integrating it into |
This implements ir2obj cache pruning in the compiler itself, and also through a stand-alone tool (they share the cache pruning D implementation).
Cache pruning scheme:
-ir2obj-cache-prune-intervalandldc-prune-cache --interval) to see if pruning is already needed again. A timestamp file is written to the cache for this.-ir2obj-cache-prune-expirationandldc-prune-cache --expiry).--ir2obj-cache-prune-maxbytesandldc-prune-cache --max-bytes) and/or relatively (--ir2obj-cache-prune-maxpercentageandldc-prune-cache --max-percentage-of-avail).Pruning is done after codegen, but before linking. So the prune parameters must be such that the object files needed for linking will survive (e.g. the cache size limit should be large enough).
Pruning is off by default, but specifying any pruning parameters on the commandline implies enabling it (
--ir2obj-cache-prune).The reasoning for having it off per default is that in projects using parallel compilation threads, you'd want the pruning to happen at the end of a build cycle (using the standalone tool). Otherwise, the first finishing compilation thread may start pruning stuff that you might cache-hit in another compile thread. Also, multi-threaded pruning is untested at best.
The pruning code uses the file's "last access time". But because this is not automatically updated upon symlinking or usage by the linker on some platforms (Windows 7, from what I read online), I added an extra "touch" upon a cache hit such that the last access time is refreshed at least during compilation.
Resolves #1747