clcache.py - a compiler cache for Microsoft Visual Studio
clcache.py is a little Python script which attempts to avoid unnecessary recompilation by reusing previously cached object files if possible. It is meant to be called instead of the original cl.exe executable. The script analyses the command line to decide whether one source files is to be compiled. If so, a cache will be queried for a previously stored object file.
If the script is called in an unsupported way (multiple source files in one invocation, compiler called for linking), the script will simply relay the invocation to the real cl.exe program.
Create a wrapper batch file called cl.bat and put it into a directory
which comes first in the
PATH. In my case, I put the batch file into
%HOME%\bin and prepended that directory to the
PATH. The batch
file should simply call the Python script, forwarding all the arguments.
Here’s what I use:
@C:\Python26\python.exe C:\clcache\clcache.py %*
This way, simply running
cl will invoke the script instead of the real
Print usage information
Print some statistics about the cache (cache hits, cache misses, cache size etc.)
- -M <size>
Sets the maximum size of the cache in bytes. The default is 1048576000 Bytes.
If set, points to the directory within which all the cached object files should be stored. This defaults to
Can be set to the actual cl.exe executable to use. If this variable is not set, the clcache.py script will scan the directories listed in the
PATHenvironment variable for cl.exe.
If this variable is set, a bit of diagnostic information is printed which can help with debugging cache problems.
Setting this variable will disable clcache.py completely. The script will relay all calls to the real compiler.
How clcache works
clcache.py was designed to intercept calls to the actual cl.exe compiler binary. Once an invocationw as intercepted, the command line is analyzed for whether its a command line which just compiles a single source file into an object file. This means that all of the following requirements on the command line must be true:
/linkswitch must not be present
/cswitch must be present
There must be exactly one source file present on the command line.
If all the above requirements are met, clcache forwards the call to the
preprocessor by replacing
/EP in the command line and then
invoking it. This will cause the complete preprocessed source code to be
printed. clcache then generates a hash sum out of
The complete preprocessed source code
The ‘normalized’ command line
The file size of the compiler binary
The modification time of the compiler binary
The ‘normalized’ command line is the given command line minus all switches
which either don’t influence the generated object file (such as
which have already been covered otherwise. For instance, all switches which
merely influence the preprocessor can be skipped since their effect is already
implicitely contained in the preprocessed source code.
Once the hash sum was computed, it is used as a key (actually, a directory name) in the cache (which is a directory itself). If the cache entry exists already, it is supposed to contain a file with the stdout output of the compiler as well as the previously generated object file. clcache will copy the previously generated object file to the designated output path and then print the contents of the stdout text file. That way, the script behaves as if the actual compiler was invoked.
If the hash sum was not yet used in the cache, clcache will forward the invocation to the actual compiler. Once the real compiler successfully finished its work, the generated object file (as well as the output printed by the compiler) is copied to the cache.