How to compile ccache for Windows with MSYS2

[mumpitzstuff]

Description

It is possible to compile ccache for windows by following these simple steps:

1.) Install MSYS2.
2.) Open MSYS2 MSYS console and run "pacman -Syu" multiple times until everything is updated.
3.) Close MSYS2 MSYS console and open MSYS2 MinGW 64-Bit console.
4.) Install "pacman -S mingw-w64-x86_64-gcc".
5.) Install "pacman -S mingw-w64-x86_64-cmake".
5.) Install "pacman -S make".
6.) Install "pacman -S zstd".
7.) Switch to ccache directory and create build directory "mkdir build".
8.) Edit CMakeLists.txt and add this line to create static linked exe file: "set(CMAKE_EXE_LINKER_FLAGS "-static")".
9.) Edit C:\msys64\home\username\.bashrc and add this line: "PATH=$PATH:/mingw64/bin/gcc:/mingw64/bin/cmake".
10.) Restart the MSYS MinGW 64-Bit console.
11.) Switch to build directory within ccache directory and run: "cmake -DCMAKE_BUILD_TYPE=Release -DZSTD_FROM_INTERNET=ON -G "MSYS Makefiles" ..".
12.) Run "make".

May be the docu can be enhanced because it costs me quite a while to get everything to work...

[AlexanderLanin]

Hi,

looks nice, that's a step by step instruction 👍
Ideally we could add this to CI.

Most of this seems more MSYS2 related then ccache related, is that right?
Don't get me wrong, I'm not against a section on "how to compile for MSYS2".
Although I do wonder about relevance a little bit considering the jump in 2015:
https://trends.google.de/trends/explore?date=all&q=mingw,msys2,wsl

Could you explain step 8?

I assume step 9 is for making steps 4+5 available?

The use case is having ccache in MSYS2 environment where you use it with gcc/clang?

[mumpitzstuff]

It is more or less the preparation of MSYS2 to get the ccache build to work. MSYS2 is just the base to install gcc, cmake and other tools as easy as possible.

Step 8 is needed to compile the exe without any dll dependencies (needed dlls will become part of the executable).
Step 9 was needed because the msys version of gcc and cmake is may be installed in parallel and without this path, the wrong version was used. It is also possible to uninstall the msys versions of gcc and cmake before with "pacman -R gcc" and "pacman -R cmake" and install the mingw versions afterwards. If you dont do that or if you try to use the msys console instead of the mingw console, you will get also an exe file but you will always need a seperate dll msys-2.0.dll. Another strange thing was that all _WIN32 parts where completly ignored and e.g. InodeCache.cpp was part of ccache and it was possible to activate it but the generated inodes where always 0 or something like that. It was not clear to me what was really working in this version and what was not working.

The use case for me is to use it as normal compiler cache for some gcc derivates (embedded c software) completly outside of the msys2 environment. msys2 was just used to compile ccache. I never used it before for anything else, but i was really impressed how easy it was to get it to work.

[jrosdahl]

May be the docu can be enhanced because it costs me quite a while to get everything to work...

Another option would be to add hints like this to the ccache wiki (which is currently unused). I think that would make more sense than keeping detailed instructions in the official documentation.

[jimklimov]

TL;DR: I had fun time exploring the build to try finding and fixing the problem for MSYS2 in ccache codebase, but ended up discovering that carefully using the PATH (with mingw64 builds of packaged ccache and gcc in my case) addresses it well enough too. On the upside, it seems that the custom-built ccache has no qualms with default or mingw-tuned variants of gcc.exe, so the exercise was not in vain :)

Wrappers scripts are available as e.g. /mingw64/lib/ccache/bin/gcc and lack a set for clang tools; easy-peasy with:

cd /mingw64/lib/ccache/bin
for T in clang clang++ clang-cpp ; do sed "s/gcc/$T/" < gcc > "$T" ; chmod +x "$T" ; done

---

For posterity:

MSYS2 currently includes ccache: https://packages.msys2.org/search?t=binpkg&q=ccache but it seems to not work in practice on Windows MSYS2, possibly due to mixing the path separator characters. It seems unable to create the cache files after it has created the cache directory tree (though you might have to pre-create the ${HOME}/.ccache anyway.

• https://sourceforge.net/p/msys2/tickets/253/

So I found this effort to try rolling my own... Commenting as I go...

Slight update for step(s) 5: per other docs, seems a better choice is

# pacman -S --needed \
    mingw-w64-x86_64-cmake cmake \
    mingw-w64-x86_64-gcc gcc \
    zstd make \
    mingw-w64-x86_64-hiredis 

...due to:

• running once (less overhead to start pacman itself)
• pulling more dependencies (as "needed")
• including cmake and gcc tools themselves, not just the ecosystem they want (might be something that changed in MSYS2 between 2020 and 2022?) :)
• adds deps for redis backend (since ccache 4.4 I guess)

For the build steps 9+10+11, PATH changes seem bogus (why adding full program name?) but including the ${MINGW_PREFIX} in some way does seem needed, see e.g. https://stackoverflow.com/questions/37365355/cmake-msys-makefiles-generator-missing and using a cmake line with full path to the tool seemed(*) to just work without a terminal restart:

### Also add this to ~/.bashrc
# PATH="/mingw64/bin:$PATH"; export PATH

# /mingw64/bin/cmake -DCMAKE_BUILD_TYPE=Release -DZSTD_FROM_INTERNET=ON -DHIREDIS_FROM_INTERNET=ON -G "MSYS Makefiles" ..

(*) UPDATE: In the end it was necessary to set the PATH as added above, can be done in-shell also to avoid a terminal restart, so that the cmake, make, gcc etc. are all aligned and coming from /mingw64/... namespace.

---

It failed to build however, with -- Performing Test HAVE_NEON - Failed as the last message.

Tried to add pkgs:

# pacman -S --needed \
    libneon-devel mingw-w64-x86_64-neon 

because the test for whatever reason includes the )missing) arm_neon.h :\

Then discovered that this is not about libneon (XML parsing) but about ARM Neon CPU support... why here?..

While MSYS2 clang-11 includes the header, the compiler is not usable for ccache (lacks std::atomic it seems). Oh joy.

Added include_directories(SYSTEM /mingw64/include/c++/11.3.0) to CMakeLists.txt but it did not help, the path is not seen in error log :\

Eventuantly went for blunt brute force with ln /usr/lib/clang/11.0.0/include/arm_neon.h /usr/include which passed the build further, to fail on the pragma that the system is not for this header :)

Seems this stems from src/third_party/blake3/CMakeLists.txt where they check CPU support features, including

# Neon is always available on AArch64
if(CMAKE_SIZEOF_VOID_P EQUAL 8)
  # https://developer.arm.com/architectures/instruction-sets/simd-isas/neon/intrinsics
  check_c_source_compiles(
    [=[
      #include <arm_neon.h>
      int main() { vdupq_n_s32(42); return 0; }
    ]=]
    HAVE_NEON)
  if(HAVE_NEON)
    target_sources(blake3 PRIVATE blake3_neon.c)
    target_compile_definitions(blake3 PRIVATE BLAKE3_USE_NEON)
  endif()
endif()

which I am not sure why fails fatally and not informatively. Neuthering the block (checking for bogus pointer size 81 instead) might help - the check is skipped. But cmake still fails.

---

Also, for "sources from internet", tar complained that it somehow found Windows paths with colons and disliked those "URLs"; but it did not seem to really be a fatal error (cmake marched on):

...
tar: Cannot connect to C: resolve failed
FATAL extracting C:/Users/jim/ccache/build/hiredis-1.0.2.tar.gz failed

# pwd
/c/Users/jim/ccache/build

# realpath .
/c/Users/jim/ccache/build

MSYS2 shell seems to recognize it both ways.

But in the end, seems this was the problem for cmake; fixing up the Find* recipes helped (at expense of requiring GNU tar here), e.g.:

   execute_process(
-    COMMAND tar xf "${zstd_dir}.tar.gz"
+    COMMAND tar x --force-local -f "${zstd_dir}.tar.gz"

However this is a more generic problem, as cmake embedded e.g. C:/Users/... everywhere instead of /c/Users/... so then the make fails with includes not found (which do exist):

# make
Scanning dependencies of target blake3
[  0%] Building C object src/third_party/blake3/CMakeFiles/blake3.dir/blake3.c.obj
<command-line>: fatal error: C:/Users/jim/ccache/build/config.h: No such file or directory
compilation terminated.
...

(*) UPDATE: In the end, setting PATH="/mingw64/bin:$PATH" was usefully required to avoid this last hiccup. Is building now...