Use ccache in Github Actions

[WeiqunZhang]

This works very well for CUDA and HIP builds. However, the benefit is relatively small for GCC, Clang and Mac builds because most of their time are spent on clang-tidy and running tests.

There are two remaining issues. Ccache works with Intel classic compiler icpc, but it does not work with the Intel SYCL compiler, icpx. On Windows, MSVC works, but I cannot get ClangCl to work. We can revisit these two issues in the future.

In earlier versions of this PR, we compiled ccache from source because the package version on Ubuntu 20.04 is too old. Because of that, we had to move env: {CXXFLAGS: ...} down to AMReX's Build & Install section to avoid the flags being picked up when compiling ccache. In the current version, we download the binary version of ccache 4.8. However, we did not move the env setup back. It seems to make sense to keep it in the Build & Install section.

To use github cache, we need to provide a primary key and optionally restore keys. If the primary key is found, it will be used, but the change to the cache during the CI will NOT be saved at the end of the job. That's how github cache works. If it is not found, partially matched restore keys will be considered and the latest one will be used. At the end, the cache (whether it's updated or not) will be saved under the name of the primary key. Below is an example of our setup.

    with:
        path: ~/.cache
        key: ccache-${{ github.workflow }}-${{ github.job }}-git-${{ github.sha }}
        restore-keys: |
             ccache-clang.yml-${{ env.cache-name }}-git-

Here, we use the workflow string and job string as part of the names for the keys. We also include the git commit hash ${{github.sha}} in the name of the primary key. In this setup, the CI will always fail to find the primary key. This is what we want because it guarantees the latest ccache data will be saved. Furthermore, the most recently saved cache from the last CI job will be used to restore the cache during initialization.

[WeiqunZhang]

After this is merged, we can do a follow-up to fine tune the cache eviction policy. https://github.com/actions/cache/blob/main/tips-and-workarounds.md#force-deletion-of-caches-overriding-default-cache-eviction-policy

[WeiqunZhang]

The idea for cache eviction policy is as follows. Using https://github.com/actions/gh-actions-cache/, we can delete all caches associated with the current PR after they are used. Note that the post-cache step will still store the cache's cache under the primary github cache key. So after the action finishes, there will still be a github cache for that PR. Additionally we remove all its caches upon the close of a PR.

[WeiqunZhang]

The follow-up is #3226.

[WeiqunZhang]

Regarding the Intel SYCL compiler issue, there is an open issue here intel/llvm#5260.