Skip to content

Commit d629173

Browse files
committed
fix(ci): unify redundant LLVM cache keys
Recent PR runs exceeded GitHub Actions cache limits, causing LLVM to be recompiled on every workflow run. This led to hours-long turnaround times to verify changes. As a workaround, the fork workflow was being used to test the PR. This commit simplifies the cache key structure to reduce redundant cache entries: - Add `llvm-os-key` using container name or runs-on for OS versioning - Make `llvm-archive-basename` conditional based on MSan only - Transform cache keys to replace ":" with "-" for safety - Only build instrumented libc++ for MSan where it's required MSan requires all code, including libc++ to be instrumented to track initialization state correctly. ASan and UBSan can effectively catch errors without instrumenting dependencies, making custom libc++ builds low-ROI for these sanitizers. Resulting cache key structure: - MSan builds: include compiler, version, and sanitizer type (e.g., llvm-dc4cef8-release-ubuntu-24.04-clang-21-MSan) - All other builds: omit compiler info since ABI is compatible (e.g., llvm-dc4cef8-release-ubuntu-24.04) Non-sanitizer builds, ASan builds, and UBSan builds across gcc/clang versions on the same OS now share a single cache entry, as they produce ABI-compatible LLVM binaries. Fixes #981
1 parent 4e7ef04 commit d629173

1 file changed

Lines changed: 13 additions & 5 deletions

File tree

.github/workflows/ci.yml

Lines changed: 13 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -74,16 +74,16 @@ jobs:
7474
clang: git build-essential pkg-config python3 curl unzip openjdk-11-jdk pkg-config libncurses-dev libxml2-utils libxml2-dev g++-14=14.2.0-4ubuntu2~24.04
7575
msvc: ''
7676
extra-values: |
77-
use-libcxx: {{#if (and (ieq compiler 'clang') (ge major 19)) }}true{{else}}false{{/if}}
77+
use-libcxx: {{#if (and (ieq compiler 'clang') msan) }}true{{else}}false{{/if}}
7878
libcxx-runtimes: libcxx{{#if (ne compiler 'msvc')}};libcxxabi{{/if}}
7979
llvm-runtimes: {{#if (ine use-libcxx 'true') }}{{{ libcxx-runtimes }}}{{/if}}
8080
llvm-hash: dc4cef81d47c7bc4a3c4d58fbacf8a6359683fae
8181
llvm-build-preset-prefix: {{{lowercase build-type}}}
8282
llvm-build-preset-os: {{#if (ieq os 'windows') }}win{{else}}unix{{/if}}
8383
llvm-sanitizer: {{#if (eq compiler 'gcc')}}{{else if ubsan}}-UBSan{{else if asan}}-ASan{{else if msan}}-MSan{{/if}}
8484
llvm-build-preset: {{{ llvm-build-preset-prefix }}}-{{{ llvm-build-preset-os }}}
85-
llvm-compiler-version: {{#if (or (contains version '*') (contains version '^'))}}{{else}}-{{{ version }}}{{/if}}
86-
llvm-archive-basename: llvm-{{{ lowercase os }}}-{{{ compiler }}}{{{ llvm-compiler-version }}}-{{{ llvm-build-preset-prefix }}}{{{ llvm-sanitizer }}}-{{{ substr llvm-hash 0 7 }}}
85+
llvm-os-key: {{#if container}}{{{ container }}}{{else}}{{{ runs-on }}}{{/if}}
86+
llvm-archive-basename: {{#if msan}}llvm-{{{ substr llvm-hash 0 7 }}}-{{{ llvm-build-preset-prefix }}}-{{{ llvm-os-key }}}-{{{ compiler }}}-{{{ version }}}-MSan{{else}}llvm-{{{ substr llvm-hash 0 7 }}}-{{{ llvm-build-preset-prefix }}}-{{{ llvm-os-key }}}{{/if}}
8787
llvm-archive-extension: {{#if (ieq os 'windows') }}7z{{else}}tar.bz2{{/if}}
8888
llvm-archive-filename: {{{ llvm-archive-basename }}}.{{{ llvm-archive-extension }}}
8989
llvm-sanitizer-config: {{#if (and (ne compiler 'clang') (ne compiler 'apple-clang'))}}{{else if ubsan}}Undefined{{else if asan}}Address{{else if msan}}MemoryWithOrigins{{/if}}
@@ -322,12 +322,16 @@ jobs:
322322
fi
323323
echo "common-ldflags=$common_ldflags" >> $GITHUB_OUTPUT
324324
325+
llvm_cache_key="${{ matrix.llvm-archive-basename }}"
326+
llvm_cache_key="${llvm_cache_key//:/-}"
327+
echo "llvm-cache-key=$llvm_cache_key" >> $GITHUB_OUTPUT
328+
325329
- name: Cached LLVM Binaries
326330
id: llvm-cache
327331
uses: actions/cache@v4
328332
with:
329333
path: ${{ steps.rmatrix.outputs.llvm-path }}
330-
key: ${{ matrix.llvm-archive-basename }}
334+
key: ${{ steps.rmatrix.outputs.llvm-cache-key }}
331335

332336
# Installs libc++ separately, using the LLVM standalone runtimes build.
333337
# The libc++ built here will be built using the host compiler, so this is
@@ -1229,6 +1233,10 @@ jobs:
12291233
llvm_path=$third_party_dir/llvm
12301234
echo "llvm-path=$llvm_path" >> $GITHUB_OUTPUT
12311235
1236+
llvm_cache_key="${{ matrix.llvm-archive-basename }}"
1237+
llvm_cache_key="${llvm_cache_key//:/-}"
1238+
echo "llvm-cache-key=$llvm_cache_key" >> $GITHUB_OUTPUT
1239+
12321240
- name: Install packages
12331241
uses: alandefreitas/cpp-actions/package-install@v1.9.1
12341242
id: package-install
@@ -1240,4 +1248,4 @@ jobs:
12401248
uses: actions/cache@v4
12411249
with:
12421250
path: ${{ steps.rmatrix.outputs.llvm-path }}
1243-
key: ${{ matrix.llvm-archive-basename }}
1251+
key: ${{ steps.rmatrix.outputs.llvm-cache-key }}

0 commit comments

Comments
 (0)