Enable sccache for macOS CoreCLR legs#128675
Open
steveisok wants to merge 5 commits into
Open
Conversation
…ch wrapper) Consolidate the -DCMAKE_C/CXX_COMPILER_LAUNCHER injection into eng/native/build-commons.sh so every native component invoked through build_native picks up sccache when USE_SCCACHE=true (CoreCLR, corehost, native libraries, etc.) instead of only the components that hand-rolled the flag themselves. On macOS, CMake wraps PCH includes in -Xarch_<arch> -include<path>, which sccache cannot parse and which also strips bare -include during preprocessing. Use a thin wrapper script that drops -Xarch_<arch> (safe in single-architecture builds) and rewrites -include<path> to -Xclang -include -Xclang <path>, which sccache passes through to the clang frontend correctly. Remove the now-redundant per-component sccache flag injection from src/coreclr/build-runtime.sh; build-commons.sh handles it for all build_native invocations. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Widen the gate in setup-sccache.yml and sccache-stats.yml to also include osx + arm64 alongside linux + x64. The CoreCLR_Libraries leg in runtime.yml already lists osx_arm64 and references both templates, so widening the gate activates Mac sccache there without any change to runtime.yml. On linux/freebsd we keep using the linux-musl-x64 sccache NuGet package (also works for arm64 cross-compile on x64 hosts, per dotnet#128065). The NuGet package has no Mac binary, so on macOS we download the official tarball from the mozilla/sccache GitHub release matching the host CPU (uname -m, not archType: the sccache executable must match the agent's CPU). A pinned sha256 guards against tarball tampering and surfaces transient GitHub-side issues. --strip-components=1 places the binary directly in the tools directory so the existing prependpath line still finds it. Bump SCCACHE_CACHE_SIZE to 5120M to match dotnet#128065. The 3584M limit was evicting useful entries; the larger cache keeps hit rates consistent across the now-larger fleet of jobs. Starting narrow on osx_arm64 only: the xarch wrapper in eng/native/sccache-xarch-wrapper.sh assumes single-arch builds, so we widen to osx_x64 / additional Mac legs in follow-ups after validating server startup and cache hits in CI. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Two changes together actually activate sccache on the x64 leg: 1. Widen the gate in setup-sccache.yml and sccache-stats.yml to also include (osGroup == osx && archType == x64). Without this, the templates short-circuit and do nothing on the x64 platform. 2. Reference both templates from the osx_x64 CoreCLR_Libraries job in runtime.yml (the one that builds for installer tests). That job previously had no sccache plumbing at all, so widening the gate alone would have been a no-op. The xarch wrapper script is target-arch agnostic — it strips any -Xarch_<arch> (matches both -Xarch_arm64 cross-builds and -Xarch_x86_64 native builds) and its only correctness precondition is single-arch builds, which build-commons.sh already enforces via single-valued CMAKE_OSX_ARCHITECTURES. The setup-sccache.yml binary picker uses uname -m to select the sccache executable, so an x64 host that cross-compiles arm64 still gets the correct x86_64-apple- darwin sccache binary while the cache key continues to differentiate target archs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Wire setup-sccache.yml / sccache-stats.yml into the macOS legs in runtime.yml whose buildArgs are clean CoreCLR/libs/host invocations. Mono-flavored legs are deliberately excluded for this PR (see below). Legs added: - Libraries_CheckedCoreCLR (osx_arm64, osx_x64) - NativeAOT (osx_arm64) - NativeAOT_Libraries (osx_arm64) - Installer_Build_And_Test (osx_x64, osx_arm64) For Installer_Build_And_Test the new setup-sccache template is appended to the existing preBuildSteps (which downloads the CoreCLR_Libraries artifact); sccache-stats is inserted as the first postBuildStep so stats are emitted even if later steps fail. Mono legs (AllSubsets_Mono, AllSubsets_Mono_LLVMAOT, the *_RuntimeTests variants, Mono_MiniJIT_LibrariesTests) are intentionally NOT wired. src/mono/mono.proj invokes cmake directly via _MonoCMakeArgs and bypasses eng/native/build-commons.sh, so the launcher injection on this branch does not reach Mono's native build. Wiring sccache into those legs would download the binary and report a misleading hit rate dominated by Mono cache-misses without speeding the dominant work. Extending Mono itself to use sccache (by adding the launcher flags to _MonoCMakeArgs when USE_SCCACHE=true) is left as a follow-up. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
github-actions
Bot
added
the
area-Infrastructure-coreclr
Contributor
There was a problem hiding this comment.
Pull request overview
This PR wires up sccache for additional macOS CI legs by centralizing the compiler-launcher injection in eng/native/build-commons.sh, adding a macOS-specific argument-rewrite wrapper, and enabling the existing sccache setup/stats templates across the targeted pipeline jobs.
Changes:
- Move
CMAKE_*_COMPILER_LAUNCHERinjection behindUSE_SCCACHE=trueintoeng/native/build-commons.sh(and remove the older injection fromsrc/coreclr/build-runtime.sh). - Add
eng/native/sccache-xarch-wrapper.shto rewrite macOS-Xarch_*/-include...argument patterns before invokingsccache. - Extend pipeline templates and job definitions to download/configure sccache on macOS and print cache stats in the selected CI legs.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/coreclr/build-runtime.sh | Removes redundant sccache launcher injection now handled in shared native build logic. |
| eng/native/build-commons.sh | Adds centralized USE_SCCACHE handling and selects the macOS wrapper as the launcher on Apple targets. |
| eng/native/sccache-xarch-wrapper.sh | New wrapper script to rewrite Apple-specific compiler flags for sccache compatibility. |
| eng/pipelines/runtime.yml | Enables sccache setup + stats templates on additional macOS-related pipeline legs. |
| eng/pipelines/coreclr/templates/setup-sccache.yml | Adds macOS download path (GitHub release + SHA256 pin) and bumps cache size. |
| eng/pipelines/coreclr/templates/sccache-stats.yml | Expands stats collection gating to include macOS legs. |
Comment on lines
+771
to
+774
| preBuildSteps: | ||
| - template: /eng/pipelines/coreclr/templates/setup-sccache.yml | ||
| postBuildSteps: | ||
| - template: /eng/pipelines/coreclr/templates/sccache-stats.yml |
Comment on lines
+14
to
18
| # sccache version. On linux/freebsd we use the linux-musl-x64 NuGet | ||
| # package; on macOS we download the matching tag from the mozilla/sccache | ||
| # GitHub release. Keep the version in sync across platforms so all | ||
| # jobs cache against the same compiler-launcher behavior. | ||
| sccacheVersion: '0.15.0' |
This was referenced May 28, 2026
Open
Member
Author
|
I ran builds on this PR twice. On the second run it shows sccache being used. Note the installer leg was still running when I collected this data.
|
Member
|
@steveisok Awesome! Sorry, I merged and now you have merge conflicts :) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bring up sccache compile caching for macOS CI legs in dotnet/runtime, mirroring the linux/freebsd rollout from #128065.
Scope
Wires sccache into 8 macOS CI legs that build native code through
eng/native/build-commons.sh:The two template files (
setup-sccache.yml,sccache-stats.yml) are gated onosGroup == osx && archType ∈ (arm64, x64), so adding them to mixed-platform matrix entries is a no-op on non-macOS platforms.How it works
eng/native/build-commons.sh— whenUSE_SCCACHE=trueis set in the environment,build_native()injects-DCMAKE_C_COMPILER_LAUNCHER=…and-DCMAKE_CXX_COMPILER_LAUNCHER=…into the CMake arguments. On linux/freebsd the launcher is baresccache; on macOS/maccatalyst it is the new wrapper described below.eng/native/sccache-xarch-wrapper.sh(new) — On macOS, CMake emits-Xarch_<arch> -include<path>for precompiled headers. Two issues for sccache:-Xarch_<arch>is an Apple-specific flag sccache doesn't recognize.-include<path>(glued together) prevents sccache from seeing the PCH path and hashing it correctly.The wrapper strips
-Xarch_<arch>and rewrites-include<path>into-Xclang -include -Xclang <path>beforeexec-ing sccache. Stripping-Xarch_<arch>is safe becausebuild-commons.shalways setsCMAKE_OSX_ARCHITECTURESto a single value (lines 94/96, 147/149, 161, 173/175, 187), so each compiler invocation has exactly one target arch.eng/pipelines/coreclr/templates/setup-sccache.yml— Downloads the sccache binary per-osGroup: NuGet for linux/freebsd (existing), GitHub release tarball for macOS with SHA-256 pinning. Tarball usestar --strip-components=1. Cache size bumped 3584M → 5120M (matching Enable sccache for arm64 and FreeBSD cross-builds #128065). Cache key includes${osGroup}${osSubgroup}|${archType}|${nameSuffix}|$(Build.BuildId)with rolling restoreKeys so per-platform/arch caches stay isolated.eng/pipelines/coreclr/templates/sccache-stats.yml— Same gate; runssccache --show-statsafter the build.Inspired by
#128065 (sccache for linux/freebsd cross-compile scenarios) — we adopt its cache-size bump and scope-widening pattern.