Skip to content

Limit sccache cache size to 3.5 GB to reduce transfer overhead#128048

Merged
agocke merged 2 commits into
dotnet:mainfrom
agocke:sccache-cache-size-limit
May 11, 2026
Merged

Limit sccache cache size to 3.5 GB to reduce transfer overhead#128048
agocke merged 2 commits into
dotnet:mainfrom
agocke:sccache-cache-size-limit

Conversation

@agocke
Copy link
Copy Markdown
Member

@agocke agocke commented May 11, 2026

Note

This PR was generated with the assistance of GitHub Copilot.

Summary

The sccache local cache grows unboundedly (default 10 GB max) as stale entries accumulate across rolling builds. It's currently at ~6-7 GB and growing ~0.5 GB per rolling build cycle. The cache restore/save overhead is ~3 minutes per leg, eating into the ~9 minute build time savings sccache provides.

Change

Sets SCCACHE_CACHE_SIZE=3.5G to force LRU eviction of stale entries, keeping the cache close to its working set size (~3 GB for ~3,482 compiled objects).

Expected Impact

Metric Before After
Cache size ~6-7 GB (growing) ~3.5 GB (stable)
Restore time ~1 min ~30s
Save time ~1:45 ~50s
Total overhead/leg ~3 min ~1.5 min
Net savings/leg ~6 min ~7.5 min

If hit rates drop due to premature eviction, the limit can be raised.

Copilot AI review requested due to automatic review settings May 11, 2026 18:02
The sccache local cache grows unboundedly (default 10 GB max) as stale
entries accumulate across rolling builds. Currently at ~6-7 GB and
growing ~0.5 GB per rolling build cycle, the cache restore/save
overhead is ~3 minutes per leg, eating into the ~9 minute build time
savings.

Setting SCCACHE_CACHE_SIZE to 3.5G forces LRU eviction of stale
entries, keeping the cache close to the working set size (~3 GB for
~3,482 compiled objects). This should roughly halve the cache
transfer times.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@dotnet-policy-service
Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR primarily updates the CoreCLR pipeline sccache setup to reduce cache transfer overhead by bounding the local cache size, but it also includes unrelated System.Security.Cryptography signature changes that don’t match the stated scope.

Changes:

  • Set SCCACHE_CACHE_SIZE in the sccache setup pipeline step (currently to 3.5G).
  • Adjust the Azure Pipelines Cache key/restoreKeys to exclude buildConfig.
  • Mark X25519DiffieHellmanCng.DeriveRawSecretAgreementCore partial overrides as unsafe across Windows + not-supported shims.

Reviewed changes

Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.

File Description
eng/pipelines/coreclr/templates/setup-sccache.yml Sets sccache environment variables and modifies cache keying/restoration behavior.
src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/X25519DiffieHellmanCng.Windows.cs Changes the override signature to add unsafe on the Windows implementation.
src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/X25519DiffieHellmanCng.cs Changes the partial override declaration to add unsafe.
src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/Cng.NotSupported.cs Changes the PNSE shim override signature to add unsafe.
Comments suppressed due to low confidence (1)

eng/pipelines/coreclr/templates/setup-sccache.yml:35

  • The PR description/title talks about limiting the local sccache size, but this hunk also changes the cache key/restoreKeys to drop buildConfig (affecting cache sharing behavior between PR/rolling builds). Either update the PR description to cover this behavior change and its rationale/risks, or keep this PR focused by reverting the key change.
    #
    # buildConfig is intentionally excluded from the key: it reflects
    # the *managed* Configuration (Debug for PRs, Release for rolling
    # builds), but sccache only caches native (C/C++) compilations whose
    # flags are controlled by RuntimeConfiguration (-rc), which is
    # constant per leg regardless of buildConfig.  Omitting it lets PR
    # builds warm-start from rolling-build caches saved in the main scope.
    - task: Cache@2
      displayName: Sccache cache
      inputs:
        key: sccache | ${{ parameters.osGroup }} | ${{ parameters.archType }} | ${{ parameters.nameSuffix }} | "$(Build.BuildId)"
        path: $(Pipeline.Workspace)/.sccache
        restoreKeys: |
          sccache | ${{ parameters.osGroup }} | ${{ parameters.archType }} | ${{ parameters.nameSuffix }}

Comment thread eng/pipelines/coreclr/templates/setup-sccache.yml Outdated
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 11, 2026 18:09
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.

Comment thread eng/pipelines/coreclr/templates/setup-sccache.yml
@agocke
Copy link
Copy Markdown
Member Author

agocke commented May 11, 2026

/ba-g all affected legs have passed

@agocke agocke merged commit 4dad00f into dotnet:main May 11, 2026
109 of 122 checks passed
@agocke agocke deleted the sccache-cache-size-limit branch May 11, 2026 21:41
@github-project-automation github-project-automation Bot moved this to Done in AppModel May 11, 2026
agocke added a commit to agocke/runtime that referenced this pull request May 11, 2026
Widen the sccache condition from osSubgroup=='' to also include
osSubgroup=='_musl'. The sccache NuGet package (0.15.0) is a
statically-linked binary that works on both glibc and musl.

The cache key now includes osSubgroup to keep musl and glibc caches
separate (different system headers/compiler flags produce different
object files). For non-musl builds, osSubgroup is empty so the key
is unchanged — existing caches remain valid.

Also includes the SCCACHE_CACHE_SIZE=3.5G limit from PR dotnet#128048 to
prevent unbounded cache growth.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
agocke added a commit to agocke/runtime that referenced this pull request May 12, 2026
Widen the sccache condition from osSubgroup=='' to also include
osSubgroup=='_musl'. The sccache NuGet package (0.15.0) is a
statically-linked binary that works on both glibc and musl.

The cache key now includes osSubgroup to keep musl and glibc caches
separate (different system headers/compiler flags produce different
object files). For non-musl builds, osSubgroup is empty so the key
is unchanged — existing caches remain valid.

Also includes the SCCACHE_CACHE_SIZE=3.5G limit from PR dotnet#128048 to
prevent unbounded cache growth.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
agocke added a commit that referenced this pull request May 12, 2026
> [!NOTE]
> This PR was generated with the assistance of GitHub Copilot.

## Summary

Extends sccache caching from linux-x64 to also cover linux-musl-x64
coreclr build legs. Based on the linux-x64 results (~9 min/leg gross
savings, 97.59% hit rate at steady state), this should provide similar
speedups for the musl legs.

## Changes

- **setup-sccache.yml**: Widen condition to include `osSubgroup ==
'_musl'`; add `osSubgroup` to cache key to keep musl/glibc caches
separate
- **sccache-stats.yml**: Widen condition to match

## Why this works

- The sccache NuGet package (v0.15.0) is a **statically-linked** binary
— verified with `file` showing `static-pie linked`. Works on both glibc
and musl.
- The sccache package is already downloaded by `runtime-prereqs.proj` on
all Linux CI builds (line 13-14, condition: `IsOsPlatform(Linux)`).
- Cache keys include `osSubgroup` so musl builds get
`sccache|linux_musl|x64|...` vs `sccache|linux|x64|...` for glibc — no
cross-contamination.
- For non-musl builds, `osSubgroup` is empty, so the key is
**unchanged** — existing caches remain valid.

## Also included

`SCCACHE_CACHE_SIZE=3.5G` limit (from #128048) to prevent unbounded
cache growth.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Done
Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants