Skip to content

BUILD-11294 Add cache-size-bytes output and pipeline runtime metrics JSON#62

Merged
julien-carsique-sonarsource merged 2 commits into
masterfrom
feat/jcarsique/BUILD-11294-metrics
May 18, 2026
Merged

BUILD-11294 Add cache-size-bytes output and pipeline runtime metrics JSON#62
julien-carsique-sonarsource merged 2 commits into
masterfrom
feat/jcarsique/BUILD-11294-metrics

Conversation

@julien-carsique-sonarsource
Copy link
Copy Markdown
Contributor

@julien-carsique-sonarsource julien-carsique-sonarsource commented May 12, 2026

Summary

Adds the building block that the BUILD-11068 pipeline-runtime-metrics workstream needs from gh-action_cache: an opt-in per-invocation JSON record at ${CI_METRICS_DIR}/cache-${step}.json (defaults to /tmp/ci-metrics; emission gated by CI_METRICS_ENABLED=true) and four new top-level outputs surfacing the cache state to downstream workflow steps. The M1.3 job-completed.sh hook is the direct consumer of the JSON files.

New top-level outputs

Output Description
cache-hit (unchanged) Boolean exact-match on primary key.
cache-matched-key (new) Mirrors actions/cache's output — primary key on exact hit, restore key on partial, empty on miss.
restore-key-hit (new) The prefix-matched restore key — set only on partial hits (empty on exact hit and full miss). Higher-level convenience over cache-matched-key.
backend (new) Which backend was actually used (github or s3).
cache-size-bytes (new) Size of cached path(s) at restore-time (Linux only; empty on other platforms).

JSON record schema and the four scenarios

${CI_METRICS_DIR}/cache-${step}.json (defaults to /tmp/ci-metrics; only emitted when CI_METRICS_ENABLED=true) is written by the new cache-metrics sub-action where ${step} is the slugified github.action of the calling step. Multiple cache calls per job produce distinct files.

Scenario cache-hit restore-key-hit saved size-bytes-restored size-bytes-at-end
Exact primary-key match true null false restored size path size at end (NOT saved)
Partial / restore-key hit false <the matched restore key> true size of older partial cache size of the new cache (= what's saved)
No match at all false null true 0 size of from-scratch cache (= saved)
lookup-only: true (true or false) null false 0 path size at end (NOT saved)

saved is false whenever the underlying cache action skips the post-step save — i.e. on exact hits (cache keys are content-immutable in actions/cache / runs-on/cache) or when lookup-only is set. Full field semantics + examples are in README.md and in the Jira ticket comment 914885.

Architecture

  • New cache-metrics/ Node 20 sub-action with main + post (mirrors the credential-guard pattern at the file-layout level; lives next to credential-setup/ and credential-guard/).
  • Top-level composite invokes it on Linux runners only, after the last restore branch (Enforce fail-on-cache-miss → cache-metrics → Credential guard). LIFO post order on the S3 path: credential-guard (restore AWS creds) → cache-metrics (measure pre-save size + update JSON) → s3-cache (save to S3). cache-metrics never needs AWS creds.
  • Resolution mechanism: sub-action is referenced via ./.actions/cache-metrics, symlinked at job-start to $GITHUB_ACTION_PATH/cache-metrics. The symlink is git add -f'd so it survives git clean -ffdx (which previously broke the workspace-copy pattern — see #39). Works for both uses: ./ (dev tests in this repo) and external consumers using SonarSource/gh-action_cache@v1. Survives the test-s3-cache-survives-git-clean regression.
  • Fail-open: any measurement error surfaces as core.warning, never breaks the cache flow.

Behavioral changes

  • 4 new top-level outputs (listed above). No removed outputs.
  • New opt-in side-effect (gated by CI_METRICS_ENABLED=true): writes ${CI_METRICS_DIR}/cache-${step}.json (directory defaults to /tmp/ci-metrics; created defensively if absent). When the flag is unset/empty/any-other-value, no metrics step runs and cache-size-bytes is empty.
  • New runtime workspace mutation: creates .actions/cache-metrics symlink in $GITHUB_WORKSPACE and git add -f's it. Ephemeral; never persisted to a commit.
  • No changes to existing inputs, restore-key semantics, credential-isolation flow, or the cleanup sub-action.

Tests

  • 22 new Vitest cases in __tests__/cache-metrics.test.ts covering helpers, main, post, and all four scenarios. Repo total: 45 cases pass.
  • New CI assertion job test-cache-metrics-output validates that cache-size-bytes is set and ${CI_METRICS_DIR}/cache-*.json is written with the expected schema.
  • All existing regression tests still pass: test-github-cache, test-s3-cache* (multi-invocations, with-fallback, with-credential-interference, with-preset-aws-config, survives-git-clean, Windows), tests-credential-isolation (sonar-dummy downstream — exercises real consumer pattern), pre-commit, SonarCloud / SonarCloud Code Analysis.
  • https://github.com/SonarSource/sonar-dummy/pull/592

Test plan

  • npm test — 45 Vitest cases pass (22 new in cache-metrics.test.ts)
  • npm run build — all five ncc bundles produced
  • pre-commit run --all-files — clean
  • New CI job test-cache-metrics-output validates the JSON + the cache-size-bytes output
  • All existing test jobs still green — no regression in the credential-isolation flow
  • Windows test job passes via the runner.os == 'Linux' skip (output is empty on Windows, no JSON written)
  • SonarCloud + SonarCloud Code Analysis quality gates green

Related

Note for the reviewer

The ticket was originally scoped as 2 SP (S) for "compute du -sb once on the cached path." Scope expanded during implementation to capture the post-step size + saved flag, plus the additional top-level outputs (cache-matched-key, restore-key-hit, backend) for downstream-step convenience. Ticket re-sized to 5 SP (M) in Jira.

The PR was originally two commits (BUILD-11294 + the unrelated Add CLAUDE.md produced by /init); the BUILD-11294 work was squashed into a single commit for clean review.

Comment thread __tests__/cache-metrics.test.ts Fixed
Comment thread __tests__/cache-metrics.test.ts Fixed
Comment thread __tests__/cache-metrics.test.ts Fixed
Comment thread __tests__/cache-metrics.test.ts Fixed
Comment thread src/cache-metrics.ts Fixed
Comment thread __tests__/cache-metrics.test.ts Fixed
Comment thread __tests__/cache-metrics.test.ts Fixed
Comment thread __tests__/cache-metrics.test.ts Fixed
Comment thread __tests__/cache-metrics.test.ts Fixed
Comment thread src/cache-metrics.ts Fixed
@julien-carsique-sonarsource julien-carsique-sonarsource force-pushed the feat/jcarsique/BUILD-11294-metrics branch 5 times, most recently from 6aafd88 to 2000f2a Compare May 13, 2026 08:23
@julien-carsique-sonarsource
Copy link
Copy Markdown
Contributor Author

--- /tmp/ci-metrics/cache-cache-metrics-test.json ---
{
  "step": "cache-metrics-test",
  "key": "metrics-test-Linux-25787381819",
  "matched-key": "",
  "backend": "s3",
  "cache-hit": false,
  "size-bytes-restored": 3729300,
  "size-bytes-saved": null,
  "saved": null,
  "timestamp-restored": "2026-05-13T08:24:34.393Z",
  "timestamp-saved": null
}

@julien-carsique-sonarsource julien-carsique-sonarsource force-pushed the feat/jcarsique/BUILD-11294-metrics branch from 8797eff to d15f7f3 Compare May 13, 2026 10:39
Initial Claude Code guidance file produced via /init. Documents the build/test
commands, the critical ncc-bundle commit rule, the composite-action + sub-action
architecture, the credential-isolation rationale tied to each regression test
job, the backend & cache-key resolution chain, and the pinned tooling versions.
Points at README.md and action.yml as the source of truth for user-facing inputs
to avoid duplication.
@julien-carsique-sonarsource julien-carsique-sonarsource force-pushed the feat/jcarsique/BUILD-11294-metrics branch 2 times, most recently from 173cd91 to 1d3db98 Compare May 13, 2026 12:22
@julien-carsique-sonarsource julien-carsique-sonarsource marked this pull request as ready for review May 13, 2026 12:24
@julien-carsique-sonarsource julien-carsique-sonarsource requested a review from a team as a code owner May 13, 2026 12:24
@sonar-review-alpha
Copy link
Copy Markdown

sonar-review-alpha Bot commented May 13, 2026

Summary

⚠️ The PR description exceeded the analysis limit and was truncated. The review may not reflect all context.

This PR adds cache metrics collection to support BUILD-11068's pipeline-runtime-metrics workstream. It introduces:

4 new top-level outputs: cache-matched-key (primary on exact hit, restore-key on partial), restore-key-hit (populated only on partial hits for convenience), backend (which backend was used), and cache-size-bytes (path size at restore-time, Linux only).

Optional JSON record emission: When CI_METRICS_ENABLED=true, the action writes ${CI_METRICS_DIR}/cache-${step}.json with restore-time and job-end measurements, plus metadata about whether the cache was saved. This feeds the M1.3 job-completed.sh hook.

New sub-action: cache-metrics/ with Node 20 main + post steps that measure cache path sizes and track the four cache scenarios (exact hit, partial hit, miss, lookup-only). The main step measures after restore and outputs cache-size-bytes; the post step updates the JSON record at job-end with whether the cache was actually persisted.

Zero behavioral changes for users who don't set CI_METRICS_ENABLED. The feature is opt-in, Linux-only, and architecturally fail-open (any measurement error becomes a warning, never breaks the cache flow).

What reviewers should know

Start with: action.yml to see the orchestration — note the new output aggregation step (steps.cache-outputs) that centralizes the triple-OR logic across three possible restore backends (github, s3, import). Then read src/cache-metrics.ts for the data model and measurement logic.

Key architecture decisions:

  • Symlink trick (action.yml lines ~245-260): The sub-action is referenced via ./.actions/cache-metrics (a symlink created at job-start). It's staged with git add -f so it survives git clean -ffdx between main and post steps. This avoids the workspace-copy pattern used before PR #39.
  • Fail-open design: Both main and post steps wrap their core logic in try-catch and emit warnings on measurement errors, never breaking the cache flow. The post-if: always() in cache-metrics/action.yml ensures measurement happens even if the job partially failed.
  • State passing: Main step saves inputs to GitHub Actions state (core.saveState); post step retrieves them to know which file to update and whether the cache was hit/lookup-only. The JSON file is then reread and updated with post-step measurements.
  • Backend aggregation: The new aggregation step (lines ~241-252) consolidates cache-hit and cache-matched-key from all three possible cache restore branches into a single source of truth. Both the top-level outputs and the cache-metrics sub-action read from this, avoiding duplication. Adding a fourth backend would only touch this step.
  • Linux-only measurement: The du command is Linux-specific; the main and post steps early-return on other platforms.
  • Feature flag: CI_METRICS_ENABLED (case-insensitive true) gates the entire metrics flow. When unset/false, no metrics step runs and cache-size-bytes output is empty.

Test coverage: 22 new Vitest cases in cache-metrics.test.ts cover helpers (slugify, measurement, file I/O), the four cache scenarios, and error paths. All 45 total tests pass. CI adds a test-cache-metrics-output job that validates the JSON schema and presence of cache-size-bytes output on Linux.

Things to verify:

  1. The symlink resolution (lines ~245-260): ensure ./.actions/cache-metrics points to the right place on both uses: ./ (this repo) and external consumers.
  2. The aggregation logic (lines ~241-252): confirm the triple-OR correctly prioritizes github > s3 > import for cache-hit and cache-matched-key.
  3. The restore-key-hit top-level output expression (action.yml line ~56): it's a complex conditional; verify it matches the sub-action's calculation (cache-metrics-main.ts line 32).
  4. The state flow: main saves 5 pieces of state (file path, path input, cache-hit, lookup-only, and implicitly the metrics file contents), post retrieves 3 of them. Confirm nothing is lost between steps.
  5. Post-step ordering on the S3 path: credential-guard (restore creds) → cache-metrics (measure + update JSON) → s3-cache (save). cache-metrics never needs AWS creds, so this is safe.

  • Generate Walkthrough
  • Generate Diagram

🗣️ Give feedback

sonar-review-alpha[bot]

This comment was marked as resolved.

@julien-carsique-sonarsource julien-carsique-sonarsource force-pushed the feat/jcarsique/BUILD-11294-metrics branch 2 times, most recently from 3ae9aea to a8cc7b6 Compare May 13, 2026 13:30
sonar-review-alpha[bot]

This comment was marked as resolved.

@julien-carsique-sonarsource julien-carsique-sonarsource force-pushed the feat/jcarsique/BUILD-11294-metrics branch from a8cc7b6 to 712fb29 Compare May 13, 2026 13:46
Comment thread __tests__/cache-metrics.test.ts Fixed
Comment thread __tests__/cache-metrics.test.ts Fixed
Comment thread src/cache-metrics.ts Fixed
Comment thread __tests__/cache-metrics.test.ts Fixed
Comment thread __tests__/cache-metrics.test.ts Fixed
Comment thread src/cache-metrics.ts Fixed
sonar-review-alpha[bot]

This comment was marked as resolved.

@julien-carsique-sonarsource julien-carsique-sonarsource force-pushed the feat/jcarsique/BUILD-11294-metrics branch from 712fb29 to a65e68b Compare May 13, 2026 14:20
sonar-review-alpha[bot]

This comment was marked as resolved.

@julien-carsique-sonarsource julien-carsique-sonarsource force-pushed the feat/jcarsique/BUILD-11294-metrics branch from a65e68b to e2439c6 Compare May 13, 2026 16:32
sonar-review-alpha[bot]

This comment was marked as resolved.

@julien-carsique-sonarsource julien-carsique-sonarsource force-pushed the feat/jcarsique/BUILD-11294-metrics branch from e2439c6 to 337e669 Compare May 13, 2026 16:48
sonar-review-alpha[bot]

This comment was marked as resolved.

@julien-carsique-sonarsource julien-carsique-sonarsource force-pushed the feat/jcarsique/BUILD-11294-metrics branch from 337e669 to dfe936e Compare May 13, 2026 16:53
sonar-review-alpha[bot]

This comment was marked as resolved.

Copy link
Copy Markdown
Contributor

@jayadeep-km-sonarsource jayadeep-km-sonarsource left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The approach looks good to me. I didn't fully understand the cache size calculation part. It would be nice if we can add some tests to ensure that the calculation is correct

sonar-review-alpha[bot]

This comment was marked as outdated.

…step}.json

Add a new `cache-metrics` Node 20 sub-action (main + post) that captures cache
size and save status for every invocation of `gh-action_cache`, feeding the
pipeline-runtime-metrics workstream (BUILD-11068 / job-completed.sh, BUILD-11291).

## Action outputs (top-level)

- `cache-hit` (unchanged): exact match on primary key.
- `cache-matched-key` (new): mirrors `actions/cache`'s output — primary key on
  exact hit, restore key on partial, empty on miss.
- `restore-key-hit` (new): the prefix-matched restore key, set ONLY on partial
  hits (empty on exact hit and full miss).
- `backend` (new): which backend was used (`github` or `s3`).
- `cache-size-bytes` (new): size of cached path(s) at restore-time (Linux only).

## Per-invocation JSON record (Linux only)

`/tmp/ci-metrics/cache-${step}.json` is written by the cache-metrics sub-action
where `${step}` is the slugified id of the calling step. Schema covers four
scenarios (see Jira BUILD-11294 comment 914885 + README):

  | scenario       | cache-hit | restore-key-hit  | saved | restored size | end-of-job size |
  |----------------|-----------|------------------|-------|---------------|-----------------|
  | exact hit      | true      | null             | false | restored      | path at end     |
  | partial hit    | false     | <restore-key>    | true  | older partial | new (saved) cache|
  | no match       | false     | null             | true  | 0             | new (saved) cache|
  | lookup-only    | -         | null             | false | 0             | path at end     |

The cache action skips save on an exact hit (cache keys are content-immutable
in actions/cache and runs-on/cache), so `saved=false` even though
`size-bytes-at-end` may differ from `size-bytes-restored` when the user
modified the cached path during the job.

## Architecture

- New `cache-metrics/` Node 20 sub-action with main + post.
- Top-level composite invokes it on Linux runners only, after the last restore
  branch, so both the GitHub and S3 backends are covered.
- Sub-action is referenced via `./.actions/cache-metrics` symlinked at job-start
  to `\$GITHUB_ACTION_PATH/cache-metrics`. The symlink is `git add -f`'d so it
  survives `git clean -ffdx` (which previously broke similar workspace-copy
  patterns — see PR #39). Survives both clean and post-step re-resolution.
- Step fails open: any measurement error surfaces as `core.warning`, never
  breaks the cache flow.

## Tests

- 22 new Vitest cases in `__tests__/cache-metrics.test.ts` covering helpers,
  main, post, all four scenarios.
- New CI assertion job `test-cache-metrics-output` validates the JSON file is
  written and the `cache-size-bytes` output is set end-to-end.
- All existing regression tests still pass (credential isolation, multi-cache
  invocations, Windows, preset AWS profile, git-clean survival).

## Misc

- `.pre-commit-config.yaml`: extended trailing-whitespace / EOF /
  check-added-large-files excludes to cover all `*/dist/` directories (ncc
  bundles aren't formatted by us).
- README: new Outputs rows, Pipeline runtime metrics section with two example
  JSONs (Scenario 1 exact hit + Scenario 2 partial restore) and a
  field-semantics table.
@sonarqubecloud
Copy link
Copy Markdown

@sonarqube-cloud-us
Copy link
Copy Markdown

Copy link
Copy Markdown

@sonar-review-alpha sonar-review-alpha Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! ✅

The only change in this incremental diff is a comment simplification in the test workflow — removing the reference to "runner pod template / WarpBuild AMI" from the CI_METRICS_ENABLED env var comment. This is a cosmetic-only change with no functional impact. The simplified comment is accurate for the test context (the test explicitly opts in), while the authoritative infrastructure documentation lives in action.yml.

🗣️ Give feedback

@julien-carsique-sonarsource julien-carsique-sonarsource merged commit 7f1cf5f into master May 18, 2026
25 checks passed
@julien-carsique-sonarsource julien-carsique-sonarsource deleted the feat/jcarsique/BUILD-11294-metrics branch May 18, 2026 16:52
julien-carsique-sonarsource added a commit that referenced this pull request May 19, 2026
… step

Drop the dedicated `metrics-flag` shell step and check `env.CI_METRICS_ENABLED == 'true'` directly in the
`cache-metrics-prep` step's `if:` expression. This:

* Removes the bash 3.2 incompatibility that crashed macOS runners with `bad substitution` on `${RAW,,}`
  (observed in SonarSource/sonar-dummy#592 Build macOS, run 26091741374), which also cascaded into a
  template-validity error for downstream `build-maven` `if:` expressions.
* Removes the case-insensitive variant matching. Accepting only the literal `'true'` matches the convention
  of the existing env vars in this action (`CACHE_BACKEND`, `CACHE_IMPORT_GITHUB`) and avoids the GHA-
  expression equivalent of bash 4-only `${VAR,,}` (no `tolower` in GHA expressions).
* Eliminates the bash step entirely, so there's no portability surface to worry about across runner
  platforms.

The `cache-metrics-prep` step's combined `if: runner.os == 'Linux' && env.CI_METRICS_ENABLED == 'true'`
keeps the Linux-only scope. The downstream `cache-metrics` step continues to chain on
`steps.cache-metrics-prep.outputs.prepared == 'true'`, so when the flag is unset or non-`true` the entire
metrics flow is skipped without warning, matching the pre-#62 behaviour for non-pilot consumers.

README updated to document the strict `true` requirement instead of "case-insensitive".

Surfaced during the BUILD-11297 pilot rollout on sonar-dummy.
julien-carsique-sonarsource added a commit that referenced this pull request May 19, 2026
… step

Drop the dedicated `metrics-flag` shell step and check `env.CI_METRICS_ENABLED == 'true'` directly in the
`cache-metrics-prep` step's `if:` expression. This:

* Removes the bash 3.2 incompatibility that crashed macOS runners with `bad substitution` on `${RAW,,}`
  (observed in SonarSource/sonar-dummy#592 Build macOS, run 26091741374), which also cascaded into a
  template-validity error for downstream `build-maven` `if:` expressions.
* Removes the case-insensitive variant matching. Accepting only the literal `'true'` matches the convention
  of the existing env vars in this action (`CACHE_BACKEND`, `CACHE_IMPORT_GITHUB`) and avoids the GHA-
  expression equivalent of bash 4-only `${VAR,,}` (no `tolower` in GHA expressions).
* Eliminates the bash step entirely, so there's no portability surface to worry about across runner
  platforms.

The `cache-metrics-prep` step's combined `if: runner.os == 'Linux' && env.CI_METRICS_ENABLED == 'true'`
keeps the Linux-only scope. The downstream `cache-metrics` step continues to chain on
`steps.cache-metrics-prep.outputs.prepared == 'true'`, so when the flag is unset or non-`true` the entire
metrics flow is skipped without warning, matching the pre-#62 behaviour for non-pilot consumers.

README updated to document the strict `true` requirement instead of "case-insensitive".

Surfaced during the BUILD-11297 pilot rollout on sonar-dummy.
julien-carsique-sonarsource added a commit that referenced this pull request May 19, 2026
… step

Drop the dedicated `metrics-flag` shell step and check `env.CI_METRICS_ENABLED == 'true'` directly in the
`cache-metrics-prep` step's `if:` expression. This:

* Removes the bash 3.2 incompatibility that crashed macOS runners with `bad substitution` on `${RAW,,}`
  (observed in SonarSource/sonar-dummy#592 Build macOS, run 26091741374), which also cascaded into a
  template-validity error for downstream `build-maven` `if:` expressions.
* Removes the case-insensitive variant matching. Accepting only the literal `'true'` matches the convention
  of the existing env vars in this action (`CACHE_BACKEND`, `CACHE_IMPORT_GITHUB`) and avoids the GHA-
  expression equivalent of bash 4-only `${VAR,,}` (no `tolower` in GHA expressions).
* Eliminates the bash step entirely, so there's no portability surface to worry about across runner
  platforms.

The `cache-metrics-prep` step's combined `if: runner.os == 'Linux' && env.CI_METRICS_ENABLED == 'true'`
keeps the Linux-only scope. The downstream `cache-metrics` step continues to chain on
`steps.cache-metrics-prep.outputs.prepared == 'true'`, so when the flag is unset or non-`true` the entire
metrics flow is skipped without warning, matching the pre-#62 behaviour for non-pilot consumers.

README updated to document the strict `true` requirement instead of "case-insensitive".

Surfaced during the BUILD-11297 pilot rollout on sonar-dummy.
julien-carsique-sonarsource added a commit that referenced this pull request May 19, 2026
… step

Drop the dedicated `metrics-flag` shell step and check `env.CI_METRICS_ENABLED == 'true'` directly in the
`cache-metrics-prep` step's `if:` expression. This:

* Removes the bash 3.2 incompatibility that crashed macOS runners with `bad substitution` on `${RAW,,}`
  (observed in SonarSource/sonar-dummy#592 Build macOS, run 26091741374), which also cascaded into a
  template-validity error for downstream `build-maven` `if:` expressions.
* Removes the case-insensitive variant matching. Accepting only the literal `'true'` matches the convention
  of the existing env vars in this action (`CACHE_BACKEND`, `CACHE_IMPORT_GITHUB`) and avoids the GHA-
  expression equivalent of bash 4-only `${VAR,,}` (no `tolower` in GHA expressions).
* Eliminates the bash step entirely, so there's no portability surface to worry about across runner
  platforms.

The `cache-metrics-prep` step's combined `if: runner.os == 'Linux' && env.CI_METRICS_ENABLED == 'true'`
keeps the Linux-only scope. The downstream `cache-metrics` step continues to chain on
`steps.cache-metrics-prep.outputs.prepared == 'true'`, so when the flag is unset or non-`true` the entire
metrics flow is skipped without warning, matching the pre-#62 behaviour for non-pilot consumers.

README updated to document the strict `true` requirement instead of "case-insensitive".

Surfaced during the BUILD-11297 pilot rollout on sonar-dummy.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants