Skip to content

ci: Coverage Floor — opt-in pre_measured_coverage_artifact#73

Merged
topcoder1 merged 1 commit into
mainfrom
ci/coverage-floor-pre-measured-artifact
May 24, 2026
Merged

ci: Coverage Floor — opt-in pre_measured_coverage_artifact#73
topcoder1 merged 1 commit into
mainfrom
ci/coverage-floor-pre-measured-artifact

Conversation

@topcoder1
Copy link
Copy Markdown
Owner

Summary

Adds a new optional input pre_measured_coverage_artifact to the Coverage Floor reusable. When set, the reusable downloads a coverage-percent.txt from the named artifact, parses the percentage, and skips the install + test step entirely — runs only the floor comparison + seed + sticky-comment logic.

Why

GitHub Actions hit 90% of the May 2026 budget on the topcoder1 account on 2026-05-23. The fleet audit traced the top burn to Coverage Floor + CI running the same pytest --cov invocation twice across ~20 repos. webcrawl alone: ~290 min/wk on Coverage Floor + ~287 min/wk on CI's test job, mostly the same work.

This change is the architectural fix. ~50% reduction in Coverage Floor minutes for any repo that opts in.

Design doc

Full architecture write-up + alternatives considered: proposals/2026-05-23-coverage-floor-dedup-design.md in product-center (not yet committed there — local-only for now, but the relevant excerpt:

  • Caller pattern shifts from two separate workflows (ci.yml + coverage-floor.yml) to one workflow with a needs: dependency
  • Chose same-workflow + needs over cross-workflow workflow_run because PR context (head SHA, PR number, base ref) is preserved natively
  • Artifact format is a simple text file with the coverage percent number — language-agnostic, caller's CI computes it however

Backwards compatibility

100%. Existing ~20 fleet callers don't pass the new input → identical behavior. Self-test of this reusable (selftest/sample.py @ 99% coverage) also unaffected. Defaults preserved everywhere.

What changes in this PR

  • New input pre_measured_coverage_artifact: string (default empty)
  • New step Download pre-measured coverage artifact gated if: input != ''
  • All language setup steps (setup-python/go/node, install uv, git cross-org config) gated to skip when input is set
  • Existing Measure coverage step renamed to id measure_fresh, gated if: input == ''
  • New step Read coverage from pre-measured artifact id measure_cached, gated if: input != ''
  • All downstream consumers updated to ${{ steps.measure_fresh.outputs.measured || steps.measure_cached.outputs.measured }} — GHA's || returns the first truthy value, so only the running step's output is consulted

Diff: +71/-11 lines in .github/workflows/coverage-floor.yml, nothing else touched.

Follow-up PRs

  • PR 3 (topcoder1/webcrawl): wires up the opt-in as the proof point. Will be filed in parallel; its first CI run will fail until this PR merges (the new input doesn't exist on @main yet).
  • Fleet rollout (post-validate, ~1 week): port to remaining ~12 repos via scripts/install-coverage-floor-dedup.sh.

Side note (out of scope)

The services_postgres and services_redis declarations remain at job level (GHA doesn't support conditional services). When pre_measured is set, these still spin up but unused — ~5-10 sec overhead per run. Acceptable; could refactor later by splitting into two jobs gated on input.

Auto-merge rationale

Change to a shared reusable workflow that ~20 callers reference. Touches .github/workflows/** — high-risk per claude-author-automerge.yml policy → manual click-merge required. actionlint clean. Backwards-compatible by design.

Test plan

  • actionlint passes on this branch (verified locally)
  • Self-test of the reusable still passes (the pull_request trigger runs against selftest/sample.py @ 99% coverage; new if: gates evaluate to "run the existing path" because input is empty)
  • After merge: PR 3 in webcrawl validates the new path end-to-end
  • After 1 week of webcrawl runs: weekly Coverage Floor minute burn drops from ~290 → ~30 (≈90% reduction on that specific check; ~$9/mo on webcrawl alone)

🤖 Generated with Claude Code

Adds a new optional input `pre_measured_coverage_artifact` to the
Coverage Floor reusable. When set, the reusable:

- Downloads an artifact of that name from the same workflow_run
- Reads `coverage-percent.txt` (single-line numeric) from the artifact
- Skips the language-specific install + test invocation entirely
- Runs only the floor comparison + seed + sticky-comment logic

Use case: repos that already run pytest --cov (or go cover / vitest)
in their own CI workflow can have Coverage Floor read the result
instead of re-running the whole test suite. Saves ~50% of Coverage
Floor's per-run minutes for these repos.

Background: GitHub Actions hit 90% of the May 2026 budget on the
topcoder1 account. Audit traced the top burners to Coverage Floor +
CI running the same pytest invocation twice across ~20 fleet repos
(webcrawl alone: ~290 min/wk Coverage Floor + ~287 min/wk CI test
job, mostly the same work). This change is the architectural fix.

Implementation:
- New input `pre_measured_coverage_artifact: string`, default empty
  (existing behavior preserved)
- New step `Download pre-measured coverage artifact` gated on the
  input being non-empty
- All language setup steps (setup-python/go/node, install uv, git
  cross-org config) gated to skip when the input is set
- Existing `Measure coverage` step renamed to `Measure coverage` /
  id `measure_fresh`, gated `if: input == ''`
- New step `Read coverage from pre-measured artifact` /
  id `measure_cached`, gated `if: input != ''`
- All downstream consumers updated to read
  `steps.measure_fresh.outputs.measured || steps.measure_cached.outputs.measured`
  (GHA's `||` returns the first truthy value, so only the running
  step's output is consulted)

Backwards compatibility: 100%. Existing ~20 fleet callers don't pass
the new input, get identical behavior. The self-test of this
reusable (selftest/sample.py at 99% coverage) also unaffected.

Caller migration plan: opt-in per repo. First proof point is
topcoder1/webcrawl (separate PR — see [topcoder1/webcrawl#TBD]).
After 1 week of clean runs there, port to remaining ~12 repos via
a rollout script in scripts/install-coverage-floor-dedup.sh.

Side note: services_postgres and services_redis declarations remain
at the job level (GHA doesn't support conditional services). When
pre_measured is set, these still spin up but unused — ~5-10 sec
overhead per run. Acceptable for now; could be refactored later
by splitting into two jobs gated on input.

Auto-merge rationale: change to a shared reusable workflow that ~20
callers reference. Touches .github/workflows/** which is high-risk
per claude-author-automerge.yml policy — manual click-merge required.
actionlint passes. Backwards-compatible by design.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown

Coverage Floor — mode: enforce

metric value
measured 100.0%
floor (current) 99.0%
target 100.0%
last bumped 2026-05-12

@claude
Copy link
Copy Markdown

claude Bot commented May 24, 2026

No issues found. Logic is sound: skipped-step outputs are correctly falsy for the || coalescing, artifact path and shell working-directory are consistent, input validation guards the artifact content before use, and all language-setup gates are paired correctly with measure_fresh.

@topcoder1 topcoder1 merged commit 607276a into main May 24, 2026
11 checks passed
@topcoder1 topcoder1 deleted the ci/coverage-floor-pre-measured-artifact branch May 24, 2026 03:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant