Skip to content

ci.yml's Control-plane coverage step swallows failures with an unjustified || true, unlike REES's documented one #8392

Description

@JSONbored

Context

.github/workflows/ci.yml's "Control-plane coverage" step (in the control-plane-scoped section of
the validate job) runs:

- name: Control-plane coverage
  if: ${{ github.event_name == 'push' || needs.changes.outputs.controlPlane == 'true' }}
  run: npm run control-plane:coverage || true

The || true unconditionally swallows a non-zero exit from npm run control-plane:coverage
(scripts/control-plane-coverage.mjs, which re-runs control-plane/test/**/*.test.ts under c8 and
writes control-plane/coverage/lcov.info). The next step, "Verify control-plane coverage report
exists", only checks the lcov file is non-empty — it never checks the coverage run itself exited
cleanly.

The identical || true shape exists on the "REES coverage" step immediately above it in the same file,
but that one carries an explicit justifying comment explaining why a non-zero exit there is expected
and safe: c8's instrumentation inflates review-enrichment's timing-sensitive linear-time/ReDoS-guard
assertions past their (uninstrumented) budgets, so a non-zero child exit is a known false alarm — the
real pass/fail already happened in the uninstrumented npm --prefix review-enrichment test step just
before it.

The "Control-plane coverage" step has no such comment, and it needs none: a repo-wide search
(grep -rln "budget\|timing-sensitive\|instrumentation\|c8" control-plane/) confirms control-plane's
suite (14 files under control-plane/test/**) has no timing-budget or ReDoS-guard assertions that
would explain an instrumentation-induced false failure. This reads as the REES step's || true having
been copied over without its justification, not a verified decision for control-plane. As written,
if the c8-instrumented re-run crashes or exits non-zero for a real reason (a genuine bug or flake
under instrumentation, a c8 misconfiguration) after still managing to write a non-empty but
incomplete/wrong lcov.info, CI stays green and silently uploads bad coverage data to Codecov under
the control-plane flag — exactly what "Verify control-plane coverage report exists" is meant to catch
but, as written, cannot.

npm run control-plane:test (the step immediately before this one, uninstrumented) remains the real
correctness gate for the suite itself — this issue is only about the coverage-harvest step's own
unjustified error-swallowing, not about control-plane's test correctness.

Requirements

  • Remove || true from the "Control-plane coverage" step in .github/workflows/ci.yml, so a genuine
    coverage-harvest failure fails the validate job instead of silently passing through.
  • Do not touch the "REES coverage" step's own || true (that one is justified and documented — leave
    it as-is) or scripts/control-plane-coverage.mjs itself.
  • Confirm npm run control-plane:coverage currently exits 0 against a clean main checkout (run it
    locally from the repo root) before removing the swallow, so this change doesn't itself immediately
    break CI on an unrelated pre-existing issue. If it does NOT currently exit 0, stop and report that in
    the PR instead of force-fixing an unrelated failure — this issue is scoped to the swallow itself.

Deliverables

  • .github/workflows/ci.yml's "Control-plane coverage" step has || true removed.

Test Coverage Requirements

This is a .github/workflows/** change; neither .github/** nor scripts/** is in Codecov's
coverage.include (codecov.yml), so there is no codecov/patch obligation. Verification is: the
workflow file still passes npm run actionlint, and a local run of npm run control-plane:coverage
from the repo root exits 0 and produces a valid control-plane/coverage/lcov.info (confirming the
swallow was safe to remove).

Expected Outcome

The "Control-plane coverage" CI step fails the build on a genuine coverage-harvest failure instead of
silently swallowing it, closing the gap where broken coverage data could upload to Codecov undetected.

Links & Resources

  • .github/workflows/ci.yml — "REES coverage" step (the justified || true to leave untouched, for
    contrast) and "Control-plane coverage" step (the one to fix), both in the same job.
  • scripts/control-plane-coverage.mjs (what the step runs; no change expected here).
  • Issue Wire control-plane/ into CI, mirroring the existing rees job pattern #7743 (closed) — original wiring of control-plane into CI; this is a follow-up correctness gap
    in that wiring, not a duplicate.

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:bugGittensor-scored bug fix — scores a 0.05x multiplier.help wantedExtra attention is needed

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions