Skip to content

feat(coverage-gate): systematic new-code coverage tool [mache-66d8df]#390

Merged
jamestexas merged 1 commit into
feat/evolve-coverage-trunkfrom
evolve/coverage-gate-tool
May 19, 2026
Merged

feat(coverage-gate): systematic new-code coverage tool [mache-66d8df]#390
jamestexas merged 1 commit into
feat/evolve-coverage-trunkfrom
evolve/coverage-gate-tool

Conversation

@jamestexas
Copy link
Copy Markdown
Contributor

Summary

A diff-aware NEW-CODE coverage gate (tools/coverage-gate/) that enforces
"every new prod line is covered" per PR. Pure stdlib Go, no new module deps.
Lives alongside the existing one-off tools in tools/.

This PR ships the tool only. CI wiring (calling it from task check
or a GHA job) is a deliberate follow-up so the contract can land and be
exercised locally first.

Contract

coverage-gate <cover.out> <diff.patch>
  • cover.out — produced by go test -coverprofile=cover.out ./...
  • diff.patchgit diff base..HEAD > diff.patch or gh pr diff <num> > diff.patch

Exit codes:

  • 0 — every new prod line is either covered, not a statement (decl, blank,
    comment), or annotated // coverage:ignore
  • 1 — at least one new prod line is uncovered; per-file report on stdout
  • 2 — usage / I/O / parse error

What counts as "new prod line":

  • +-prefixed line in a unified-diff hunk
  • file is .go, NOT _test.go, NOT under /testdata/
  • line is not blank
  • line does not start with // or /* (after trim)
  • line does not contain // coverage:ignore suffix

Output format (failure)

NEW PROD LINES NOT COVERED:

internal/leyline/socket.go
  L190-191
  L256-257

4 new prod line(s) uncovered. Either add tests or annotate the line with `// coverage:ignore`.

Contiguous uncovered lines collapse into Lstart-end blocks. Files sort
alphabetically. Total count at the bottom.

Per-PR verification

PR #389 (DiscoverOrStart rewrite)

$ go run ./tools/coverage-gate /tmp/pr389.cov /tmp/pr389.diff
NEW PROD LINES NOT COVERED:

internal/leyline/socket.go
  L190-191
  L256-257

4 new prod line(s) uncovered.   exit 1

L190-191 is the stale-managed-daemon reap branch; L256-257 is the
last-chance pre-spawn liveness check + leftover-socket cleanup. Both are
real new prod lines without test coverage — exactly the kind of thing
this gate exists to surface.

PR #388 (download-leyline retarget)

$ go run ./tools/coverage-gate /tmp/pr388.cov /tmp/pr388.diff
NEW PROD LINES NOT COVERED:

internal/leyline/socket.go
  L583
  L605

2 new prod line(s) uncovered.   exit 1

Both are inside downloadLeylineassetName := fmt.Sprintf(...) and
the http.StatusNotFound branch. Exercised only via network, hence
no automated coverage. Honest report; humans decide whether to backfill
with an httptest-driven test or annotate // coverage:ignore.

Self-tests

15 tests, all pass:

  • TestParseProfile / TestParseProfileMalformed
  • TestParseDiff / TestParseHunkNewStart
  • TestIgnoreTestFiles / TestIgnoreTestdataAndNonGo / TestIgnoreComments
  • TestIntersect / TestIntersectOverlappingCoveredWins / TestIntersectStableSort
  • TestCollapseBlocks
  • TestNormalizeProfilePaths
  • TestExitCodeZeroWhenCovered / TestExitCodeOneWhenUncovered / TestExitCodeOneReportFormat
    (integration via built binary + os/exec)
ok  github.com/agentic-research/mache/tools/coverage-gate  1.18s

task check clean (fmt + vet + golangci-lint + full test suite +
validate + docs:lint).

Test plan

Files

Single new package, single new directory:

  • tools/coverage-gate/main.go (~340 LOC)
  • tools/coverage-gate/main_test.go (~290 LOC, 15 tests)

A diff-aware NEW-CODE coverage gate written in pure Go (stdlib only,
no new module deps). Intersects a Go cover profile with a unified diff
and reports any newly-added or modified PROD lines (excluding tests,
testdata, blank lines, comment-only lines) that have zero coverage.

Contract:
  coverage-gate <cover.out> <diff.patch>

Inputs:
  - cover.out: produced by `go test -coverprofile=cover.out ./...`
  - diff.patch: unified diff from `git diff base..HEAD` or `gh pr diff`

Output (on failure):
  NEW PROD LINES NOT COVERED:

  internal/leyline/socket.go
    L189-191
    L256-257

  4 new prod line(s) uncovered. Either add tests or annotate the
  line with `// coverage:ignore`.

Exit codes:
  0 — every new prod line is either covered, not a statement, or
      explicitly annotated `// coverage:ignore`
  1 — at least one new prod line is uncovered; per-file report on stdout
  2 — usage / I/O / parse error

Implementation:
  - parseProfile: streams the cover profile (file, range, hits)
  - normalizeProfilePaths: strips the module prefix (read from ./go.mod)
    so profile keys match diff paths (repo-relative)
  - parseDiff: walks unified diff hunks, tracks +newstart, records added
    .go prod lines (filters _test.go, testdata, non-.go, blank, // and /*)
  - intersect: per-file, per-line lookup against cover ranges; "covered
    range wins" semantics for overlapping ranges; lines outside any
    range are silently skipped (not statements)
  - `// coverage:ignore` suffix suppresses a single added line

Self-tests (15, all pass): parse/diff/intersect/overlap/comment-skip/
test-file-skip/testdata-skip/non-go-skip/coverage-ignore/exit-code-zero/
exit-code-one/report-format/normalize-paths/hunk-header-parsing/
stable-sort.

Per-PR verification:
  PR #389 (DiscoverOrStart rewrite):
    NEW PROD LINES NOT COVERED:
    internal/leyline/socket.go
      L190-191
      L256-257
    4 new prod line(s) uncovered.   exit 1 OK

  PR #388 (download-leyline retarget):
    NEW PROD LINES NOT COVERED:
    internal/leyline/socket.go
      L583
      L605
    2 new prod line(s) uncovered.   exit 1 OK
    (these are inside downloadLeyline, exercised only via network)

No edits to Taskfile.yml, .github/workflows/, or other files — this PR
ships ONLY the tool. CI wiring is a deliberate follow-up.
@github-actions
Copy link
Copy Markdown

find_smells (advisory)

Scoped to files changed in this PR. Rules below run on the standalone (no-LLO) cross-ref tables; _ast rules (cyclomatic_complexity, long_function, long_file, magic_int_in_comparison) are not exercised here.

fan_out_skew — 1 finding(s) in changed files
Source Node Metric
tools/coverage-gate/main.go main/functions/parseProfile/source 22

Rules: see docs/ARCHITECTURE.md for the full registry. Advisory only — these are heuristics, not gates.

@jamestexas jamestexas merged commit 97de073 into feat/evolve-coverage-trunk May 19, 2026
1 check passed
@jamestexas jamestexas deleted the evolve/coverage-gate-tool branch May 19, 2026 00:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant