Skip to content

v1.4.0 — CI gates, SARIF, strictness presets & a reproducible benchmark

Choose a tag to compare

View all tags
@hyhmrright hyhmrright released this 18 Jun 16:10
v1.4.0
ec44ec8

brooks-lint v1.4.0 turns the six-skill review engine into something you can wire into CI with hard gates, tune for legacy codebases, and — for the first time — measure. Everything below is on main and live in the marketplace.

🚦 CI quality gates + SARIF

  • SARIF output for GitHub Code Scanning. ci-review.mjs gains --format sarif / --sarif-out, and the GitHub Action a sarif-file input — findings now surface inline on the PR's Files changed tab. (New report-parse.mjs + sarif.mjs.)
  • Severity & regression gates. On top of fail-below, the Action adds fail-on: critical | warning (fail on any finding at/above a severity) and fail-on-regression: true (fail only when the Health Score drops vs the last run). Backed by a unit-tested ci-gate.mjs.
- uses: hyhmrright/brooks-lint/.github/actions/brooks-lint@v1.4.0
  with:
    mode: review
    anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }}
    fail-on: critical
    fail-on-regression: true
    sarif-file: brooks-lint.sarif   # needs `security-events: write`

🎚️ Strictness presets

A new strictness config key tunes Health-Score deductions for teams at different maturity stages:

Preset 🔴 Critical 🟡 Warning 🟢 Suggestion
strict −20 −8 −2
balanced (default) −15 −5 −1
legacy-friendly −8 −3 −1

legacy-friendly also leads the Summary with the three highest-leverage fixes — so a legacy codebase's first run isn't a demoralizing wall of Criticals.

📊 Reproducible benchmark (new)

The report parser used to be covered only by hand-written fixtures. v1.4.0 adds a deterministic benchmark against 30 real, model-generated reports (evals/benchmark-corpus.json — all six modes, 9 false-positive / tradeoff cases), each independently graded and hand-checked. Run npm run benchmark:

Metric (n = 30, frozen corpus) Result
Exact severity-count match (parser vs. graded truth) 30 / 30
Risk-code precision / recall 100% / 100% (56 finding-level codes, 0 FP / 0 FN)
Valid SARIF 2.1.0 emitted 30 / 30

Given a fixed 2 Critical / 3 Warning / 1 Suggestion finding set, the strictness presets produce exactly the scores their table predicts: 34 / 54 / 74. npm test guards all of this as a regression.

Scope (honest): the parser numbers are deterministic and exactly reproducible by anyone. Whether the model finds the right risks on real code is a separate question, measured by the 57-scenario suite (npm run evals:live). The parser benchmark measures report-parsing fidelity, not whether a given finding is "correct."

🧪 Eval suite 49 → 57

Full Sweep gets its first benchmark scenarios; Architecture Audit and Tech Debt are deepened; every new mode group includes a false-positive / tradeoff guard.

🌍 Platforms (bundled, previously unreleased)

  • One-command scripts/install.sh <platform> for OpenCode, Cursor, Windsurf, Antigravity, pi, Kiro, GitHub Copilot, Factory Droid, Gemini, Codex, and Claude — flat layout so _shared/ always resolves.
  • Per-platform setup guides under docs/.
  • Fixed: the Gemini/Codex manual install nested skills too deep to be discovered — now a flat install. (#14)

✅ Quality

97 unit tests · 57 eval scenarios · 30-report parser-fidelity benchmark — all green.

Upgrade: /plugin marketplace update → /plugin install brooks-lint@brooks-lint-marketplace, or scripts/install.sh <platform>.
Full changelog: https://github.com/hyhmrright/brooks-lint/blob/v1.4.0/CHANGELOG.md

Assets 2
Loading