Skip to content

v0.21.0 — Sentinel: eval harness, telemetry, delta-scoped re-review

Choose a tag to compare

View all tags
@pedrofuentes pedrofuentes released this 03 Jul 01:52
· 17 commits to main since this release
v0.21.0
69d145f
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Sentinel quality/reliability/speed release. Ruleset stays v1 — the report-protocol contract (Status: line, Required action mapping, verdict semantics) is unchanged; the Execution Log gains an additive Duration/Tokens column, the header an additive Elapsed: line, and Follow-ups now emits only the verdict-matching action line.

Added

  • evals/ behavioral regression harness — 10 synthetic PR fixtures with expected verdicts, a two-lane runner, and a dimension-coverage matrix. Structural CI checks the docs' shape; evals check the ruleset's behavior. Baseline: 10/10 PASS pre-edit; post-edit re-run confirmed the re-review fixtures still mandate REJECTED.
  • Execution-Log telemetry — per-dimension Duration/Tokens + report Elapsed: line, verbatim-from-platform only (estimated numbers count as fabricated evidence; N/A is always compliant). Non-normative; makes #11/#14's revisit criteria measurable.
  • Delta-scoped Phase 1 on re-review (fail-closed): requires Sentinel-authenticated prior report at its Phase 5 persisted location + git merge-base --is-ancestor-verified base with a self-recomputed delta. Test/fixture/mock edits re-open checks 1–4 for all affected code; CI/test/build-config deltas and unverifiable conditions → full Phase 1. Applies only when the prior cycle's Phase 1 was green.
  • Known-issues fetch guidance — exact gh issue list --json/--jq startswith("sentinel:") command (--label does not wildcard-match); fetched issue text fenced as untrusted, Known-marking only.
  • .github/PULL_REQUEST_TEMPLATE.md; scripts/validate.sh grew from 8 to 10 checks + a non-blocking evals reminder.

Changed

  • Report Follow-ups & Actions: single verdict-matching line, retaining the sentinel:important/sentinel:minor label literals (closes #9).
  • Phase 3 / rubric Known-matching tightened to "same specific defect mechanism + fix".
  • Re-review Phase 2 re-dispatch explicitly includes dimensions whose only findings were Known.
  • SEVERITY-RUBRIC.md: stale template-semver pin dropped (ruleset-v1 binding stays, now CI-checked).

SENTINEL.md budget: 177/178 non-blank lines.

Assets 2
Loading