Skip to content

v0.5.0 — M4 hardening + M5 anti-gaming scoring

Choose a tag to compare

View all tags
@matedev01 matedev01 released this 09 Jul 20:20
· 384 commits to main since this release
v0.5.0
87a6f42
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

402 commits since v0.3.0. Two milestones landed:

M4 — Hardening & release readiness

  • Agent hardening: every field the LLM emits is guarded against non-string types (#297, #313, #317).
  • Benchmark scoring: module-recall farming fixed (#289), backlog threshold reachable for single-word titles (#308), composite-score wiring (#341).
  • Leakage lockout: tag-creation-date filter for frozen releases (#332), release-tag scrubbing (#330), forward-reference masking in the git-only fallback (#312).
  • Tooling: compare_eval CLI for diffing replay artifacts (#306), --fail-under score floor for CI gating (#318, #367).
  • M3 acceptance run completed clean: generalization_gap = 0.097, zero crashes across 5 repos.

M5 — Measured, anti-gaming contribution scoring

  • scripts/score_pr_delta.py: diffs two run_eval artifacts (baseline vs. a PR's agent) and applies a Pareto floor — a PR's value label now requires a measured, non-regressing benchmark delta, not a maintainer's read of the diff (#1295).
  • Merge-block + ceiling label: a measured regression is a hard merge block for agent/ PRs; a large, clean win on every axis unlocks a new ceiling label, mult:breakthrough (×3.0), above mult:core-correctness (×2.0) (#1302).
  • REVIEW.md documents the full evidence requirement and tier ladder for agent/ PRs.
  • Public CI smoke check (agent-benchmark-smoke.yml) on every agent/-touching PR — crash/shape only, explicitly not the scoring evidence itself.

Full test suite green (3675 passed). See ROADMAP.md for the complete milestone breakdown.

Assets 2
Loading