Skip to content

Metis 1.1.0

Choose a tag to compare

View all tags
@gitRasheed gitRasheed released this 04 Jul 19:34
v1.1.0
1b0dbdd

Metis 1.1.0

A minor release, but the largest since 1.0: the skill was restructured, extended with a code review mode, and — for the first time — every change was validated against measured evals.

New

  • Code review mode: four sequential lens passes (correctness/contracts, data/state, control flow/API shape, tests/slop), verify-before-reporting, a fixed file:line + severity + one-sentence finding format, lens attribution tags, and sub-agent fan-out gated to large diffs (>~400 lines / 8 files, max 4 agents clustered by lens).
  • references/review-examples.md: compact review-time contrasts (replay safety, comment discipline, cleanup), loaded by default in review mode — chosen over the full examples file by A/B measurement.
  • New design principles (eval- and source-backed): design for the hardest real requirement first; classify a whole batch before applying any element; pair assertions where data enters and leaves.
  • New agent-process rules: distinguish essential from accidental complexity (don't replicate tech debt); detect thrash and re-derive from the spec after repeated failed fixes.
  • Ground-truth eval suite (evals/cases-v4/): two PR-review cases with seeded defects and objective recall/precision scoring, including a cross-domain holdout.

Changed

  • SKILL.md cut from 294 to 175 lines (~40% less always-loaded context) via phase gating and moving inline example code to cue lists pointing at references/examples.md — all content preserved in references.
  • Sharper phrasing restored from source material: wishful-thinking call-site design, boundaries around what systems do rather than what entities are, plain flat records for the simple case, "tests are code: minimize test logic".
  • README now publishes measured results.

Measured (Codex gpt-5.5 xhigh, 60+ isolated runs)

  • +53–60% defect value vs baseline on seeded-ground-truth review cases, winning or tying every rep — including a held-out case in an unseen domain.
  • Zero false positives in 39 consecutive review runs.
  • Five real, unplanted bugs found by Metis conditions across the review cases; baseline found one.
  • Changes that failed their A/B (a mandatory per-lens ledger) were rejected rather than shipped.

Full methodology and per-run scoring: evals/.

Assets 2
Loading