Skip to content

Metis 1.2.0

Choose a tag to compare

@gitRasheed gitRasheed released this 19 Jul 15:31

P15: the phase-scoped skill text

Winner of a 16-permutation eval campaign (~90 hand-scored runs, five cases, preregistered ship rules). The skill diff is +15/−5 lines on the previous text:

  • New implementation-rules section: behavior-test-first with an anti-assert-spam clause, fix-causes-not-sites, optimize-for-the-next-change, no quality decay on long task lists, campsite rule.
  • Review lens 3 now prescribes structural remedies (config object / responsibility split / seam), not cosmetic fixes.
  • Review lens 1 distinguishes untrusted-data validation from internal-invariant assertions: invariant gaps are substance, duplicate validation is noise.
  • Small diffs get one combined review pass; a lens may legitimately produce no findings.
  • Agent-process rule: search the codebase for an existing helper, type, or constant before writing a new one (owner addition after the campaign gates; A/B queued for next round).

Measured (vs v1.1.0 text, gpt-5.6-sol): 7/10 blinded pairwise code-quality wins (mean rating 8.1 vs 7.8), review parity at n=5/n=3, perfect hidden-gate record, favorable direction on an untuned holdout case. Core finding of the campaign: skill text taxes whichever phase holds it — guidance scoped into the phase that needs it keeps the benefit and drops the tax.

Full report, per-finding score derivations, and the permutation texts: evals/cases-v5/.