v0.2.0

skill-harness v0.2.0

Screen a skill and the harness now gives you a decision — KEEP, CUT, or CAN'T-TELL-YET — instead of a raw pass rate. When the evidence can't tell whether the skill or the model did the work, it says UNMEASURED rather than guess a number.

(a) "Every task is run twice: with the skill and without it." → today's screens run the no-skill arm first and only escalate to the paired run when the model fails without help; (b) "a handful of ~26" → same epoch/verdict correction as this PR; (c) "an interrupted run needs redoing" for llm-judge-calibration → per the pre-registration the screen completed 3/3; what's pending is store backfill (per-trial assembly across logs from the credit outage), not a rerun.

What's new

• The keep/cut decision. A screen result becomes a plain call: keep the skill, cut it (the model already does the task, or the skill didn't help when measured), or not enough evidence yet. The thresholds are set before the run, not adjusted afterward to fit the result.
• An append-only record of every screen. Results are written once and never edited; pass rates are recomputed from the raw runs, not cached. Invalid runs are kept with a reason and left out of the totals.
• The batch-1 screens now trace to that record instead of prose write-ups. All came back CUT — the model already handled the task. (One, llm-judge-calibration, is still pending; an interrupted run needs redoing.)

Where it stands

• No skill has scored KEEP yet. The code that would award one is written and tested, but it only runs when a skill's no-skill baseline comes back below a perfect score — and so far, every skill screened, the model already passes without it.
• Two skills have run end to end, both CUT: append-only-evidence-design and a hardened git-pull-rebase-trap. The model passed every run without the skill, so there was nothing to add.
• The LLM judge stays off until it's calibrated against human ratings. It isn't yet, so anything relying on it returns UNMEASURED.
• Most screens aren't in the record yet — a handful of ~26 are; the rest are being moved over from write-ups.

The plan behind the v0.1 → v0.2 change was written down first, and it's public: docs/findings/v0.2-reaim-gate.md. This is also the tool behind the evidence in MrBinnacle/skills.