Repository navigation
v1.19.0
Highlights
- All four skills load less up front. Every
SKILL.mdbody now carries the triggers, gates, safety rules and output contract, and moves its step-by-step procedure into sibling files behind pointers that say when to read them.e2e-reviewergoes from about 16.7k tokens to 3.8k,playwright-debuggerfrom 8.9k to 4.6k,cypress-debuggerfrom 9.9k to 5.2k, andplaywright-test-generatorfrom 11.6k to 5.4k. Every sentence of every old body is preserved verbatim in the file that now owns it. - Reviewer holdout v7 has a result, and it is
FAIL. 180 Codexgpt-5.6-solcalls, three prompt arms, zero infrastructure errors. Against the no-skill control the skill passes all six lift checks (stable F1 +0.50). Against a catalog-only control that lists the 24 patterns and nothing else, it does not: +0.017 F1 where the preregistered threshold is +0.05. What the full skill clearly adds over the bare catalog is precision and clean-case specificity (1.00 versus 0.75), not recall. Three earlier attempts are archived and disclosed rather than dropped. - The run found a real bug, and it is fixed.
#3said "never wrap assertions intry/catch", but its confirmation step asked whether an "independent meaningful verification" survived, so a later assertion on a different element excused a swallowed one. The reviewer missed that case in every repetition, with the full skill and with the catalog alone. A real assertion swallowed by a non-rethrowing catch is now P0 unless the identical check is re-asserted unswallowed, control flow fails the test regardless, or documented intent applies. - The reviewer now sees a
try/catchthat swallows a gate. A controlled ablation found the reviewer caught the.catch(() => {})swallow every time but thetry/catchform at most once in six, although the contract already forbade it. The scanner raises it as a triage candidate, pairing each catch with its own try on lexed source. - An apostrophe in JSX text no longer breaks that pairing.
<p>Don't panic</p>used to open a string in the lexer and leave a realtry/catchunpaired. Quotes are now inert between an opening and closing tag on the same line, which leaves TypeScript generics and comparisons untouched. - Far fewer false credential candidates.
#14used to raise every.fill()or.type()with a string literal, so typing a todo title counted. Candidates now need a credential-shaped field or value: on the seven-repository field scan, 94 candidates drop to 25. - The generator's command approval table says what each command does. An
Effectcolumn labels every row, so atest:e2ewhose config resets a database no longer looks like one that only runs tests. - Clearer generator contracts from live runs. When an exploration observation is sufficient without another probe, how Step 5b is ordered on a run with no tracer scenario, and what makes a real-write server "disposable" — the approved run starts and stops it, and loopback is a location, not a lifecycle.
See CHANGELOG.md for details.
What the v7 result does and does not support
It is one host, one model, and a public development corpus that anyone can read, so it is not sealed, release-grade, cross-provider, or evidence of generalization. It supports the recorded FAIL and the descriptive per-arm numbers, nothing more. No skill-lift claim is made.
An uncounted addendum reruns the same protocol on the trimmed reviewer with the #3 fix. Its full arm passes every absolute threshold, which shows the trim did not regress the tool-less holdout, and the lift gap against catalog-only is unchanged. That rerun is contaminated for the one case the fix was written against, and it is published as an addendum rather than as a result.
Verification
/bin/bash -p scripts/ci/ci-local.sh/bin/bash -p scripts/ci/pre-push-security.shscripts/dev/linux-script-check.sh: shipped-script suites pass on Ubuntu 24.04- Codex smoke: all five checks pass against these skills, exercising the trimmed bodies end to end — the reviewer reaches
#4f, both debuggers classify F2 through their bundled bounded readers, and the generator quotes its framed-stdin preflight command - Reviewer holdout v7: 180 calls, 0 infrastructure errors, comparator
FAIL - Live reviewer A/B on the trimmed skill: matches or beats the untrimmed one, and opens more of its own references, because the untrimmed body let required pointers be skipped
- Pre- and post-implementation adversarial reviews for each batch, with every must-fix and should-fix item addressed
Not tested
- Live debugger and generator runs reading the new references on demand; only the reviewer was A/B'd with tools, and the smoke proves the pointers are followed, not accuracy
- An uncontaminated rerun on a corpus the
#3fix never saw