Skip to content

v1.16.1

Choose a tag to compare

@voidmatcha voidmatcha released this 12 Sep 12:35
· 64 commits to main since this release
v1.16.1
43520af

Changed

  • Bundle version bumped to 1.16.1 across all four SKILL.md files and all three plugin manifests. This deliberately invalidates the sha256_at_preparation digests that benchmarks/healer-perturbation-v1/protocol.json and benchmarks/subagent-routing-v1/protocol.json had pinned against three of the four SKILL.md files (playwright-test-generator, e2e-reviewer, playwright-debugger) — both protocols are still NOT_RUN and both now record the invalidation and the re-preparation requirement in their own protocol.json and README.md. Shipping this release was prioritized over keeping those still-unexecuted protocols' preparation-time digests valid; neither protocol's design, arms, or decision rule changed.
  • docs/llm-generated-e2e-test-evidence.md corrected three overclaimed external-evidence rows. NAVER's "explain the harness" framing was not actually adopted anywhere in this repo and the claim was removed; Slack's adapter-comparison claim was flagged as unmeasured and cross-referenced to benchmarks/exploration-fallback-v1/; Shopify's writer/verifier framing was reworded from "borrowed idea" to "converges with," since this repo's design predated reading that article. A Toss FE platform team talk and a flex.team backend post were independently evaluated and recorded as not adopted, each with a specific reason (native-app-platform infrastructure the talk assumes this repo doesn't have; a JVM backend stack out of this repo's Playwright/Cypress scope, whose underlying principle is already covered by reviewer patterns #10b/#19/#20).

Fixed

  • The hosted HOL Plugin Scanner false-positived on six long-standing files. .plugin-scanner.toml gained dated ignore_paths entries (scan.sh, three SKILL.md files, preflight_target.py) explaining each false-positive class, and scripts/ci/test-security-gates.py's HOL_SCANNER_FALSE_POSITIVE_PATHS set was kept in sync with it — the two had silently drifted apart.
  • scripts/ci/test-parity.sh's CHANGELOG mutation test broke when this changelog was reflowed to one line per paragraph. The mutate-string literal still assumed the pre-reflow hard-wrapped bytes (with eighteen tokens\n of headroom), so the check silently stopped exercising the real file. Restored to a single-line literal matching this file's current format.
  • scripts/ci/lib/scan-security-policy.py's hardcoded-home rule missed two real leak shapes. It only matched a literal /Users/ or /home/ substring, so a dash-encoded session-scratchpad path and an embedded ls -la tool-output owner column both evaded it — exactly how a real account-name leak reached committed pilot evidence in this release before being caught and redacted. Added a dash/underscore-bound Users pattern and an ls -l-shaped owner-column pattern; deliberately excluded a bare lowercase home pattern after a first attempt produced ~524 false positives from ordinary compound identifiers.

Added

  • Roadmap item 1's real 36-cell pilot ran to completion: ours_only vs planner_plus_ours, result CONDITIONAL. benchmarks/ours-vs-planner-pilot-v1/ records the full run (46 total top-level sessions, 17.3 serial hours, $276.38 nominal cost) plus a human adjudication (adjudication.md) of the mechanically computed preliminary label, required because target A showed no majority-stable complete-rate lift. The planner's absorbed deltas on target A were real and correctly load-bearing within their own sessions, but ours_only's own repetitions reached at least as good a majority-stable outcome on both measured scenarios without a planner step (tied on A-S1, and — see pilot-results.json's known_scoring_corrections — actually ahead on A-S2), so the "independently verified new failure condition" escape clause does not apply either. No product text changes: the Planner step remains optional, not default. Two infrastructure incidents (keychain OAuth token issues mid-run) are documented in pilot-results.json's auth_events: one used the protocol's pre-registered infrastructure-only retry allowance (cap 1), and one exceeded that cap and was recovered via an explicit operator-approved exception, also documented in the same record.
  • Four roadmap items (3, 4, 5, 6) gained preregistered protocol designs; none is authorized to run. benchmarks/healer-perturbation-v1/ defines the Healer-vs-honesty-control comparison (5 perturbations, 2 arms) and wires its contract tests into ci-local.sh. benchmarks/subagent-routing-v1/ defines the named-subagent-vs-inline-fallback comparison for e2e-finding-verifier/e2e-failure-classifier. benchmarks/stryker-feasibility-v1/ gained a full protocol.json (previously README-only) for the already-decided REFERENCE_ONLY-by-default Stryker adjunct question; it pins no skills/*/SKILL.md file, so it never competes for the same frozen surface as items 3/6. benchmarks/exploration-fallback-v1/ records two deferred candidate exploration arms (recorded-trace, adapter-usage logging) with concrete re-trigger conditions instead of a time-based reconsideration schedule.