e2e-skills 2.0.1

e2e-skills 2.0.1

Four Agent Skills for Playwright and Cypress E2E work: e2e-reviewer, playwright-test-generator, playwright-debugger, cypress-debugger.

A debugger-only patch. Apart from the version field, the reviewer, generator, Cypress debugger, scanner, installs, pattern IDs, F-codes, and every existing safety rule are unchanged, and no rule is weakened. No user action is required to upgrade.

What changed

• playwright-debugger states a root cause only as far as the artifact evidences it. Anything beyond that is marked as a hypothesis with the missing evidence named. Without the spec source it describes the fix in prose instead of inventing selectors, endpoints, or line numbers.
• The report-reader command is one line with a literal absolute --project-root. The previous PROJECT_ROOT=$(...) form could not be matched by Claude Code's command check, so a non-interactive run spent one or two turns on rejected attempts. The spec source is read only from a user-supplied path or a relative, non-traversing regular file named like a test file, so a report cannot point it at .env or other config files.
• Docs and repository. Per-skill evidence paragraphs now describe the 2.0.1 evidence. Raw benchmark run artifacts (23 folders, 2,152 files, about 51 MB) moved to the voidmatcha/e2e-skills-benchmarks Hugging Face dataset, pinned to one revision in benchmarks/external-evidence.json and restorable byte-identically with scripts/dev/fetch-benchmark-evidence.sh (--dry-run shows the plan). Protocols, results, and runners stay in the repository. Ten reviewed scanner false positives are suppressed by exact path.

Evidence, with its limits

On four real OpenMetadata CI reports (12 runs per arm), unsupported claims per answer fell from 5.4 to 2.9 and answers that said the cause is not in the report rose from 58% to 100%. Strict cause agreement did not change (3 of 12 in every round), so this does not show better diagnosis accuracy. The candidate was tuned on those same four reports, so this is a development signal, not a held-out result. Details: debugger-dev-comparison-v1.

Full details are in the CHANGELOG.

e2e-skills 2.0.0

e2e-skills 2.0.0

Four Agent Skills for Playwright and Cypress E2E work: e2e-reviewer, playwright-test-generator, playwright-debugger, cypress-debugger.

Installs, catalog severities, and scanner flags are unchanged. No existing pattern ID changed, and no user action is required to upgrade.

What changed

• Evidence system rebuilt. Preregistered reviewer holdouts v8-v21, schema-enforced output, usage capture, the real-code fixpair-v1 corpus, and a developer-side Lean model of the severity contract. See benchmarks/STATUS.md.
• Reviewer. Findings now carry the pattern ID in every heading and instance severities for #1 and #23. New #8c (under #8): a Playwright matcher that is read but never called, or a bare expect(x);, runs no assertion. The scanner reports both shapes as confirmed [P0].
• Scanner. No longer exits incomplete when the optional #13 lookup fails.
• Security. Tier 2 now runs ast-grep with a skill-owned, rule-free sgconfig.yml (--config) from a private temporary directory, so a reviewed repository can no longer supply an sgconfig.yml that the host binary would load. Findings, flags, and exit codes are unchanged.

What the evidence does and does not show

• Reviewer holdout v19 passed its frozen full-versus-no-skill gate (stable F1 difference +0.571, 95% CI +0.411 to +0.738). It measured the pre-release 2.0.0 candidate, not the shipped reviewer, which later received small changes that no holdout has measured. It reruns v18's corpus, so it is confirmation, not fresh independent evidence. The corpus is single-host and model-authored.
• Generator v8 passed on its public pool (markup-refactor survival 0.954 versus 0.435, 6.74x generation time). Fault-detection improvement was not shown. The shipped generator text adds changes v8 did not measure.
• Debugger skills: no measured lift over a taxonomy-free prompt on real failures. The debugger text is unchanged in this release.
• The measurements are public development evidence, not sealed release results.

Tag note

An earlier v2.0.0 tag pointed at 8ec2cb8, a 2026-09-30 scanner snapshot that no release or GitHub Release ever referenced. On 2026-10-10 the tag was re-pointed to 6ded5e9, the actual 2.0.0 release commit.

If you fetched the old tag, delete it locally and fetch again:

git tag -d v2.0.0
git fetch origin tag v2.0.0

Repository size

The repository is about 34 MB to clone and about 180 MB checked out; most of it is benchmarks/ run records. The four skills themselves are the skills/ directories (about 2 MB). Use git clone --depth 1 if you only need the current tree.

Full details: CHANGELOG.

v1.19.0

Highlights

• All four skills load less up front. Every SKILL.md body now carries the triggers, gates, safety rules and output contract, and moves its step-by-step procedure into sibling files behind pointers that say when to read them. e2e-reviewer goes from about 16.7k tokens to 3.8k, playwright-debugger from 8.9k to 4.6k, cypress-debugger from 9.9k to 5.2k, and playwright-test-generator from 11.6k to 5.4k. Every sentence of every old body is preserved verbatim in the file that now owns it.
• Reviewer holdout v7 has a result, and it is FAIL. 180 Codex gpt-5.6-sol calls, three prompt arms, zero infrastructure errors. Against the no-skill control the skill passes all six lift checks (stable F1 +0.50). Against a catalog-only control that lists the 24 patterns and nothing else, it does not: +0.017 F1 where the preregistered threshold is +0.05. What the full skill clearly adds over the bare catalog is precision and clean-case specificity (1.00 versus 0.75), not recall. Three earlier attempts are archived and disclosed rather than dropped.
• The run found a real bug, and it is fixed. #3 said "never wrap assertions in try/catch", but its confirmation step asked whether an "independent meaningful verification" survived, so a later assertion on a different element excused a swallowed one. The reviewer missed that case in every repetition, with the full skill and with the catalog alone. A real assertion swallowed by a non-rethrowing catch is now P0 unless the identical check is re-asserted unswallowed, control flow fails the test regardless, or documented intent applies.
• The reviewer now sees a try/catch that swallows a gate. A controlled ablation found the reviewer caught the .catch(() => {}) swallow every time but the try/catch form at most once in six, although the contract already forbade it. The scanner raises it as a triage candidate, pairing each catch with its own try on lexed source.
• An apostrophe in JSX text no longer breaks that pairing. <p>Don't panic</p> used to open a string in the lexer and leave a real try/catch unpaired. Quotes are now inert between an opening and closing tag on the same line, which leaves TypeScript generics and comparisons untouched.
• Far fewer false credential candidates. #14 used to raise every .fill() or .type() with a string literal, so typing a todo title counted. Candidates now need a credential-shaped field or value: on the seven-repository field scan, 94 candidates drop to 25.
• The generator's command approval table says what each command does. An Effect column labels every row, so a test:e2e whose config resets a database no longer looks like one that only runs tests.
• Clearer generator contracts from live runs. When an exploration observation is sufficient without another probe, how Step 5b is ordered on a run with no tracer scenario, and what makes a real-write server "disposable" — the approved run starts and stops it, and loopback is a location, not a lifecycle.

See CHANGELOG.md for details.

What the v7 result does and does not support

It is one host, one model, and a public development corpus that anyone can read, so it is not sealed, release-grade, cross-provider, or evidence of generalization. It supports the recorded FAIL and the descriptive per-arm numbers, nothing more. No skill-lift claim is made.

An uncounted addendum reruns the same protocol on the trimmed reviewer with the #3 fix. Its full arm passes every absolute threshold, which shows the trim did not regress the tool-less holdout, and the lift gap against catalog-only is unchanged. That rerun is contaminated for the one case the fix was written against, and it is published as an addendum rather than as a result.

Verification

• /bin/bash -p scripts/ci/ci-local.sh
• /bin/bash -p scripts/ci/pre-push-security.sh
• scripts/dev/linux-script-check.sh: shipped-script suites pass on Ubuntu 24.04
• Codex smoke: all five checks pass against these skills, exercising the trimmed bodies end to end — the reviewer reaches #4f, both debuggers classify F2 through their bundled bounded readers, and the generator quotes its framed-stdin preflight command
• Reviewer holdout v7: 180 calls, 0 infrastructure errors, comparator FAIL
• Live reviewer A/B on the trimmed skill: matches or beats the untrimmed one, and opens more of its own references, because the untrimmed body let required pointers be skipped
• Pre- and post-implementation adversarial reviews for each batch, with every must-fix and should-fix item addressed

Not tested

• Live debugger and generator runs reading the new references on demand; only the reviewer was A/B'd with tools, and the smoke proves the pointers are followed, not accuracy
• An uncontaminated rerun on a corpus the #3 fix never saw

v1.18.0

Highlights

• Fixes the reviewer scanner on Debian and Ubuntu hosts without ast-grep. Every release from 1.11.0 through 1.17.0 looked up /usr/bin/sg as an ast-grep candidate. On those distributions that path is the shadow-utils group command, so each scan ran newgrp, ended INCOMPLETE, and exited 2 instead of reaching the pinned npx fallback. The lookup now names only ast-grep; E2E_SMELL_AST_GREP_BIN still accepts an explicit path to ast-grep. If you run the scanner on Linux, upgrade.
• Fixes the reviewer scanner on macOS for code with non-ASCII identifiers. Under a UTF-8 locale, macOS awk aborted on identifiers such as Angular's ɵ-prefixed exports and the whole scan exited 2; the scanner now pins the C locale, with byte-identical output. A full scan of a large Angular monorepo that used to stop after 24 seconds now completes. If you scan Angular projects on macOS, upgrade.
• Large scans are several times faster. The scope worker re-checked every recorded dependency (about 240,000 paths on ever-gauzy) before and after every query, which took 4,683 of the scan's 5,932 seconds. Each query now re-checks only the files it read, and the full set is still validated at every tier checkpoint and before any Summary, so a file changed mid-scan still fails the scan with exit 2. The same scan now takes 1,226 seconds with byte-identical output; small projects are unchanged.
• A correct generator run can reach Complete in three cases the text used to block. V2-V4 probe copies are covered by a <spec> slot on the approved targeted-run row instead of needing new approvals; a parallel mode a serial suite does not use is N/A in V5 rather than CANNOT_VERIFY; and a V6 verdict relayed by another agent counts when its verdict line is quoted verbatim with the reviewer's identifier. About twenty smaller clarifications from the same live runs ship with them.
• The generator discloses every file it will write and checks what it actually wrote. Step 4 adds a Proposed generated files table (specs, Page Objects, helpers, fixtures, setup projects, playwright.config.* edits, snapshot directories, planner output), covered by the approval gate. Step 3 records git status and SHA-256 hashes before exploration; the user's uncommitted work is never reverted or blamed on the candidate, and any written path outside the approved tables is reported, not deleted, and returns PARTIAL/BLOCKED.
• "Nothing was stored" must be re-read. After a rejected write, the generator reloads or waits for the re-fetch before asserting absence; an absence read from a view the failure never refreshed passes even if the server stored the data. The reviewer gains the matching Phase 2 judgement next to #4i, reported as #4 P1. No pattern ID, severity, or scanner rule changed.
• A verified guard for Playwright CLI exploration. The CLI's network.allowedOrigins config, from outside the worktree, in a fresh named session; a fetch to an unapproved loopback port must fail with net::ERR_BLOCKED_BY_CLIENT before navigation.
• Clearer verdicts. Step 3 asks before starting webServer.command; mixed per-scenario verdicts report the most restrictive status with a breakdown; the completion matrix now covers V1-V3.
• For contributors: a pre-push Linux check for the shipped-script suites, a digest helper, a live generator runbook, and a Releases section in CONTRIBUTING.md: the skills CLI updates on content while plugin installs update on version, so every push that changes skills/ bumps the version.
• Bumps all four skills and all three plugin manifests to 1.18.0.

See CHANGELOG.md for details, including a real-repository scanner timing (212 s to 192 s on cal.com's Playwright directory) and the ever-gauzy profile behind the scope-worker change.

Verification

• /bin/bash -p scripts/ci/ci-local.sh
• /bin/bash -p scripts/ci/pre-push-security.sh
• scripts/dev/linux-script-check.sh: six suites pass on Ubuntu 24.04; v1.17.0 fails seven scanner checks there
• Three live generator runs on a local todo app, the last two from the runbook with no command pre-approved: all Complete, every written path matched an approved row, the user's files unchanged; the third reused the <spec> slot for ten probe runs
• Pre- and post-implementation adversarial reviews for each batch, with every must-fix and should-fix item addressed

v1.17.0

Highlights

• The generator runs the approved existing tests once before the tracer scenario. A green run becomes the baseline; a red run outside the candidate's surface is recorded and never blamed on the candidate; a red run on the candidate's own surface returns PARTIAL/BLOCKED. A target area with no existing coverage records baseline not applicable instead of being unable to complete.
• Error scenarios must name the signal that tells their failure cause apart. A scenario that only proves "an error appeared" passes for every cause that reaches the same screen. Products that show one message by design are recorded as GENERIC_BY_CONTRACT, with the rule that requires it or NEEDS_PRODUCT_CONTEXT for the approval gate, and V3 then faults the failure into a success instead of swapping one cause for another.
• Scenarios may carry up to three named secondary outcomes (survives a reload, side effect proved, error cause distinguished), each recorded as selected or skipped. V1 still keeps one primary outcome.
• Exported manual test cases from TestRail, Zephyr, Xray, Qase, or similar can serve as the requirement source. They are treated as untrusted data, and steps that contradict the observed product are reported at the approval gate.
• A proposed reviewer rule was tested before it was written, and refused. benchmarks/stub-echo-v1 preregistered its thresholds, then adjudicated 78 real-repository candidates: 71 real backend, 7 app-mediated, 0 stub echoes. The mechanical rule would have been a false positive every time, so it became a Phase 2 judgement item instead.
• The scanner is about 4x faster on hit-dense files (30.9s to 7.5s on the recorded fixture) with byte-identical output, and no longer loses its place when a regex literal contains a quote, a case that could hide P0 findings for the rest of the file.
• The opt-in ast-grep download now works. It previously always failed; it now fetches the platform package and verifies its SHA-256 against pinned values. Hosted CI installs ast-grep from an integrity-pinned package without lifecycle scripts.
• Skill prose is one physical line per paragraph, as CONTRIBUTING.md already required. No wording changed. The four SKILL.md files now run 302 to 434 lines.
• The evidence ledger now audits 62 sources. It records that no published measurement of Playwright's own Test Agents healer was found, and it does not cite the widely repeated "~75% success" figure, which appears in no Playwright source.
• Two unpublished interim versions (1.16.4, 1.17.1) are folded into this release. All four skills and all three plugin manifests carry 1.17.0.

See CHANGELOG.md for the full list of fixes, including scanner JUSTIFIED scope, #7/#8a accuracy, debugger execution gates, report-reader edge cases, secret-scanner token patterns, and pre-push hook safety.

Verification

• /bin/bash -p scripts/ci/ci-local.sh
• /bin/bash -p scripts/ci/pre-push-security.sh
• Pre-push review: 16 passed
• Pre-push security: 10 passed, 0 warnings, 0 blockers

v1.16.3

Highlights

• Publishes the completed Codex-only healer perturbation benchmark: all 30 frozen revision 12 cells completed and the preregistered decision is REJECT.
• Records 9 mechanical repairs, 2 honest no-fix outcomes, and 4 weakening repairs for the official arm; 9, 3, and 3 respectively for the proposed arm. Both arms weakened all three impossible-repair cases.
• Preserves every protocol revision, smoke run, authorization, result, and attempt artifact; pre-squash history remains available at healer-perturbation-v1-history.
• Refreshes benchmark status, CHANGELOG, all four README language editions, and the dated GitHub-star snapshot.
• Raises only the archive-heavy shell enumeration timeout from 180 to 300 seconds. The frozen healer digest contract now ignores release-only skill version metadata while continuing to reject behavior or fixture drift, including in shallow clones and source archives.
• Bumps all four skills and all three plugin manifests to 1.16.3. Skill behavior is unchanged.

Verification

• /bin/bash -p scripts/ci/ci-local.sh
• /bin/bash -p scripts/ci/pre-push-security.sh
• Pre-push review: 16 passed
• Pre-push security: 10 passed, 0 warnings, 0 blockers

Claude execution was intentionally excluded by the benchmark protocol.

v1.16.2

Highlights

• Completed the Claude-host subagent-routing pilot and its full-rigor confirmation. Both finding verification and failure classification now use INLINE_DEFAULT: work inline by default, delegate only when the inline result is uncertain, and keep the inline verdict on disagreement.
• Added the three-tier contribution process and strengthened the routing contract test against semantically inverted wording.
• Documented why Codex named/native-role delegation remains unavailable in the measured environment while preserving the reliable inline fallback.
• Moved conditional Playwright/Cypress debugger artifact procedures into one-hop references while retaining required routing and safety rules in each skill entrypoint.
• Corrected Playwright trace comparison so timing and product failures defer to the canonical F1-F15 classification instead of being assigned the wrong F-codes.
• Bumped all four skills and all three plugin manifests to 1.16.2.

See CHANGELOG.md for the full benchmark record, limitations, and detailed changes.

v1.16.1

Changed

• Bundle version bumped to 1.16.1 across all four SKILL.md files and all three plugin manifests. This deliberately invalidates the sha256_at_preparation digests that benchmarks/healer-perturbation-v1/protocol.json and benchmarks/subagent-routing-v1/protocol.json had pinned against three of the four SKILL.md files (playwright-test-generator, e2e-reviewer, playwright-debugger) — both protocols are still NOT_RUN and both now record the invalidation and the re-preparation requirement in their own protocol.json and README.md. Shipping this release was prioritized over keeping those still-unexecuted protocols' preparation-time digests valid; neither protocol's design, arms, or decision rule changed.
• docs/llm-generated-e2e-test-evidence.md corrected three overclaimed external-evidence rows. NAVER's "explain the harness" framing was not actually adopted anywhere in this repo and the claim was removed; Slack's adapter-comparison claim was flagged as unmeasured and cross-referenced to benchmarks/exploration-fallback-v1/; Shopify's writer/verifier framing was reworded from "borrowed idea" to "converges with," since this repo's design predated reading that article. A Toss FE platform team talk and a flex.team backend post were independently evaluated and recorded as not adopted, each with a specific reason (native-app-platform infrastructure the talk assumes this repo doesn't have; a JVM backend stack out of this repo's Playwright/Cypress scope, whose underlying principle is already covered by reviewer patterns #10b/#19/#20).

Fixed

• The hosted HOL Plugin Scanner false-positived on six long-standing files. .plugin-scanner.toml gained dated ignore_paths entries (scan.sh, three SKILL.md files, preflight_target.py) explaining each false-positive class, and scripts/ci/test-security-gates.py's HOL_SCANNER_FALSE_POSITIVE_PATHS set was kept in sync with it — the two had silently drifted apart.
• scripts/ci/test-parity.sh's CHANGELOG mutation test broke when this changelog was reflowed to one line per paragraph. The mutate-string literal still assumed the pre-reflow hard-wrapped bytes (with eighteen tokens\n of headroom), so the check silently stopped exercising the real file. Restored to a single-line literal matching this file's current format.
• scripts/ci/lib/scan-security-policy.py's hardcoded-home rule missed two real leak shapes. It only matched a literal /Users/ or /home/ substring, so a dash-encoded session-scratchpad path and an embedded ls -la tool-output owner column both evaded it — exactly how a real account-name leak reached committed pilot evidence in this release before being caught and redacted. Added a dash/underscore-bound Users pattern and an ls -l-shaped owner-column pattern; deliberately excluded a bare lowercase home pattern after a first attempt produced ~524 false positives from ordinary compound identifiers.

Added

• Roadmap item 1's real 36-cell pilot ran to completion: ours_only vs planner_plus_ours, result CONDITIONAL. benchmarks/ours-vs-planner-pilot-v1/ records the full run (46 total top-level sessions, 17.3 serial hours, $276.38 nominal cost) plus a human adjudication (adjudication.md) of the mechanically computed preliminary label, required because target A showed no majority-stable complete-rate lift. The planner's absorbed deltas on target A were real and correctly load-bearing within their own sessions, but ours_only's own repetitions reached at least as good a majority-stable outcome on both measured scenarios without a planner step (tied on A-S1, and — see pilot-results.json's known_scoring_corrections — actually ahead on A-S2), so the "independently verified new failure condition" escape clause does not apply either. No product text changes: the Planner step remains optional, not default. Two infrastructure incidents (keychain OAuth token issues mid-run) are documented in pilot-results.json's auth_events: one used the protocol's pre-registered infrastructure-only retry allowance (cap 1), and one exceeded that cap and was recovered via an explicit operator-approved exception, also documented in the same record.
• Four roadmap items (3, 4, 5, 6) gained preregistered protocol designs; none is authorized to run. benchmarks/healer-perturbation-v1/ defines the Healer-vs-honesty-control comparison (5 perturbations, 2 arms) and wires its contract tests into ci-local.sh. benchmarks/subagent-routing-v1/ defines the named-subagent-vs-inline-fallback comparison for e2e-finding-verifier/e2e-failure-classifier. benchmarks/stryker-feasibility-v1/ gained a full protocol.json (previously README-only) for the already-decided REFERENCE_ONLY-by-default Stryker adjunct question; it pins no skills/*/SKILL.md file, so it never competes for the same frozen surface as items 3/6. benchmarks/exploration-fallback-v1/ records two deferred candidate exploration arms (recorded-trace, adapter-usage logging) with concrete re-trigger conditions instead of a time-based reconsideration schedule.

v1.16.0

Added

• A preregistered reviewer release protocol now defines what release-grade
  evidence would require. The new 120-case, three-arm design freezes
  correctness, skill-lift, stability, and blinded helpfulness gates alongside
  external-custody and signed-isolation requirements. It remains NOT_RUN /
  INCONCLUSIVE; this release adds and validates infrastructure only and makes
  no new accuracy claim. The model executor and human-adjudication operation
  are not implemented.
• e2e-reviewer now constrains what a review says, not only what it
  detects. A clean result reports the clean verdict and its scope
  limitations without volunteering selector, payload, coverage, style, or
  general-improvement advice outside the 24-pattern catalog; limitations stay
  in the evidence header rather than inside a finding or the summary, and may
  not be relabelled as a non-blocking observation to get past that; each
  finding carries one minimal evidence-backed fix; no alternative may reduce
  what the test proves; and always is reserved for evidence proving an
  unconditional outcome. Its contract test also asserts the detection surface
  is unchanged, so tightening what the reviewer says cannot quietly shrink
  what it finds. Eval 4 previously required coverage-gap suggestions on a
  clean fixture, which contradicted the first rule.

Changed

• Local CI uses four workers for its two process-heavy suites by default.
  The reviewer scanner still derives its worker count from available cores but
  now caps it at four; the disposable parity suite also defaults to four
  full-tree copies instead of six. Both retain their existing environment
  overrides. This avoids oversubscribing a ten-core host: four scanner workers
  completed faster than five in adjacent runs, and four parity workers passed
  all 51 smoke checks after a six-worker run produced an incomplete disposable
  snapshot.
• Playwright generation now uses a CLI-first, verification-first workflow.
  Browser exploration prefers a compatible project-local Playwright CLI
  (playwright cli), then the playwright-cli command from an already-installed
  standalone @playwright/cli package, agent-browser, an existing Playwright
  MCP surface, and finally the restricted ARIA fallback. The deprecated
  unscoped playwright-cli package is not used. Exploration remains separate
  from execution: every candidate still runs with the repository's native
  Playwright Test command and passes the applicable V1–V6 checks before it is
  accepted. Playwright's first-party agents are optional planning aids behind
  an admission gate, not competing final implementers.
• CLI-first is now documented as a portability default, not a universal
  efficiency claim. It avoids requiring MCP registration, while the evidence
  ledger records Slack's environment-specific result where MCP used fewer
  tokens and completed faster than its CLI harness.
• Generation now admits risk before producing code and expands from one
  proven tracer. Every scenario identifies its distinct user risk, why E2E
  is the right layer, and how a later failure will be diagnosed. First-run,
  three-or-more-scenario, authentication, write, fixture, and runner-sensitive
  work marks a representative tracer; any requested expansion waits until that
  scenario passes review and V1–V6.
• Local CI now fails earlier on stale reviewer evidence. The evaluated-skill
  digest check runs immediately after shell trust-boundary validation, while
  the remaining inexpensive release contracts run before the multi-minute
  review stage.
• Historical evidence now states its actual limits. Six internally
  contradictory rows in the archived 100-PR AI-reviewer pilot reduce its
  model-judged issue denominator from 110 to 104, so the pilot is explicitly
  invalidated as a current performance claim. Reviewer holdout v3 prose now
  matches its committed report, and the v6 record distinguishes first-run
  report loss from the later CLI-rotation rerun obstacle.

Fixed

• Scanner limits now apply after exact file-scope checks and necessary
  discovery guards. Unrelated files and expressions no longer exhaust rules
  that would reject them. Focused-test discovery retains computed member calls
  without matching every array expression. The default limits and downstream
  classifiers remain unchanged; any suppressed rule still makes the scan
  incomplete.
• Scope analysis reuses one private worker and its lexical metadata.
  Ordered, depth-bounded import traversal and dependency validation remain in
  place. Small ASCII sources can use an equivalent Python lexer; unsupported
  inputs and explicit tool overrides retain the shell path. Ordinary pathname
  operations avoid extra processes while preserving physical-root checks.
• Field-scan rendering requires completion evidence. A process that exits
  with an error, lacks a complete Summary, suppresses a rule, or has
  unreconciled counts is incomplete. Partial counts remain visible as floors;
  timeouts and fetch failures also make aggregate counts floors.
• e2e-reviewer #3 error swallowing no longer confirms what it has not
  proven. Every #3 hit — including a .catch( attached to the assertion
  itself where the assertion returns a promise that can actually reject (a
  web-first matcher, toPass(), or a matcher under expect.poll) — now
  reports as [LLM-TRIAGE] rather than a confirmed P0 defect; the default
  E2E_SMELL_FAIL_ON=p0 gate does not exit non-zero for any #3 hit. Use
  E2E_SMELL_FAIL_ON=p0-candidate to also gate on these triage candidates.
  Previously any assertion-looking token anywhere in the evidence window was
  enough to confirm the finding, so a catch on a neighbouring statement, a
  synchronous expect(1).toBe(2) whose .catch() never runs, and value
  conversions such as .toString() were all reported as confirmed defects.
  Swallowed actions, navigation, queries and readiness gates were already
  [LLM-TRIAGE] and remain so. The pattern id and its documented P0 severity
  are unchanged and no candidate is dropped.
• Field scan v1 evidence regenerated under its frozen conditions.
  10/12 pinned repositories completed with no suppressed rules; 2 timed out.
  Complete scans reported 0 P0 hits. The generated ledger and documentation
  preserve the historical correction to the over-promoted 294 #3 hits and
  distinguish unavailable results from zeroes.
• Field scan v1's designed 12-repository sample is now fully measured.
  v1's own selection rule set the sample at 12 repositories and its own
  results self-declared "Incomplete: 10/12 repositories completed with no
  suppressed rules" — ever-co/ever-gauzy and open-mercato/open-mercato,
  the two rows in the entry above, never produced a measurement under the
  frozen 30-minute budget. A separate,
  disclosed benchmarks/field-scan-v1-extension
  protocol (no wall-clock cutoff; bounded per-rule hit/byte limits raised to
  their documented hard ceilings, E2E_SMELL_MAX_RULE_HITS=10000 /
  E2E_SMELL_MAX_RULE_BYTES=16777216) completes both rows: natural scanner
  exit (exit 0), zero suppressed rules, a reconciled hit count, unchanged
  source fingerprint, 0 P0 hits, 0 model calls. The frozen
  benchmarks/field-scan-v1/ledger.json from the entry above is unchanged;
  this is scanner-completion evidence under a separate disclosed protocol,
  not a rerun of the v1 pool and not reviewer-accuracy evidence.
• #4i no longer contradicts itself on where a locator may be proven. The
  Rule required the locator to be proven "somewhere in that test's execution
  path" and the FLAG criterion fired only when it "appears nowhere else", but
  the SKIP bullet credited proof appearing "earlier in the test or its
  beforeEach". A locator proven later satisfied the Rule and the FLAG
  criterion while failing the literal SKIP bullet, which produced false
  positives on specs whose absence assertion was followed by toHaveCount(1)
  and a real action on the same locator. Direction cannot matter: a later use
  fails just as loudly when the selector rots. The SKIP wording is now
  direction-neutral across pattern-reference.md, SKILL.md,
  grep-patterns.md, and the scanner's triage comment; absence-assertion.spec.ts
  gains a fifth case for the later-proof shape and eval 22 asserts it by line
  alongside the unchanged true positive. No pattern id, title, severity, or
  scanner regex changed.
• #3 no longer exempts a swallowed gate because of where it sits. The
  rule excused "try/catch in non-assertion code (setup, teardown, optional
  cleanup)", which covers a swallowed readiness waitFor inside an imported
  helper and a swallowed response-status assertion inside a
  cy.wait().then() callback. Both look ancillary; both decide whether the
  assertion that follows means anything. The exemption is now stated by
  consequence — exempt only when the swallowed failure cannot change what the
  test proves — so best-effort teardown stays exempt while a wait, gate, or
  status check a later assertion depends on does not. The contract also names
  where a swallow can hide: any file the spec reaches, including an imported
  helper or support module, a custom command, and callback bodies. Reviewing
  the spec alone missed one of these entirely, so eval 39 requires the helper
  file to be named. A controlled ablation on a fresh corpus found no
  measurable detection improvement from this wording — 0.19 with the new text
  against 0.17 with the old, one detection out of thirty-six, and no difference
  at all for one of the two providers. Five of six fresh error-swallow
  positives went undetected in both arms. The wording is more accurate than
  what it replaced and is kept on that basis, but it is not a detection
  improvement: a reviewer told to read imported helpers sti...

v1.15.1

Added

• The reviewer now covers assertion loops over unproven collections (#4k).
  It triages Playwright locator.all() loops and Cypress .each() callbacks
  whose assertions can execute zero times, while preserving guards for explicit
  non-empty and collection-size proofs.
• The reviewer now identifies unexplained test skips (#11c). Bare
  skip/fixme calls are P2 candidates, while conditional skips, reason
  strings, ticket/date comments, and JUSTIFIED markers remain intentional.
• A public workflow for reviewing AI-generated Playwright and Cypress tests.
  The guide includes copyable prompts, a false-green before/after example,
  deterministic scanner output, the semantic review evidence fields, and
  direct FAQ answers.
• Playwright debugger guidance for the supported trace CLI. On trusted
  projects using Playwright 1.59 or newer, the debugger can use trace actions,
  snapshot evaluation, pass/fail trace comparison, and multi-trace failure
  clustering after the execution gate passes.

Changed

• README discovery now leads with AI-generated-test review and false-green
  proof. It links the public workflow, explains how the bundle complements
  Playwright Test Agents and Cypress AI Skills, and records the current Kimi
  listing and skills.sh install milestone without treating either as accuracy
  evidence.
• Local CI now reports stage timings and avoids duplicate validation work.
  The parity smoke suite no longer reruns an unchanged security gate for every
  ordinary mutation, and the reference-tokenizer unit suites no longer rebuild
  three identical standalone wrapper environments after already exercising
  those wrappers under adversarial conditions. Security-specific mutations and
  the canonical pre-push gate still execute the real security script.
• Scanner contract runs report their slowest checks. The worker ceiling is
  unchanged after a higher concurrency setting measured slower on the current
  host.

Fixed

• test-fixture-faults.py no longer fails under CPU contention. Two
  process-cleanup cases gave the parent 300ms to start a Python interpreter,
  spawn a nested one, and flush a line before the timeout fired, so a loaded
  machine failed the output assertion instead of a real regression. Every
  interval is rescaled and the ordering the test proves is unchanged.

• Behavioral process-cleanup checks no longer race their PID fixtures. The
  two live process-group cases now allow enough startup time for a loaded host
  to launch Bash, spawn the nested process, and flush its PID before timeout;
  the TERM/KILL cleanup assertions and their failure conditions are unchanged.

• manifest_phrase_contract.py is now reviewer_taxonomy_contract.py. It
  stopped holding manifest phrases when 1.15.0 moved the taxonomy out of the
  package descriptions; the name described what it no longer did.

• Corrected a false claim about why holdout v5 and v6 stalled. The 1.14.0
  notes, benchmarks/STATUS.md, benchmarks/reviewer-holdout-v6/README.md,
  and AGENTS.md all said the pinned Claude Code builds had been pruned and the
  protocols were "permanently unrunnable". Only the local installer rotated
  those builds away. The vendor release channel still serves both 2.1.220 and
  2.1.239, and the runner accepts an explicit --runner-path, so re-fetching
  a pinned build completes the matrix. The minimum version policy is unchanged
  and still worth having — a protocol that needs a manual download before it can
  start is one nobody reruns — but its stated justification was wrong and is now
  accurate.

• Restored the live-exploration scope to the package descriptions. The
  1.15.0 shortening dropped "live exploration only on local/disposable or
  externally isolated approved non-production targets" from all three manifests,
  and narrowed the CI guard to the generator's trigger surfaces in the same
  commit, so nothing caught it. A user reading the marketplace listing had no
  indication the generator drives a real browser. The phrase is back and the
  guard now covers the package descriptions again.

• Reports no longer claim an identity policy that was never applied.
  execution_identity_policy was written as "exact" even when no identity
  check ran — a custom runner, or a protocol without execution_identity. It is
  now None in that case, so a provenance artifact stops asserting a check the
  harness did not make.

• The Quick Reference order check no longer hardcodes the pattern set. It
  derived the expected order from a literal 1..23 + "3b", so adding pattern
  #24 in the correct position failed with a row-order error. The order is now
  derived from the IDs present, and AGENTS.md records that review.sh needs no
  edit when a pattern is added.

• Documented the verified source of the skill-manifest limits. The 64/1024
  caps in skill_metadata_contract.py carried no citation. They are correct,
  confirmed against openai/codex rust-v0.151.0 (MAX_NAME_LEN,
  MAX_DESCRIPTION_LEN), and the module now records that an over-length field is
  dropped rather than truncated, and that Codex counts Unicode characters after
  collapsing whitespace.

• Corrected the v6 archive's account of full-opus. The README said it
  never ran. opus.log shows it completed all 60 scheduled runs with zero
  infrastructure errors; only its report file was lost. The conclusion that no
  usable full-arm report survives is unchanged, because the per-run records
  the primary metric needs are gone, but the stated fact was false and
  contradicted the same file's own list of lost reports.