Skip to content

v1.15.1

Choose a tag to compare

@voidmatcha voidmatcha released this 01 Sep 18:38
241b5cd

Added

  • The reviewer now covers assertion loops over unproven collections (#4k).
    It triages Playwright locator.all() loops and Cypress .each() callbacks
    whose assertions can execute zero times, while preserving guards for explicit
    non-empty and collection-size proofs.
  • The reviewer now identifies unexplained test skips (#11c). Bare
    skip/fixme calls are P2 candidates, while conditional skips, reason
    strings, ticket/date comments, and JUSTIFIED markers remain intentional.
  • A public workflow for reviewing AI-generated Playwright and Cypress tests.
    The guide includes copyable prompts, a false-green before/after example,
    deterministic scanner output, the semantic review evidence fields, and
    direct FAQ answers.
  • Playwright debugger guidance for the supported trace CLI. On trusted
    projects using Playwright 1.59 or newer, the debugger can use trace actions,
    snapshot evaluation, pass/fail trace comparison, and multi-trace failure
    clustering after the execution gate passes.

Changed

  • README discovery now leads with AI-generated-test review and false-green
    proof.
    It links the public workflow, explains how the bundle complements
    Playwright Test Agents and Cypress AI Skills, and records the current Kimi
    listing and skills.sh install milestone without treating either as accuracy
    evidence.
  • Local CI now reports stage timings and avoids duplicate validation work.
    The parity smoke suite no longer reruns an unchanged security gate for every
    ordinary mutation, and the reference-tokenizer unit suites no longer rebuild
    three identical standalone wrapper environments after already exercising
    those wrappers under adversarial conditions. Security-specific mutations and
    the canonical pre-push gate still execute the real security script.
  • Scanner contract runs report their slowest checks. The worker ceiling is
    unchanged after a higher concurrency setting measured slower on the current
    host.

Fixed

  • test-fixture-faults.py no longer fails under CPU contention. Two
    process-cleanup cases gave the parent 300ms to start a Python interpreter,
    spawn a nested one, and flush a line before the timeout fired, so a loaded
    machine failed the output assertion instead of a real regression. Every
    interval is rescaled and the ordering the test proves is unchanged.

  • Behavioral process-cleanup checks no longer race their PID fixtures. The
    two live process-group cases now allow enough startup time for a loaded host
    to launch Bash, spawn the nested process, and flush its PID before timeout;
    the TERM/KILL cleanup assertions and their failure conditions are unchanged.

  • manifest_phrase_contract.py is now reviewer_taxonomy_contract.py. It
    stopped holding manifest phrases when 1.15.0 moved the taxonomy out of the
    package descriptions; the name described what it no longer did.

  • Corrected a false claim about why holdout v5 and v6 stalled. The 1.14.0
    notes, benchmarks/STATUS.md, benchmarks/reviewer-holdout-v6/README.md,
    and AGENTS.md all said the pinned Claude Code builds had been pruned and the
    protocols were "permanently unrunnable". Only the local installer rotated
    those builds away. The vendor release channel still serves both 2.1.220 and
    2.1.239, and the runner accepts an explicit --runner-path, so re-fetching
    a pinned build completes the matrix. The minimum version policy is unchanged
    and still worth having — a protocol that needs a manual download before it can
    start is one nobody reruns — but its stated justification was wrong and is now
    accurate.

  • Restored the live-exploration scope to the package descriptions. The
    1.15.0 shortening dropped "live exploration only on local/disposable or
    externally isolated approved non-production targets" from all three manifests,
    and narrowed the CI guard to the generator's trigger surfaces in the same
    commit, so nothing caught it. A user reading the marketplace listing had no
    indication the generator drives a real browser. The phrase is back and the
    guard now covers the package descriptions again.

  • Reports no longer claim an identity policy that was never applied.
    execution_identity_policy was written as "exact" even when no identity
    check ran — a custom runner, or a protocol without execution_identity. It is
    now None in that case, so a provenance artifact stops asserting a check the
    harness did not make.

  • The Quick Reference order check no longer hardcodes the pattern set. It
    derived the expected order from a literal 1..23 + "3b", so adding pattern
    #24 in the correct position failed with a row-order error. The order is now
    derived from the IDs present, and AGENTS.md records that review.sh needs no
    edit when a pattern is added.

  • Documented the verified source of the skill-manifest limits. The 64/1024
    caps in skill_metadata_contract.py carried no citation. They are correct,
    confirmed against openai/codex rust-v0.151.0 (MAX_NAME_LEN,
    MAX_DESCRIPTION_LEN), and the module now records that an over-length field is
    dropped rather than truncated, and that Codex counts Unicode characters after
    collapsing whitespace.

  • Corrected the v6 archive's account of full-opus. The README said it
    never ran. opus.log shows it completed all 60 scheduled runs with zero
    infrastructure errors; only its report file was lost. The conclusion that no
    usable full-arm report survives is unchanged, because the per-run records
    the primary metric needs are gone, but the stated fact was false and
    contradicted the same file's own list of lost reports.