Repository navigation
v3.3.0
Added
-
Optional test-authoring memo for a project's
CLAUDE.mdorAGENTS.mdthat targets failure-path behaviors (rejection rollback, validation, server confirmation, duplicate submit, pending state), with a compact evidence record and its limits. An independent replication with failure-path faults on two public example apps (48 cells, two models) found 0/120 without the memo and 98/120 with it; an earlier replication with only ordinary-path faults found no difference. No skill loads the memo. -
Optional test-failure policy for the workspace-root
CLAUDE.mdorAGENTS.md, with a measurement summary and the verbatim session summary. "With the always-loaded test-failure policy, the models we measured rewrote the failing test much less often, and when they left it unchanged they reported the app/test mismatch (3 apps, 24 pairs over two iterations, 3 models; the reduction differed by model, e.g. gpt-5.6-sol still had 69% risk with snippet v1)." Unresolved CI-red cells grew beyond the pre-registered limit; a blinded reading found that those cells report the mismatch (coarse check: two blind judges, 126/126, but the rubric did not discriminate). The same rule inside a skill file did not meet its threshold (a -9 pp reduction on different pairs; placement not isolated). No skill loads the policy. -
Opt-in
e2e-skills guidance --host codex|claude --project PATH [--memo] [--policy] [--dry-run]writes the memo, the policy or both, as requested (at least one flag is required), unchanged between HTML comment markers into the project'sAGENTS.mdorCLAUDE.md, replacing only its own marked blocks on rerun; see installation.installanddoctornever write them. -
Runtime-probe measurement in the same summary: "In our setup, when the runtime probe actually ran, Claude caught false-green tests that it missed by reading alone (2 of 13 pairs had such misses; direction only)." Limits: one reviewer model; every cell of both arms also loaded the operator's user-level
~/.claude/CLAUDE.md(equal across arms); the probe was run through a prepared wrapper and plan, so this measures using the verdict, not finding the probe; few pairs with headroom; static apps only; not a claim about real repositories. -
Behavior-fault probes expose an
INVERTED_RESPONSEreview diagnostic when the same test passes with an observed promise violation and fails with the pinned error text and location in both normal and restored phases. The probe remainsCANNOT_VERIFY; the diagnostic requires assertion review and does not establish assertion execution, a catalog finding or author intent. -
Diff reviews compare changed test evidence with sourced acceptance requirements, distinguish approved requirement changes from bug-conforming expectations, and retain missing-contract uncertainty.
-
Source-pinned review tasks for unsupported try/catch and parameterized catch propagation. The
propagation-unsupporteddiagnostic identifies these gaps without adding findings or clearing incomplete status. -
Source-pinned review tasks for Promise receivers whose fixture import is unresolved. Review checks the surrounding call and Promise handling without changing findings, diagnostics or incomplete exit status.
Changed
expected_failure.locationin a probe plan is resolved against the phase project and must be a project-relativepath:line[:column](for exampletests/checkout.spec.ts:12:5). 3.2.1 matched it as a substring, so a basename or original-checkout absolute location that linked there now returnsCANNOT_VERIFY. Plans that pinned a basename or absolute location must be rewritten as project-relative.
Fixed
-
Trust a coordinate written inside an error message, inline or in parentheses, only when the text has no real stack frame, so a message-embedded location cannot outrank a frame in another file.
-
Link Cypress default-webpack error frames only to exact selected test paths; retain rejection of other files, namespaces and mismatched coordinates.
-
Match complete native failure location tokens and phase file paths (including
at asyncandfile://Node frames in text-only stacks), and exclude separate source snippets and numbered native code frames from assertion-failure evidence, preventing different files or source context from establishing detection or an inverted-response diagnostic. -
Correct the sidebar-toggle review eval: a verified hidden sidebar does not imply an unverified content-expansion requirement.
-
Skip zero-argument action calls when checking force options; calls with options or spread arguments retain receiver diagnostics.
-
Clarify that a completed source review task can produce no catalog finding while the scanner remains incomplete.
-
Align the README generator description with the shipped workflow (approved project test command, new specs only, stop when issues remain or scanning is incomplete), and correct the AGENTS.md subagent-registration note and skills.sh link.
-
Avoid unresolved catch tasks for a helper that only rethrows its identifier-bound error. Immediate finally blocks and generator helpers remain outside this narrow exclusion; enclosing catch/finally evidence is preserved.