Skip to content

docs(review): v0.66.0 clean-room cold review of 2348afe0 - #1252

Closed
avrabe wants to merge 1 commit into
mainfrom
docs/v066-cold-review
Closed

docs(review): v0.66.0 clean-room cold review of 2348afe0#1252
avrabe wants to merge 1 commit into
mainfrom
docs/v066-cold-review

Conversation

@avrabe

@avrabe avrabe commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Clean-room cold review of 2348afe0 (release(v0.66.0)), performed with no stake in the release shipping and no PR body/CHANGELOG prose trusted as evidence on its own — every number re-derived directly from the tree.

Result up front

Zero shipped-byte defects. Every finding is in the release's account of its own work, not in a compiler output, encoder byte, or frozen golden.

Lead finding — FALSE

"84 of 89 pinned parity entries (585 of 595 assertions)" contradicts the release's own scorecard fifteen lines above it ("known_open_pins went 104 -> 27 -> 127. It CLOSED 77 wrong-answer pins..."). Independently extracted the KNOWN dict from scripts/repro/selector_parity_197_differential.py at both the v0.65.0 tag and HEAD via ast.literal_eval: the true closure is 77 of 89 entries (567 of 595 assertions), and the true residual is 12 entries / 28 assertions across three issues (#1204 6/15, #1206 1/2, #1215 5/11) — not "5 entries / 11 assertions" as claimed in CHANGELOG.md, RQ-66-BOTHWRONG.yaml, and this PR's own body. #1204 and #1206 are untouched by this release and still compute wrong answers live (confirmed by running the differential against a freshly built binary) — neither is disclosed in the residuals list. Root cause: the artifact conflates "the five issues RQ-66-BOTHWRONG addresses" (82 entries/578 assertions) with "the whole tracked population" (89/595, which also includes #1204/#1206), plus a bare arithmetic slip (595-11=584, not 585).

Also found

  • RQ-66-UNWATCHED's artifact status was never updated after its work shipped — still status: proposed, no landed:, verified-by: still pre-work "PENDING" — even though claims.yaml and CHANGELOG.md both describe its four new oracles as landed. This is a second, live instance of the exact gate hole this release itself filed as #1250 (which fixed three other artifacts but missed this fourth). Consequence: only 6 of 8 artifacts actually carry status: implemented — the release commit subject and this PR's own title ("8 artifacts, 7 implemented") aren't supported by the tree today. The fix is the artifact field, not the title — the work genuinely shipped.
  • A leftover CHANGELOG.md [Unreleased] section (above [0.66.0], never folded in — confirmed via git show 2348afe0 -- CHANGELOG.md, a pure insertion below it) states a stale emulation floor, 367751, vs. the live 367759 (oracle_wiring_check.py, matching this PR's own body). Exactly explained: ra003_join_1230_differential.py (one of RQ-66-UNWATCHED's new oracles) adds 8, and 367751 + 8 = 367759.
  • Two stale comments in claims.yaml (no gate impact — verified against claim_check.py's own logic): the known_open_pins ratchet's prose says "104 -> 27 -> 133" / "+29" in two places, which matches neither the pinned value: 127 nor CHANGELOG.md's own correct "+23"; and "THE POPULATION TRIPWIRE" comment says "six tables (6+2)" where the tree has ten (the machine-checked expect: 12 is itself correct).
  • "Five frozen goldens re-frozen" matches no reading — three files, seven distinct pinned-value changes, five in one file alone. The executing-differential claim (flight_seam_differential.py, unicorn vs. wasmtime) was built and run for real and is confirmed genuine.
  • RQ-66-PINDEBT's verified-by: is also still pre-work "PENDING" text despite its landed: field reporting the work as done.
  • Claim 13's "six merges so far" does not appear anywhere in CHANGELOG.md, this PR's body, or any artifact — it's a stale number in the review's own input briefing, not a release claim, so it is reported as coverage rather than a defect. The "0-line diff" half of that claim is confirmed universally across all 41 recorded merges.

Confirmed clean

The other nine claims (bug-attribution for #1210/#1211/#1215, #1209 memory64 refusal and unmeasured reach cost, the ARM census's load64.wast-alone departure, the memory64-family filename-regex false positives, the pin-count rise disclosed plainly as a headline, the waiver mechanism, mutants_untested, the mutation survey's 27% figure, the VARVE hazard's RECORDED-not-WATCHED disposition, and ARCHMODEL's cut-time re-verification) were independently re-derived and hold. Full detail, evidence, and exact file/line citations in docs/reviews/v0.66-cold-review.md.

Refs #1189 #1210 #1229 #1236

🤖 Generated with Claude Code

https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L

Independent verification of the 14 claims in the v0.66.0 release. Zero
shipped-byte defects found. Lead finding: the "84 of 89 pinned parity
entries (585 of 595 assertions)" headline contradicts the release's own
scorecard fifteen lines above it ("CLOSED 77 wrong-answer pins"); the true
residual is 12 entries/28 assertions across three issues (#1204, #1206,
#1215), not 5/11. Also found: RQ-66-UNWATCHED's artifact status was never
updated after its work shipped (a second live instance of the #1250 gate
hole, making the release's own "7 implemented" count unsupported by the
tree); a leftover CHANGELOG [Unreleased] section states a stale emulation
floor (367751 vs live 367759); several claims.yaml comment/count slips with
no gate impact; and a "five frozen goldens" count that matches neither the
three files touched nor the seven pinned values changed.

Refs #1189 #1210 #1229 #1236

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L
@avrabe

avrabe commented Sep 10, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by #1253, which carries this review document verbatim plus the corrections it demanded.

Closing rather than merging both, because #1253 must land as one commit: the review's findings and the fixes for them belong in the same change, so that a reader of the tag sees the false claim and its correction together rather than having to reconstruct the order.

This review's findings are not dismissed — every one is acted on in #1253:

Thank you for the base-naming diagnosis in particular. "84 of 89" was a real measurement against a base nobody wrote down, and identifying that — rather than just flagging the number — is what made the fix correct everywhere instead of in one file.

@avrabe avrabe closed this Sep 10, 2026
@codecov

codecov Bot commented Sep 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant