Correct documentation claims that the shipped revision made false - #5
Merged
Conversation
|
Warning Review limit reached
Next review available in: 38 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (10)
📝 WalkthroughWalkthroughDocumentation now reflects IR 0.4, expanded Python support, frozen-image migration, compatibility measurements, current status, and 412 tests. New tests validate documentation consistency against the runtime and CLI definitions. ChangesDocumentation alignment
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6e254ae060
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@ARCHITECTURE.md`:
- Line 54: Update the version and IR scope described in the architecture
document so the introductory scope matches the current implementation referenced
by the blocks entry, or explicitly mark the older Version 0.2 and IR 0.3
description as historical.
In `@tests/test_documentation_consistency.py`:
- Around line 309-325: Update the documentation consistency test’s DOCS
inventory to include every Markdown file covered by the repository, not just the
current ten entries. Expand CURRENT_IR to detect both “current” preceding the IR
revision and “IR <revision>” preceding or followed by wording that marks it
current, while retaining sentence-boundary limits to avoid historical
references.
- Around line 458-467: Update
test_not_working_python_version_claim_matches_the_allowlist to extract the
Python versions documented in the NOT WORKING section, then compare that set for
exact equality with abi.VERIFIED_PYTHON_VERSIONS so both missing and unexpected
versions fail the test.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: eeac43ca-fcd8-4eae-9757-fcff3d061458
📒 Files selected for processing (10)
ARCHITECTURE.mdFORMAT.mdLIMITATIONS.mdPERFORMANCE.mdPORTABILITY.mdREADME.mdSTATUS.mddocs/RELEASE_NOTES_0.5.0a1.mddocs/TESTING.mdtests/test_documentation_consistency.py
A documentation accuracy pass over all 16 markdown files. Every change below is a claim the tree contradicts, not a rewording. Verified against the code, the raw result JSON, and the compiler rather than against other prose. STATUS.md carried the most, and several contradicted itself: - "No cross-platform proof has been run for IR 0.4" sat fifteen lines below a bullet in the same section citing that proof's passing run (30592158078). Removed; the section's remaining bullets already carry the evidence. - The IR 0.4 milestones were filed under IN PROGRESS with all four marked complete. Retitled to record them as shipped, and IN PROGRESS now lists work that is actually outstanding. - NOT WORKING listed closures, classes/instances, `try/except`, and variadic parameters. All four compile; they are IR 0.4 features announced in this same file. A reader deciding whether their program could run was told no when the answer was yes. - NOT WORKING listed "Python versions other than 3.12.13", contradicting the document's own header and the release's headline feature. The allowlist is 3.12.13 and 3.13.14. - The corpus bullet published the IR 0.3 rate (35/50, 70.0%) as current when IR 0.4 measures 40/50 (80.0%), and attributed it to Linux when that run was Windows x86_64. - UNVERIFIED claimed the corpus had never run on Windows. It has; that run is the IR 0.4 column in COMPATIBILITY.md. - The public command inventory omitted `plan-upgrade`, `inspect-upgrade`, and `verify-upgrade` -- the three commands this release is named for. - Quoted 159x slowdown, a 0.1.1.dev0/IR 0.2 historical figure that PERFORMANCE.md explicitly forbids quoting for the current release. Current is 144x on 3.12.13 and 206x on 3.13.14. LIMITATIONS.md kept its own copy of the same stale support list, and six entries were wrong: classes and instances, closures, `nonlocal`, positional-only, keyword-only, and variadic parameters all compile, and `try/except` is listed as unsupported while being an IR 0.4 feature. The supported list above it was corrected to match. Its corpus paragraph repeated the Linux-only claim. Elsewhere: - PORTABILITY.md said "Current IR 0.3" nine lines above "current IR 0.4". - PERFORMANCE.md said current development uses IR 0.3, and told readers not to quote historical figures for 0.4.0a1 rather than 0.5.0a1. - FORMAT.md attributed `default_count` to IR 0.3 in a spec for shipping 0.4. - ARCHITECTURE.md described the frame block stack as holding `try/finally` handlers; `vm.py` pushes `SETUP_EXCEPT` onto it too. - README.md listed classes and closures as unsupported, said the corpus had never been regenerated off Linux, and documented none of the three migration commands -- the flagship 0.5.0a1 feature had no entry in the front door. Added a walkthrough for the plan/inspect/verify/resume --upgrade path. Four regression tests lock these down, following this file's existing rule that documentation drift is a test failure rather than a review comment: no document may call a superseded IR revision current; STATUS.md's command inventory must equal the parser's subcommands; neither STATUS.md nor LIMITATIONS.md may name syntax the compiler accepts; and the version entry must name the whole verified Python allowlist. Each fails against the parent: the support guard alone reports six wrong entries in the old LIMITATIONS.md. Checked what is already right rather than assuming: COMPATIBILITY.md's rates, its full per-gate table, all eight rejected program names, and every test name cited in docs/TESTING.md all verify against the raw JSON and the tree. Published counts 408 -> 412. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
byte271
force-pushed
the
docs/accuracy-pass
branch
from
August 1, 2026 16:42
6e254ae to
c78c3a0
Compare
Four findings from the Codex and CodeRabbit reviews on #5. Each reproduced before being changed. The migration walkthrough this branch added to README.md said "None of these commands executes the program" directly beneath a block listing four commands, the last of which is `resume --upgrade`. `_resume` calls `vm.run()` (`continuum/cli.py:1126`), so that command executes the program and produces its ordinary side effects. The sentence was wrong about the feature the section exists to document. Now says the first three do not execute and `resume --upgrade` does, with the no-write guarantee stated for all four. ARCHITECTURE.md's scope paragraph still read "Version 0.2" and "IR 0.3 places them after statements" -- the same rot this branch fixes elsewhere, in a file whose block-stack line the branch had already corrected to IR 0.4. Missed on the first pass. The two guards were weaker than they looked: - `SupersededRevisionTests` listed ten markdown files by hand while the tree has sixteen, so a stale claim in an unlisted file passed. The inventory is now discovered by glob, with a test asserting the inventory is complete: a new document cannot silently opt out. The pattern also matched only "current ... IR X"; it now matches "IR X is current" too. Verified by appending "IR 0.3 is current." to CONTRIBUTING.md -- previously unlisted, previously unmatched phrasing -- which the widened guard fails on. - The Python-allowlist check tested containment, so a section naming every verified version *and* an unverified one still passed. That is the more dangerous direction, since it advertises support that does not exist. Now compares sets both ways. Verified by inserting 3.11.9, which the old check accepted and the new one reports as unexpected. Published counts 412 -> 413. 413 tests, 0 failures, 5 skips on CPython 3.12.13 and 3.13.14. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A documentation accuracy pass over all 16 markdown files. Every change is a claim the tree contradicts, not a rewording. Each was verified against the code, the raw result JSON, or the compiler — never against other prose.
STATUS.md contradicted itself three times
try/except, variadic paramsplan-upgrade,inspect-upgrade,verify-upgradePERFORMANCE.mdexplicitly forbids quoting; current is 144×/206×LIMITATIONS.md kept its own copy of the same stale list
Six entries were wrong. Probed against the compiler rather than assumed:
nonlocalglobalgenuinely does not)try/exceptThe supported list above it was corrected to match, and its corpus paragraph repeated the Linux-only claim.
Elsewhere
default_countto IR 0.3 in a spec for the shipping IR 0.4.try/finallyhandlers;vm.pypushesSETUP_EXCEPTonto it too.plan-upgrade→inspect-upgrade→verify-upgrade→resume --upgradepath.Guards
Four regression tests, following this file's existing rule that documentation drift is a test failure rather than a review comment:
Each fails against the parent commit. Guard 3 is deliberately scoped to both documents: the first version checked only STATUS.md, which is exactly why LIMITATIONS.md kept the same rot — against the pre-fix file it now reports six failures.
What was already right
Checked rather than assumed, and left alone: COMPATIBILITY.md's three published rates (64.0/70.0/80.0%) and its full per-gate table (34/37/42, 33/36/41, 32/35/40, 32/35/40) reproduce exactly from the raw JSON; all eight rejected program names and both out-of-scope programs match; all 16 test names cited in
docs/TESTING.mdresolve, and its "three external-auditor tests" claim is exactly three.412 tests, 0 failures, 5 skips on CPython 3.12.13 and 3.13.14. Published counts 408 → 412.
Summary by CodeRabbit
Documentation
Tests