docs: publish the conformance report - #180
Conversation
The report has to be derived from a run rather than written by hand, or it becomes a claim the moment a case is added or renamed. The generator runs both vitest projects, takes the case list off the JSON reporter, and writes nothing unless every case passed on both targets, so a behaviour appears on the page only because it was compared. A generator nobody runs leaves the same stale file, so `check:conformance` regenerates and compares instead of writing, and a CI job fails when the committed report no longer matches the suite.
A reader cannot currently see which behaviours are checked against real socket.io. docs/conformance.md lists them case by case, each linking to the test that pins it, and states what the dual run is so the list can be read as evidence rather than a table of contents. The page also carries what belongs with that claim rather than beside it: the surface no case reaches yet, which is where the first approachable issues come from; the procedure for adding a case, without which a gap is not actionable; the version matrix CI actually runs, so the supported range is a fact and not a claim; and a summary of 0019, so deciding whether to take an upgrade does not need a decision record opened.
|
Warning Review limit reached
Next review available in: 32 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (8)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
github-actions
Bot
added
the
documentation
1 participant
Summary
docs/conformance.mdlists every behaviour the dual run checks against real socket.io, case by case, each linking to the test that pins it. The list is generated byscripts/conformance-report.mjs, which runs both vitest projects and writes nothing unless every case passed on both targets, so a behaviour is on the page because it was compared rather than because someone claimed it.pnpm check:conformanceregenerates and compares instead of writing, and a CI job fails when the committed report has drifted from the suite.The page also carries the surface no case reaches yet, which is where the first approachable issues come from (#117), the procedure for adding a case, the version matrix CI actually runs, and a summary of 0019. It is linked from the README,
docs/README.md,CONTRIBUTING.md, andAGENTS.md.Related issue
Closes #50
Checklist