fix(testing): the E2E board published 12 divergences that do not exist - #423
Merged
Conversation
They were not product defects. All twelve had one shape — MCP FORBIDDEN while CLI and REST succeeded — and the message names the cause: Native: []. OPA: [OPA_ERROR: ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING_FLAG: A dynamic import callback was invoked without --experimental-vm-modules] Native ABAC raised NO violation. The OPA engine could not load its wasm for want of a Node flag, and dispatch requires both engines to allow, so every `tools/call` fail-closed. The capture that produced them was made on 2026-08-03 at 09:56 by a run launched WITHOUT that flag; `npm run test:exploration` does pass it. Re-run through the npm script, the same suite reports ZERO findings — measured here, not argued. So the defect is mine, in what this board did with its input. It rendered a stale capture with no indication of age, and twelve non-existent divergences read as observed fact. A derived artifact exists to make exactly that impossible. `coverage.json` carries no timestamp of its own, so the page now stamps the file's mtime and names the command that produces it. A reader can see the measurement predates the tree it describes. Four hypotheses died on the way and none was the answer: classification drift between the rego and the TS map (all twelve tools ARE classified), a role the policy does not know (`reader` needs no named role for read tools), a missing scope (none of the twelve declares one), and a harness that never authenticates (it starts the server with `allowNoAuth`). The message was in `howto-capture.json` the whole time; reading it first would have cost four fewer steps. Not changed here, and worth its own look: the exploration suite is red on this machine for an unrelated and ALREADY RECORDED reason — the how-to capture embeds rule ids that differ by environment (`DRIFT-01` on the runner, `INH-03` here), documented in GT-650. Regenerating locally fixes one half and breaks the other, so the files are left untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
📊 Bilingual Coverage ImpactPR Changes
Repository Coverage
✅ Good: All EN changes have ES counterparts. Generated by GitHub Actions |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
They were not product defects. All twelve had one shape — MCP
FORBIDDENwhile CLI and REST succeeded — and the message names the cause:Native ABAC raised no violation. The OPA engine could not load its wasm for want of a Node flag, and dispatch requires both engines to allow, so every
tools/callfail-closed. The capture that produced them was made on 2026-08-03 at 09:56 by a run launched without that flag;npm run test:explorationdoes pass it. Re-run through the npm script, the same suite reports zero findings — measured, not argued.The defect is in this board, not in the product
It rendered a stale capture with no indication of age, and twelve non-existent divergences read as observed fact. A derived artifact exists to make exactly that impossible.
coverage.jsoncarries no timestamp of its own, so the page now stamps the file's mtime and names the command that produces it. A reader can see the measurement predates the tree it describes.Four hypotheses died on the way, and none was the answer
read_toolsreaderneeds no named role — read tools allow any rolescoperequiredScoperesolves toreadallowNoAuth: true→ READER_CONTEXTThe message was in
howto-capture.jsonthe whole time. Reading it first would have cost four fewer steps.Not changed here
The exploration suite is red on this machine for an unrelated and already recorded reason: the how-to capture embeds rule ids that differ by environment (
DRIFT-01on the runner,INH-03here), documented inGT-650. Regenerating locally fixes one half and breaks the other, so those files are left untouched.ci-runner.mjs governancegenerate-e2e-scenario-board --checkgitleaks dir .🤖 Generated with Claude Code