feat(skills): a new oracle may not introduce a fresh mirror - #186
Merged
Conversation
…29.0) Closes #180. A test that hardcodes a value the implementation also defines — a base address, an ABI convention, a wire-format constant, a version — is a finding. Derive it at test time, or pin it in a ledger that fails on divergence. The reporter's measurement corrected the obvious hypothesis, and the skill text says so rather than stating the tidy version. The fear is a silent pass: the mirror drifts, the oracle tests the wrong thing. Measured in synth, where 38 harnesses mirror one compiler constant, moving it made the oracle fail LOUDLY, exit 1. The real damage is that the failure is detected but NOT ATTRIBUTED: all 38 go red at once, each naming its own subject (FAIL nested / FAIL br_table / FAIL spill) and none naming the constant they share. You debug 38 oracles instead of reading one line — and the cost scales with how good the suite is. Hence the ordering: derive > pin > name-the-assumption-in-the-failure-message. That last is the cheap 80%: one string per harness turns "38 red oracles" into "38 red oracles that all name the same constant." Also carries the reporter's second finding, which is a gate-potency instance: negative-control fixtures banked with canonical register assignments the real selector never emits were not wrong, they were UNREACHABLE — and looked like coverage. oracle-gate-a-change new step 3b, before "Gate the diff" pulseengine-feature-loop step 4 oracle discipline report-tool-friction a mirror you were FORCED to create is a tool gap (the compiler cannot print its own layout constants), so fixing it in the test only moves the debt Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G5qpB7zvxLvdrvn5YRyj4R
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #180.
A test that hardcodes a value the implementation also defines — a base address, an ABI convention, a wire-format constant, a version — is a finding. Derive it at test time, or pin it in a ledger that fails on divergence.
The skill text keeps your correction, not the tidy version
The obvious hypothesis is a silent pass: the mirror drifts and the oracle tests the wrong thing. Your measurement killed that — moving synth's compiler constant made the oracle fail loudly, exit 1.
So the skill states the real damage: the failure is detected but not attributed. All 38 harnesses go red at once, each reporting its own subject —
FAIL nested,FAIL br_table,FAIL spill— and none saying "a constant I copy from the implementation moved." You debug 38 oracles instead of reading one line, and the cost scales with how good your test suite is.That gives the preference order, including your cheap-80% clause:
Copying silently is the one option that isn't acceptable.
Where it landed
oracle-gate-a-changepulseengine-feature-loopreport-tool-frictionYour second finding is in there too
The negative-control fixtures banked with canonical register assignments the real selector never emits — "not wrong, unreachable, which is worse, because it looked like coverage" — is carried as a note pointing at
gate-potency. A control you have not watched fail is a hypothesis.Plugin
0.28.0 → 0.29.0.🤖 Generated with Claude Code
https://claude.ai/code/session_01G5qpB7zvxLvdrvn5YRyj4R