fix: make each signal say only what the code actually checked - #7
Conversation
A first-use field report (2026-08-24, Windows/Codex) filed 16 items. Reproducing them against
the real binaries, five of the seven that reproduce share one shape: a machine-readable signal
asserts something the code never tested.
exit code build-handoff exited 1 on the SUCCESS path only. The last line was
`[ -z "$SUMMARY" ] && echo ...`; with a closed arc found the test is false, `&&`
short-circuits, and that truth value became the script's exit status. Callers could
not tell success from failure. Both paths create dev-plan.md + TODO.md, so both are
exit 0 and the missing arc stays a warning; real failures still exit non-zero.
flag name `arc-list --all` dropped `_archive` before testing the flag, so `--all` meant "every
arc except the closed ones" — on a workspace whose only arc had been closed it
printed nothing. Archived rows now list, stamped status=Archived.
guard loop-guard's `--tokens` defaulted to 0, so a caller that never passed it accumulated
zero forever: STOP:budget could not fire while `status` printed "tokens_used=0/200000",
a line shaped exactly like a measurement. Unmeasured ticks are now counted and
reported with a denominator; a reported budget still stops the loop.
message the close printed "install mirror-stack for sealing" after testing `command -v am`.
An MCP-only install has mirror-stack present and no `am` on PATH; the text sent those
users to reinstall what they already had. It now names the condition tested.
instruction the JOIN prompt told every role its first action was `bin/arc-attach "./projects/..."`
— two relative paths, resolvable only from the repo root, while setup/install.sh
explicitly does not promise bin/ on PATH. Every role's mandated first action failed.
Both paths are resolved before they are printed, and arc-open refuses to emit a
command it could not confirm is executable.
Also atomic scaffolding: a `/` in the topic aborted `sed "s/<topic>/$TOPIC/g"` after the project
tree had already been created, leaving a half-built project that made the retry fail with
"already exists". The topic is now substituted literally (bash pattern replacement — no delimiter
to escape, no regex, and command substitution stays text), the tree is staged and moved into place
in one step, and a failing arc-open rolls the project back and propagates its real exit code.
tests/test_gates.sh: 48/48, and the summary carries its denominator — "all gate tests passed" is
also what a run that collected zero checks prints, so an empty run now fails. Each new check was
confirmed to go red with its own fix reverted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A revert-to-red control keyed on the success wording can never match a failing line, because the pass and fail branches print different prose. That produced a false KILL on three fixes that were in fact non-vacuous. Both branches now carry the item tag, so the control keys on an identifier. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ard to what ships Continuation contract (docs/BOOTSTRAP_PROMPT.md). The first external user had to type "continue" at every scaffold step, and an arc was left open when they stepped away. The prompt named exactly one legitimate stop (Gate-1) and said nothing about the rest, so an agent reasonably treated each internal step as the end of a request. It now enumerates the stops — failed tool, gate refusal or loop-guard STOP, reproducible blocker, authority not granted, a decision that changes the goal, close/archive — and says everything between them continues as commentary. The contract governs sequencing, not judgment. Sealing, PASS/KILL, convergence, graduation and publishing stay with the operator, and "keep going" is never authority to decide one; convergence and the verdict are read off role evidence by the relay, not by a fixed rule. Like the rest of the prompt it is advisory, and deliberately unenforced — an enforced "keep going" would be a mechanism for skipping gates. The enforcement runs the other way, in the scripts. Publish guard (setup/pre-publish-check.sh). It walked the whole working tree, so the gitignored KNOWLEDGE_INDEX.md that every arc close appends to made it report "personalization leaks found" about a file git will never publish — red on a clean checkout, for content that cannot ship. A red that does not mean what it says gets ignored, which is worse than no guard. All three scans now run over `git ls-files`, with a walk fallback outside a work tree. Scoping a scan is how a guard goes quietly blind, so the denominator is enforced and printed: "✓ clean (41 files scanned)", and a scan over zero files fails the run. Verified by planting each violation it claims to catch — Hangul, private absolute path, over-claim copy — in a tracked file and confirming each one fires. The first attempt at this passed all three while inspecting nothing: `python3 - <<PY` makes the heredoc stdin, so the piped file list never arrived. The positive control caught it; the file list now goes through a temp file. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
What the green actually covers (read from the run log, not from the badges — a job being green is not the same as these checks running there):
🔴 The bash suite does not run on Windows. The Windows job exercises the 8 Also not covered: no real CP949 code-page run (the hostile-locale test works in the ASCII band), and the Reproducing locally: |
A first external user filed 16 items after using Yeoul on a real project (Windows/Codex, 2026-08-24).
Each was reproduced against the real binaries rather than taken as filed. Seven reproduce, and five
of those seven share one shape: a machine-readable signal asserts something the code never tested.
build-handoffexit 1 = failure[ -z "$SUMMARY" ] && echo …arc-list --all= every arc_archivetokens_used=0/200000= measured--tokensthe caller never passedcommand -v amThat is the failure mode this project exists to catch, in its own wiring. Grouping by that class rather
than by the reporter's P0/P1 labels is why these land in one change.
The two that were worse or elsewhere than filed
build-handofffails only when it succeeds. With a closed arc found,[ -z "$SUMMARY" ]is false,&&short-circuits, and that truth value becomes the script's exit status.Both paths create
dev-plan.md+TODO.md, so both are now 0; the missing arc stays a warning. Realfailures (no project, already exists) still exit non-zero with a cause. Filed as an MCP-wrapper problem;
it is one line in the script.
loop-guard's budget can be disarmed by omission. Same rounds, one optional argument apart:Unmeasured ticks are now counted and printed with a denominator (
unmeasured=3/3-ticks); a reportedbudget still stops the loop, and measured ticks gain no noise. Filed P2; it is guard integrity.
Also here
yeoul-newis atomic. A/in the topic abortedsed "s/<topic>/$TOPIC/g"after the projecttree existed, so the retry hit "already exists". The topic is now substituted literally (bash pattern
replacement — no delimiter to escape, no regex, and
$(…)/backticks stay text), the tree is staged andmoved into place in one step, and a failing
arc-openrolls the project back and propagates its code.arc-openrefuses to print an attach command itcould not confirm is executable.
setup/install.shexplicitly does not promisebin/on PATH, and anMCP-only install has no
./binat all — so every role's mandated first action failed.docs/BOOTSTRAP_PROMPT.md. The prompt named exactly one legitimate stop(Gate-1) and was silent on the rest, so an agent reasonably treated each scaffold step as the end of a
request and the user had to type "continue" at every one. It now enumerates the stops and says
everything between them continues as commentary. It governs sequencing, not judgment: sealing,
PASS/KILL, convergence, graduation and publishing stay with the operator, and it is deliberately
unenforced — an enforced "keep going" would be a mechanism for skipping gates.
KNOWLEDGE_INDEX.mdthat every arc close appends to was reported as a "personalization leak" — about afile git will never publish. A red that does not mean what it says gets ignored. All three scans now run
over
git ls-files, print their denominator (✓ clean (41 files scanned)), and fail on a zero-file scan.Evidence
tests/test_gates.sh48/48, and the summary carries its denominator — "all gate tests passed" is alsowhat a run that collected zero checks prints, so an empty run now exits 1.
Each fix was reverted individually and its own checks confirmed to go red (6/6), with the revert
verified by hash change. No regression in the sealed gate numbers: evasive 28/28 caught, genuine-answer
false rejection 0/39, identical across both copies.
mcp/tests/test_run_contract.py8/8 on this base.The first revert-to-red run reported 3 of 6 as vacuous. That was the control keying on the success
wording, which a failing line never prints — a test built so it could not win. Checks now carry item tags
that both branches print. Recorded because the wrong verdict came first.
Three defects of this same class were introduced while fixing it, and the instruments caught all three,
not review: a backtick inside a double-quoted message ran
amas a command and leaked its status into sixseal paths;
if ! cmd; then rc=$?captured the negation's status so a failingarc-openreported exit 0;and the first publish-guard rewrite passed all three scans while inspecting zero files, because
python3 - <<PYmakes the heredoc stdin so the piped file list never arrived.Not claimed
Windows preflight) are neither confirmed nor disputed.
/. Backslash, quotes, Unicode and&all passed before the fix (5/5), sothe reporter's completion criterion is broader than the defect was.
long-topic duplication across 6 files, per-role model provenance, a common result envelope, and
structural separation of finding/interpretation.
arc-preregplus verbatimkill-condition injection); the gap there is documentation, not mechanism.
🤖 Generated with Claude Code