Skip to content

fix: make each signal say only what the code actually checked - #7

Merged
bhyi4 merged 3 commits into
mainfrom
fix/signals-say-only-what-was-checked
Aug 25, 2026
Merged

fix: make each signal say only what the code actually checked#7
bhyi4 merged 3 commits into
mainfrom
fix/signals-say-only-what-was-checked

Conversation

@bhyi4

@bhyi4 bhyi4 commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

A first external user filed 16 items after using Yeoul on a real project (Windows/Codex, 2026-08-24).
Each was reproduced against the real binaries rather than taken as filed. Seven reproduce, and five
of those seven share one shape
: a machine-readable signal asserts something the code never tested.

the signal said what the code actually checked
exit code build-handoff exit 1 = failure the truth value of the last line, [ -z "$SUMMARY" ] && echo …
flag name arc-list --all = every arc every arc except _archive
guard tokens_used=0/200000 = measured a --tokens the caller never passed
message "install mirror-stack for sealing" command -v am
instruction the JOIN prompt's mandated first action nothing — the path was never confirmed runnable

That is the failure mode this project exists to catch, in its own wiring. Grouping by that class rather
than by the reporter's P0/P1 labels is why these land in one change.

The two that were worse or elsewhere than filed

build-handoff fails only when it succeeds. With a closed arc found, [ -z "$SUMMARY" ] is false,
&& short-circuits, and that truth value becomes the script's exit status.

state meaning exit before
closed arc found success 1
no closed arc warning 0

Both paths create dev-plan.md + TODO.md, so both are now 0; the missing arc stays a warning. Real
failures (no project, already exists) still exit non-zero with a cause. Filed as an MCP-wrapper problem;
it is one line in the script.

loop-guard's budget can be disarmed by omission. Same rounds, one optional argument apart:

--tokens omitted:  CONTINUE ×5,  status → tokens_used=0/1000     ← STOP:budget can never fire
--tokens=400    :  CONTINUE ×2 → STOP:budget (used=1200 > 1000)  ← fires on round 3

Unmeasured ticks are now counted and printed with a denominator (unmeasured=3/3-ticks); a reported
budget still stops the loop, and measured ticks gain no noise. Filed P2; it is guard integrity.

Also here

  • yeoul-new is atomic. A / in the topic aborted sed "s/<topic>/$TOPIC/g" after the project
    tree existed, so the retry hit "already exists". The topic is now substituted literally (bash pattern
    replacement — no delimiter to escape, no regex, and $(…)/backticks stay text), the tree is staged and
    moved into place in one step, and a failing arc-open rolls the project back and propagates its code.
  • JOIN prompts emit resolved absolute paths, and arc-open refuses to print an attach command it
    could not confirm is executable. setup/install.sh explicitly does not promise bin/ on PATH, and an
    MCP-only install has no ./bin at all — so every role's mandated first action failed.
  • A continuation contract in docs/BOOTSTRAP_PROMPT.md. The prompt named exactly one legitimate stop
    (Gate-1) and was silent on the rest, so an agent reasonably treated each scaffold step as the end of a
    request and the user had to type "continue" at every one. It now enumerates the stops and says
    everything between them continues as commentary. It governs sequencing, not judgment: sealing,
    PASS/KILL, convergence, graduation and publishing stay with the operator, and it is deliberately
    unenforced — an enforced "keep going" would be a mechanism for skipping gates.
  • The publish guard was red on a clean checkout. It walked the whole working tree, so the gitignored
    KNOWLEDGE_INDEX.md that every arc close appends to was reported as a "personalization leak" — about a
    file git will never publish. A red that does not mean what it says gets ignored. All three scans now run
    over git ls-files, print their denominator (✓ clean (41 files scanned)), and fail on a zero-file scan.

Evidence

tests/test_gates.sh 48/48, and the summary carries its denominator — "all gate tests passed" is also
what a run that collected zero checks prints, so an empty run now exits 1.

Each fix was reverted individually and its own checks confirmed to go red (6/6), with the revert
verified by hash change. No regression in the sealed gate numbers: evasive 28/28 caught, genuine-answer
false rejection 0/39, identical across both copies. mcp/tests/test_run_contract.py 8/8 on this base.

The first revert-to-red run reported 3 of 6 as vacuous. That was the control keying on the success
wording, which a failing line never prints — a test built so it could not win. Checks now carry item tags
that both branches print. Recorded because the wrong verdict came first.

Three defects of this same class were introduced while fixing it, and the instruments caught all three,
not review: a backtick inside a double-quoted message ran am as a command and leaked its status into six
seal paths; if ! cmd; then rc=$? captured the negation's status so a failing arc-open reported exit 0;
and the first publish-guard rewrite passed all three scans while inspecting zero files, because
python3 - <<PY makes the heredoc stdin so the piped file list never arrived.

Not claimed

  • Windows is unverified. No Windows environment here; the reporter's Windows path items (YL-07 and the
    Windows preflight) are neither confirmed nor disputed.
  • YL-01 reproduced only for /. Backslash, quotes, Unicode and & all passed before the fix (5/5), so
    the reporter's completion criterion is broader than the defect was.
  • Not addressed here (feature additions, not defects): a relay round-append tool with TURN ordering,
    long-topic duplication across 6 files, per-role model provenance, a common result envelope, and
    structural separation of finding/interpretation.
  • The arc↔claim linkage the report asks for is largely already implemented (arc-prereg plus verbatim
    kill-condition injection); the gap there is documentation, not mechanism.
  • 16 items from one user, one session. Treated as evidence of existence, not of frequency.

🤖 Generated with Claude Code

Mother Seara and others added 3 commits August 25, 2026 09:42
A first-use field report (2026-08-24, Windows/Codex) filed 16 items. Reproducing them against
the real binaries, five of the seven that reproduce share one shape: a machine-readable signal
asserts something the code never tested.

  exit code  build-handoff exited 1 on the SUCCESS path only. The last line was
             `[ -z "$SUMMARY" ] && echo ...`; with a closed arc found the test is false, `&&`
             short-circuits, and that truth value became the script's exit status. Callers could
             not tell success from failure. Both paths create dev-plan.md + TODO.md, so both are
             exit 0 and the missing arc stays a warning; real failures still exit non-zero.

  flag name  `arc-list --all` dropped `_archive` before testing the flag, so `--all` meant "every
             arc except the closed ones" — on a workspace whose only arc had been closed it
             printed nothing. Archived rows now list, stamped status=Archived.

  guard      loop-guard's `--tokens` defaulted to 0, so a caller that never passed it accumulated
             zero forever: STOP:budget could not fire while `status` printed "tokens_used=0/200000",
             a line shaped exactly like a measurement. Unmeasured ticks are now counted and
             reported with a denominator; a reported budget still stops the loop.

  message    the close printed "install mirror-stack for sealing" after testing `command -v am`.
             An MCP-only install has mirror-stack present and no `am` on PATH; the text sent those
             users to reinstall what they already had. It now names the condition tested.

  instruction the JOIN prompt told every role its first action was `bin/arc-attach "./projects/..."`
             — two relative paths, resolvable only from the repo root, while setup/install.sh
             explicitly does not promise bin/ on PATH. Every role's mandated first action failed.
             Both paths are resolved before they are printed, and arc-open refuses to emit a
             command it could not confirm is executable.

Also atomic scaffolding: a `/` in the topic aborted `sed "s/<topic>/$TOPIC/g"` after the project
tree had already been created, leaving a half-built project that made the retry fail with
"already exists". The topic is now substituted literally (bash pattern replacement — no delimiter
to escape, no regex, and command substitution stays text), the tree is staged and moved into place
in one step, and a failing arc-open rolls the project back and propagates its real exit code.

tests/test_gates.sh: 48/48, and the summary carries its denominator — "all gate tests passed" is
also what a run that collected zero checks prints, so an empty run now fails. Each new check was
confirmed to go red with its own fix reverted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A revert-to-red control keyed on the success wording can never match a failing line, because the
pass and fail branches print different prose. That produced a false KILL on three fixes that were
in fact non-vacuous. Both branches now carry the item tag, so the control keys on an identifier.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ard to what ships

Continuation contract (docs/BOOTSTRAP_PROMPT.md). The first external user had to type "continue"
at every scaffold step, and an arc was left open when they stepped away. The prompt named exactly
one legitimate stop (Gate-1) and said nothing about the rest, so an agent reasonably treated each
internal step as the end of a request. It now enumerates the stops — failed tool, gate refusal or
loop-guard STOP, reproducible blocker, authority not granted, a decision that changes the goal,
close/archive — and says everything between them continues as commentary.

The contract governs sequencing, not judgment. Sealing, PASS/KILL, convergence, graduation and
publishing stay with the operator, and "keep going" is never authority to decide one; convergence
and the verdict are read off role evidence by the relay, not by a fixed rule. Like the rest of the
prompt it is advisory, and deliberately unenforced — an enforced "keep going" would be a mechanism
for skipping gates. The enforcement runs the other way, in the scripts.

Publish guard (setup/pre-publish-check.sh). It walked the whole working tree, so the gitignored
KNOWLEDGE_INDEX.md that every arc close appends to made it report "personalization leaks found"
about a file git will never publish — red on a clean checkout, for content that cannot ship. A red
that does not mean what it says gets ignored, which is worse than no guard. All three scans now run
over `git ls-files`, with a walk fallback outside a work tree.

Scoping a scan is how a guard goes quietly blind, so the denominator is enforced and printed:
"✓ clean (41 files scanned)", and a scan over zero files fails the run. Verified by planting each
violation it claims to catch — Hangul, private absolute path, over-claim copy — in a tracked file
and confirming each one fires. The first attempt at this passed all three while inspecting nothing:
`python3 - <<PY` makes the heredoc stdin, so the piped file list never arrived. The positive control
caught it; the file list now goes through a temp file.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@bhyi4

bhyi4 commented Aug 25, 2026

Copy link
Copy Markdown
Contributor Author

What the green actually covers (read from the run log, not from the badges — a job being green is not the same as these checks running there):

job what ran
test (ubuntu-latest) 48/48 gate checks passed · ✓ clean (41 files scanned)
test (macos-latest) 48/48 gate checks passed · ✓ clean (41 files scanned)
mcp (ubuntu-latest) all _run contract tests passed: 8/8
mcp (windows-latest) all _run contract tests passed: 8/8

🔴 The bash suite does not run on Windows. The Windows job exercises the 8 _run contract checks and
nothing else, so the seven fixes in this PR — all of them in bin/ shell scripts — are not exercised on
Windows
, which is the platform the field report came from. Four green jobs should not be read as
"verified on Windows."

Also not covered: no real CP949 code-page run (the hostile-locale test works in the ASCII band), and the
atomic-scaffold and JOIN-prompt fixes are verified against POSIX path semantics only.

Reproducing locally: bash tests/test_gates.sh (48/48; an empty run exits 1),
python mcp/tests/test_run_contract.py (8/8 — it is a script, not a pytest suite; pytest on it errors
with fixture 'tmp' not found), and bash setup/pre-publish-check.sh (fails on a zero-file scan).

@bhyi4
bhyi4 merged commit 81d0e42 into main Aug 25, 2026
4 checks passed
@bhyi4
bhyi4 deleted the fix/signals-say-only-what-was-checked branch August 25, 2026 01:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant