Skip to content

[AUTOMATED] fix(repipe): the arena brief told testers half the CLI did not exist - #439

Merged
mahaloz merged 1 commit into
mainfrom
fix/repipe-arena-brief-derives-cli-surface
Sep 6, 2026
Merged

[AUTOMATED] fix(repipe): the arena brief told testers half the CLI did not exist#439
mahaloz merged 1 commit into
mainfrom
fix/repipe-arena-brief-derives-cli-surface

Conversation

@mahaloz

@mahaloz mahaloz commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

The problem

Every RE-friction tester gets an AGENTS.md describing the tool it is being
measured against. That brief froze kuna's subcommand list in prose, and the
prose rotted. Round 4's arenas, built minutes ago from main:

$ grep -A2 'lists its' .kuna-repipe/arena/4/*/AGENTS.md | head -3
`kuna --help` lists its nine subcommands: `decompile`, `decompile-all`, `decompile-project`,
`functions`, `test`, `catalog`, `modes`, `specs`, `fid`.

$ grep 'There is no' .kuna-repipe/arena/4/*/AGENTS.md
- There is no `xrefs`, `strings`, `disassemble`, `rename`, `retype` or server subcommand, [...]

Against the binary those same arenas ship:

$ decompiler/target/release/kuna --help 2>&1 | head -1
usage: kuna <decompile|decompile-all|decompile-project|decompile-graph|functions|disassemble|read|xrefs|strings|unpack|docs|test|catalog|modes|specs|fid> ...

Sixteen, not nine. Three of the subcommands the brief says do not exist do.
Two more of its five "facts already verified" are also false now — kuna decompile <bin> <fn> --json exits 0 and emits JSON rather than exiting 2, and
functions --json records carry size and object_location.

The fix

  • Read the subcommand list out of kuna --help at arena-build time, so the
    brief can only ever be as wrong as the binary in the arena.
  • Drop the per-fact block. Every entry in it was a snapshot of one round's CLI;
    what replaces it is the part that does not change (no server mode, no
    persistence, cold load per call, an empty result is not proof of an empty
    binary) plus a pointer to kuna docs and an instruction to check
    kuna <subcommand> --help before concluding something is missing.
  • kuna --help writes its usage line to stderr with rc=0, so the parser reads
    both streams; failing to find the line raises rather than shipping a brief
    with a blank list.

The tests

Rebuilt all nine round-4 arenas through the changed path: workspace build --force and workspace check both report zero violations on each, and each
brief now names all sixteen subcommands.

🤖 Generated with Claude Code

…d not exist

The tool protocol handed to every tester froze kuna's subcommand list in prose.
Four of its five "facts already verified" were false by round 4: it named nine
subcommands when there are sixteen, said `decompile --json` exits 2 when it
exits 0, said `functions --json` records carry no size when they do, and said
there is no `xrefs`, `strings` or `disassemble` subcommand when all three ship.

Derive the list from `kuna --help` on the binary the arena actually ships, and
replace the per-fact block with things that stay true (no persistence, cold load
per invocation, an empty result is not proof) plus a pointer to `kuna docs`.
A brief can now only ever be as wrong as the binary in the arena.

`kuna --help` writes its usage to stderr with rc=0, so the parser reads both
streams.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@mahaloz
mahaloz merged commit 798f01e into main Sep 6, 2026
9 checks passed
@mahaloz
mahaloz deleted the fix/repipe-arena-brief-derives-cli-surface branch September 6, 2026 03:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant