Skip to content

clikae v0.26.0 — the gates were never asked to fail

Choose a tag to compare

@cverorg cverorg released this 15 Aug 19:24
· 82 commits to main since this release

The last two releases audited the docs. This one audits the audit.

v0.24.0 and v0.25.0 were both the same finding: a document that names a thing is
what an auditor reads instead of the code, so a stale one hides the gap it
would otherwise expose. The answer was gates — a suite, a doc contract, a CI
run. This release asks the next question, which is the uncomfortable one:

has any of it ever been seen to fail?

A green suite says the code behaves on the inputs someone thought to write. It
does not say a guard exists.

A test that never ran the function it named

run bash -c 'wake_ask_once claude work < /dev/null'

bash -c forks, and shell functions do not cross a fork. Measured: inside that
subshell both wake_ask_once and the confirm() stub defined two lines above
report NOT-VISIBLE.

So what the test asserted was that a command not found message does not
contain the word ASKED. True however wake_ask_once behaves — including with
its TTY check deleted, i.e. asking on every headless launch and then typing into
a live session, which is the exact harm the test names in its own comment. It
passed for two months.

Now called in the shell that holds the stub, and proven to fire: remove
[ -t 0 ] && [ -t 1 ] || return 0 and it goes red with ASKED in the output. The
other 25 bash -c sites in the suite were swept for the same shape; they are
all real subprocesses.

scripts/mutate.sh

The only evidence a guard is load-bearing is watching the suite go red when you
take it away. So, four mutations against docs/memory.md §4 — the promises
clikae makes about the human's data:

break this tests that caught it
share without ever opting in memory share: fans a tank's memory…
make a solo tank stop being solo 🔴 a SOLO tank never joins…
silence the cross-account note carry_notice_once prints the note…
turn seed-by-copy into a move memory share: … (seeded by copy)
4 guard(s) proven, 0 hollow, 0 mutation(s) that never applied

Not wired into scripts/test.sh — it copies the repo per mutation and costs
minutes. It is the ruler for a question you ask deliberately, and it is checked
in because a reading with no ruler behind it is an anecdote.

That third column exists because of its own first run, which reported three
hollow guards
. All three were the ruler: tank_is_solo lives in
profile_store.sh and the mutation edited tank.sh; notice.sh's function is
carry_notice_once, not the name that was guessed. Nothing was mutated, and the
tests were green for the most boring reason available. A mutation that did not
apply is indistinguishable from a guard that works, so every row now checksums
its target and reports ⛔ rather than a verdict.

(The reason those expressions silently did nothing: perl needs balanced braces
inside s{…}{…}, and a shell function's replacement text almost always carries
an unmatched {. Delimiter is ! now, with the reason written down.)

The doc gate had the disease it was built to cure

Two holes, both of which left it green on exactly the drift it exists to catch.

Its scope was a list written from memory. Candidate names were extracted with
a hand-written prefix list — clikae_|tmux_|soul_|wake_|…. Measured: 20 real
functions are named in the docs and were invisible to it
, including
tank_is_solo, next_tank, history_log, load_adapter and five limit_*.
Rename one in every source file, leave the docs alone: gate stays green. A gate
whose scope is an enumeration is silent on precisely the entries its author
forgot — and forgetting is the failure it was built for. Now unioned with "any
backticked all-lowercase token containing an underscore", which needs no list.

It read the working directory as the source. A sed -i.bak during that very
experiment left lib/commands/*.sh.bak on disk, and the gate counted them as
repo source in both directions at once. The renamed function still "existed",
because the backup held its old definition. And the backup counted as a caller,
so the docstring was asked to list burn.sh.bak. An editor swapfile or a merge
.orig does the same. It reads git ls-files now.

Reading the real file list turned up two more things. docs/proposals/ is out of
scope with a reason — a proposal names the function it is asking for, and that
function does not exist yet; that is what a proposal is. And the first draft used
mapfile, which macOS's stock bash 3.2 does not have: the shell this gate
actually runs on.

clikae memory status --json

clikae memory status --json | jq -r '.[] | select(.dispatchable) | .tank'

So an agent can ask which tanks it may use instead of parsing prose.
dispatchable is false for a solo tank, and false for the impossible
solo-and-shared state — there the wiring does not match the label, so nothing
about that tank is safe to reason about.

Four entries were filed under the wrong release

conduct's read-only enforcement, burn's scoped write grant, conduct legs no
longer leaving transcripts, and the Rule 8 correction were all written into the
[0.25.0] section after the v0.25.0 tag. Anyone running 0.25.0 — which is
what Homebrew serves — would have read that changelog and believed their
conduct legs cannot write. They can.

Moved to where they actually ship, labelled with when they were written. It is
the same defect the last two releases have been auditing out of the docs,
committed in the file that describes the audit.

scripts/test.sh: 765 passing.