Repository navigation
v1.9.0 — No empty PASS, define mistakes named
Added
-
sideeye completions zsh|bash|fishprints a completion script built from the usage lines (#712;src/cli.zig,docs/cli.md). It offers exactly the commands, a command's positional argument (a case file, a command name, a shell), and exactly the flags its usage lines list (explore's and preflight's two lines together, so--configand--stateare both offered), with a flag's value completed as a file where the usage names a path and as its listed words where it lists them (--observe). The bash script runs on bash 3.2, macOS's/bin/bash, quotes a file name with a space, gives a directory its slash (except one named after a:,=or@in the word, which completes without it), and completes a path holding:,=or@.spike/check-completions.pyreads the usage lines on its own and asks each shell what the script offers at every position; acceptance runs it on Linux with all three shells installed, the macOS job with/bin/bashand zsh. A Homebrew formula can take the scripts withgenerate_completions_from_executable; the tap is not changed here. -
On a terminal the verdict word is coloured — PASS green, FAIL red, UNKNOWN yellow, SETUP ERROR magenta — and nowhere else (#712;
src/report.zig,docs/cli.md). Only standard output is asked, so a report redirected into a file from a terminal carries no escape sequence of Sideeye's own;NO_COLORset to anything non-empty, orTERM=dumb, turns it off on a terminal too. The JSON report, the MCP server and every captured run (CI, acceptance) are unchanged. A CI that gives its steps a terminal and greps the text report should setNO_COLOR=1. -
The README's first screen shows a trimmed real
sideeye demoFAIL report, and CI holds every line of it to a fresh demo (#714;spike/check-readme-demo.py). Eleven lines under the opening paragraph, each one the demo prints, in its order, with the scratch directory shown as…; the macOS job's demo step and acceptance check 5 run the demo and compare, so a change to the wording of a line it shows turns CI red until the README follows. The one named exception to "never a silent PASS" — a directory a recordedrenamemoved in from outside the judged tree — went from two sentences to a clause of the promise's own sentence, with its explanation moved toDESIGN.md's known constraints, andspike/check-readme-shape.shnow holds the exception to that sentence (ADR 0032). -
docs/found.mdlists every report inspike/upstream-reports.tsv— the reports this project has filed on GitHub — with the tool, a link and the furthest state the ledgers record; generated, and compared in CI (#713;spike/outcome-funnel.py, ADR 0070). 35 reports today: 5 fixed upstream and re-measured against the fix, 1 fixed upstream, 28 filed, 1 withdrawn. The page is rendered fromspike/upstream-reports.tsvandspike/outcome-funnel.tsvby the same--write-docthat renders the funnel's counts, and--check-docnow compares both pages. The README stops quoting a count of reports (it said 27) and links the page;spike/upstream-report-template.mddoes the same.checkalso refuses a report the two ledgers disagree on whether it was withdrawn. -
The four targets #684 named, re-measured past the calls that change nothing: fish PASS, dotdrop and firewalld FAIL with no content lost, vim at its next wall (
spike/dogfood/2026-10-06-read-only-684/;docs/target-classes.md,spike/dogfood/RUNS.md,spike/outcome-funnel.tsv). A strace-only survey written before the engine changed predicted each answer: vim writes the file's ACL back after saving and now stops onsetxattr, in all three observation modes; fish's universal variables PASS 6/6; dotdropinstall -fFAILs 2/15 andfirewall-offline-cmdFAILs 1/6, each truncating a file before writing it, with the old bytes kept in its own backup (.dotdropbak,.old). Neither FAIL is reported upstream. -
Thirty tools whose state git does not hold, in six rounds of five on the released v1.8.0: 22 FAIL, 8 PASS, and four reports upstream (
spike/dogfood/2026-10-05-user-data-2/;docs/target-classes.md,spike/dogfood/RUNS.md,spike/outcome-funnel.tsv,spike/upstream-reports.tsv,spike/unknown-rate/b2-exclusions.txt). 359 names were screened, none at a project already reported to, and 57 went through the entry gate. Twenty-one FAILs are the truncating open and the kill before the write. SolveSpace'ssolvespace-cli regenerateempties the sketch and still reports success, PyMOL'ssaveempties the session, WP-CLI'sconfig setemptieswp-config.php, and ffsubsync's--overwrite-inputempties the subtitle; they are reported as solvespace/solvespace#1783, schrodinger/pymol-open-source#520, wp-cli/config-command#233 and smacke/ffsubsync#240. Fourteen of the thirty are static images named bare, and every one reached a verdict through v1.8.0's next step to--observe supervised. aliyun-cli, a static image whose child is dynamic, did not: the page's path ends atoracle_missed_operationand then at syscalls with no mode named, the shape lefthook met on v1.7.0. Two screens missed:fresh.shdid not read cohort 4's rejected candidates (ffsubsync), and uv was screened under a spelling no ledger carries; both left the slate with their explores kept, and goenv -wtook uv's place. The B2 exclusions now name each tool by the trixie package that ships it, looked up rather than recalled. -
sideeye preflight --config sideeye.tomlreads the toml as explore reads it, and refuses the define mistakes it can see before any crash world, in explore's words (#704, ADR 0094). Preflight refused--config("sideeye explore --configanswers strictly more"), so a define moved into a file — the README's next step after preflight — could no longer be asked the quick question, and a check that could not start or a marker the operation never printed surfaced only from an exploration. The refusal is gone; the toml goes through the same phases explore's does, so the detectors already there apply as they stand: the parser, thecwd,apparatusandscratchchecks, astatethat is not a directory, a command that cannot be started, a marker the recording never printed, no crash point, and thecwda toml's define needed. Two of explore's refusals are added to preflight, at the point explore reaches them: with a declared check and a crash point, a check with no command in it is refused--check is empty, and one against an initial state with nothing to corruptchecker_not_falsified— each one function,refuse.checkerArgvandrefuse.refuseNothingToCorrupt, now called from both.docs/cli.mdlists what is refused, one mistake per run as in explore; what only a crash world shows is not on it. Nothing the toml declares is run: the report addsmarker,checkerandrecoverylines saying what preflight did with each, and a recovery that cannot be started is not refused here, as it moves no verdict in explore (ADR 0072). Thenextline issideeye explore --config '<the toml>', quoted for /bin/sh and named in the directory its relative paths were resolved against, so a toml that is a symlink keeps its own side; a toml read from/dev/stdin, from a shell's<(…), or through a link into/devor/procgets a line saying to save it first, and one whose name cannot be printed as typed a line saying to copy it to a file with a plain name.--twiceworks on a toml and names[world] state. With the flags, preflight accepts and refuses what it did and its accepted report is byte for byte what it was; it still refuses--check,--markerand--recoveryby name, reworded to point at--config, and define flags beside--configget explore's "mutually exclusive" refusal. The synopsis has apreflight --configline, and #273's check covers it.spike/acceptance.shcheck 6c runs one toml per listed mistake through both commands and compares the exit code, the refusal and its detail, with the check counting its own starts; against the parent commit with only the--configrefusal removed, twelve of its legs fail, and #273's check with them.
Changed
-
A string-form command spelled for a shell runs as written, and is told so (#706, ADR 0095;
src/config.zig,src/report.zig,src/main.zig,src/refuse.zig,src/mcp.zig,docs/cli.md,docs/report-schema.md,docs/contract-freeze.md). A string-form command is split on spaces and never run by a shell, sooperation = "./kva put 'hello world'"handed the program two arguments with the quotes in them,"… && echo ok"handed it&&— and both PASSed with nothing said. Now a setup, operation, check or recovery command whose string holds a quote, or a whole word a shell would act on (&&,|,;, a redirection, a word beginning$or~/), earns one sentence naming the command and how Sideeye reads it, with the argv form that groups what quotes meant — none where a shell would have expanded a$or backquote inside them. A toml command value cut at an inner quote by a#(check = "grep -c "#include" f"runsgrep -c) is accepted as before, and the sentence says what it became. It is in the report (warning), the JSON (define_warnings, a new optional field),sideeye mcp's summary, and on stderr assideeye: warning: …, written once at the end so the first line of merged output is still the verdict — and so a run that ends in a SETUP ERROR, or a preflight, shows it too. Nothing runs differently and no spelling is refused. A symbol inside a word (x=n*10,<esc>,~>1.6) is left alone, and a replay is not looked at. A toml value refused as "not one double-quoted string" (expected_status = 0,state = './state',operation = "tool "two words"") is refused with the line it would have been read as. -
Every re-emitted checker line says which world it came from (#707;
src/main.zig,src/recovery.zig,docs/cli.md,docs/evidence.md, ADR 0071 amended). A world's checker output came out bare, so a checker that printed a traceback put fifty lines ahead of the verdict that no reader could attribute to a world. Each line now carriesworld N:— N the crash point, the number of the report'scrash point N of n— orbaseline:for the un-killed world, and a recovery's linesrecovery world N:andrecovery world N check:, where two exhibits' legs had shared one mark. The checker's stderr is in the same capture, in order, and comes out on Sideeye's standard output with the same mark;docs/cli.mdnow says so. The terminal text changes;<work>/checker-output.txt, the evidence bundle and the JSON report do not. When the capture cannot be opened the checker runs uncaptured and its output is unmarked, as before. -
An exploration in which no world could have failed is UNKNOWN
nothing_could_fail, not PASS (#682, #683, ADR 0091;src/contract.zig,src/engine/snapshot.zig,src/refuse.zig,src/report.zig,src/main.zig). The README's rule for a checker — one that cannot fail makes the run UNKNOWN — now holds for the built-in layers too. A run with no crash point, whatever was declared, and a run with no checker, no crash world that printed the marker over a path only one snapshot holds, and no judged path a crash point could change, used to PASS with exit 0; they exit 2 now. "Could change" is a union: the path ended other than it began, or a crash point other than anfsyncor amkdirnamed it, or arenamenamed a directory above it — so an atomic rewrite with the same bytes still PASSes, and a parent'sfsyncor anos.makedirs(…, exist_ok=True)does not make an untouched path count.preflightrefuses a recording with no crash point with the same reason — the oneexplorereaches once its completeness gate is met, since preflight has no such gate;replayis not asked, since its one world answers whether that world still fails. Three new next steps say where an undeclared define's commands run (nothing_in_state), or to declare a check — or a marker, where one has a created or removed path to judge. A record on a link itself (unlink,rename,link,symlink,rmdir,mkdir) names the link and not its target, both here and in the per-path reconciliation, where the record that made a link used to account for an unrecorded change at the file it points to. Theunknown_reasonclosed set goes from 34 to 35 — the fifth break ofdocs/contract-freeze.mdsurface 2, recorded there by owner ruling;contract_versiondoes not move. A read-only control row, such as topydo'slsused as a zero-operation PASS, now answersnothing_could_fail; records taken before this are left as measured. It is a floor, not a guarantee: a crash point that names a judged path without changing it — a lock file opened for writing, a failed call — still counts. -
A PASS whose atomicity invariant judged only paths the operation never touched says so on its verdict line, and the JSON counts them (#683, ADR 0091). An exploration's such PASS was carried by a checker or the marker; the headline keeps its claim and adds ", but the operation touched none of the N path(s) it judged" (or ", but it had no path to judge"). The new optional field
l0_judged_paths_touchedcomes after the twol0_fields it qualifies. A replay's PASS carries both. -
A
statethat exists and is not a directory is SETUP ERRORdefine_invalid, before anything runs (#682). It used to snapshot as an empty tree, and the run PASSed with no crash point. -
Acceptance legs that measured observation through a PASS read
nothing_could_failinstead, and the macOS kill-landing and fs_usage toys rewrite a file the state already holds (spike/acceptance.sh,.github/workflows/ci.yml,spike/thread-kill-lands.sh,spike/fsusage/acceptance-local.sh). The refusal is the last gate before a PASS and is raised after every world, so reaching it is the same evidence the PASS was; the counts those legs pin are in the JSON on both. Check 2nf is new and holds the rule's three falsifiable shapes. -
The strace oracle reads extended-attribute reads,
inotify_add_watch,preadv/preadv2and the polls on the state directory as reads (#684;src/oracle.zig).getxattr,lgetxattr,fgetxattr,listxattr,llistxattr,flistxattr,inotify_add_watch,preadv,preadv2,poll,ppoll,selectandpselect6joinfaccessat2andepoll_ctl(#542) on the oracle's read-only list, so a target that asks one of them about its state is no longer refusedunsupported_syscall_observedfor asking. These are the callsdocs/report-schema.mdnamed as "still refused"; that sentence is replaced. The writing side —setxattr,removexattrand theirl/fforms — still refuses, held by a unit test and an acceptance control (TOY_SETXATTR). It applies in every observation mode, since the oracle is strace in all three; the macOS fs_usage oracle keeps its own list. -
A mistake on the command line is named in one line, and each command has its own help (#705). The first answer to a mistyped command or flag used to be the whole reference — 218 lines, the longest 412 columns — with no line saying what was wrong:
sideeye explor,sideeye init,sideeye --versionand a baresideeye replayprinted it with exit 3,explore --frobnicatetyped last was told "an option is missing its value", typed with a value it was "unknown option" without saying which, anddemo,mcp,versionandevidencerefused an extra argument without naming it (evidenceread--frobnicateas a file). Now each of those is one line that names what was typed and, when one is close, the nearest spelling that command accepts —explore --stat xis offered--state,explore --recover x--recovery,explore --twic xnothing, because--twiceis preflight's. A value is named the same way:--observe bogus,--expect-status abc,--world-timeout abc,--apparatusand--scratchrefusals now carry what was given, and--observe syscallis offeredsyscalls. A word that names no command, and an argumenthelp,version,mcporevidencedoes not take, goes to stderr with exit 3, as those argument refusals always did; a mistake among the flags ofexplore,preflight,replayordemois still aSETUP ERRORline on stdout. A token the terminal would act on is defanged before it is shown.evidencestill reads a path that begins with a dash when the file is there, and refuses it as the option it looks like only when it is not.sideeye help <command>andsideeye <command> --helpprint that command's own help — its synopsis lines broken to fit, its paragraph, and one line per flag it takes: the first sentence of that flag's entry, and every sentence of the entry that beginsCaution:— cut out of the same textsideeye helpprints, so the two cannot disagree. Three entries of that text were reworded for it:--twicerewriting--stateand--observe syscallskilling a static helper now beginCaution:, and--oracle-fs-usagesays it needs root in its first sentence, so the command's own help does not hide what<command> --helpused to show in full.sideeye helpstill prints the whole reference, otherwise unchanged but for the synopsis linesideeye help [<command>];sideeye --versionissideeye version; a baresideeyeprints the version line and the commands instead of the reference, still with exit 3.--version,help <command>and<command> --helpformcp,helpandversionexit 0 where they exited 3, whichdocs/contract-freeze.md§3 allows: exit 0 is the success of what was asked, and none of them produces a verdict. Help typed anywhere but alone right after the command (explore --state X --help,explore --help --state X) is refused with where help is answered, not answered: the parser may already have removed the--jsonreport by then (#296).spike/acceptance.shcheck 15 now holds<mode> --helpto the mode's own help — shorter than the reference, starting with the version line, a summary line for every flag on the mode's synopsis line — and reads a table of mistakes for the words they must name, and #273's loop overmcp,helpandversionreads each refusal for the flag it got. -
The CI quickstart pins v1.8.0 (
docs/ci-quickstart.md,docs/ci-quickstart/release/install-sideeye.sh,.github/workflows/quickstart-release.yml). The release lane installs the published v1.8.0 asset instead of v1.7.0's. The pin moves after the release, as on v1.7.0, because the lane downloads the asset and the asset exists only once the release is published. Nothing else in the lane changes.
Fixed
- The setup receives
SIDEEYE_STATE_DIR, and the docs name it as where every define command finds the state directory (#708;src/main.zig,README.md,docs/cli.md,docs/checker-cookbook.md,docs/ci-quickstart.md). The operation, the check and the recovery already had it; the setup had onlyTOY_STATE, the demo toy's name, and the only variable the docs described was that one — a checker written from the README alone located the state relative to its own path.TOY_STATEstill reaches every command with the same path, so nothing written against it changes. ASIDEEYE_STATE_DIRin the caller's own environment no longer reaches the setup: the engine's value replaces it there, as it already did for the operation and the check. - A setup, operation or check that cannot be started is a SETUP ERROR naming the file, raised before it runs (#701, ADR 0092;
src/image.zig,src/main.zig,docs/cli.md,docs/report-schema.md). The engine used to spawn the command and read the failedexec's 127 as a fact about the target: an absent or mode-644 checker passed the falsification gate and then refusedbaseline_violates_invariant— a judgement by a checker that never ran — an absent operation refusedrecording_run_failedpointing at--expect-status, and an absent setup "exited 127" without naming a file. Now the run stops asenvironment, saying which command, which resolved file, and whether it is missing, not a regular file, not executable by this user or could not be examined — or that the#!interpreter it names is, or that a bare name has nothing onPATHthat would start (execvp moves past a file whose interpreter is missing, so this does too). The setup is asked about before it runs, leaving nothing on disk when refused; the operation and the check after the setup, which may build or write them — a checker the operation itself would create is now refused, so write it insetup. Exit codes move for these inputs: an absent or mode-644 checker 2 → 3 (0 → 3 where the run recorded no operation and so never ran the checker), an absent operation 2 → 3 under explore, replay and preflight; an absent setup stays 3. The[recovery]commands are not asked about: a recovery changes no verdict and no exit code (ADR 0072). - On macOS, the next step Sideeye prints is one that runs on macOS (#702;
src/main.zig,src/refuse.zig,src/cli.zig,README.md,docs/cli.md,docs/mcp.md).--oracleruns strace, which macOS has none of; it used to be accepted there and fail as "not an executable file" — or, given a real executable, run it as the witness — and the README's own explore line, preflight'snexthint and the no-oracle refusal all named it. Now--oracleon macOS is refused asplatform_unsupportedbefore anything runs, naming--oracle-fs-usageand--allow-unverifiedunder explore and replay and saying to drop it under preflight (which takes neither); preflight's note and hint end with--allow-unverified, and the hint, with its--checkplaceholder taken out, reaches PASS;completeness_not_verifiednames--oracle-fs-usagewhere it named--oracle;boundary_without_oraclesays on macOS that no witness this engine reads can account for another process, where it said to pass--oracle (Linux); the README carries a macOS explore line beside the Linux one, labelled in the prose rather than with a trailing# macOS, which zsh would hand to sideeye as an argument. Exit codes and reasons move on macOS:--oraclenaming a real executable 2 → 3 (it used to run that file as the witness and refuse later),--oraclenaming no executable file stays 3 withsetup_error_reasonenvironment→platform_unsupported. On Linux every run's output is unchanged;--helpnow says--oracleis Linux only, and the--oracle-fs-usagerefusal there, like this one, undoes the state directory the run had just made. - A toml with no
cwd, run from elsewhere, is told which line to add (#700, ADR 0093;src/refuse.zig,src/main.zig,src/contract.zig,README.md,docs/cli.md,docs/report-schema.md). A toml resolves its own paths and argv[0] against its directory and every other argument against the directory the commands run in — Sideeye's own when nocwdis declared — so a define in the README's shape with a relative argument failed from any other directory, as--setup exited 1orrecording_run_failedpointing at--expect-status. Now, when the command that failed carries an argument present under the toml's directory and absent where it ran — or, for a file it is about to create, a directory above it that is — the refusal says so and namescwd = ".": a newnext_step,declare_cwd, onrecording_run_failed,marker_never_observed,nothing_could_failwith no crash point,checker_not_falsified(the corrupted state accepted) andbaseline_violates_invariant's checker layer, and the same in the sentence ofsetup_failedand, under--observe syscalls, beside the mode's own step. An observation, not a guess at the cause — ADR 0086's reason for leaving the wording alone stands, and ADR 0093 records why this is not the guess it declined. The README's toml example declarescwd = ".", so a define copied from it means the same thing from any directory. Not seen: a relative argument that finds a different file and succeeds, one that names nothing on either side, a config read through a link or a pipe. - On macOS, refusing a framework Python's launcher names the interpreter to run instead (#703).
python3 tool.pywith Homebrew's Python, or with the Command Line Tools' frameworkbin/python3— and a console script such asblack, whose#!line names that Python — was refusedchild_process_detectedwith the step "check whether the operation is a shell script wrapping another command", which sent the reader looking for a wrapper that was not there.bin/python3in a framework build is CPython'spythonw.clauncher: it replaces itself withResources/Python.app/Contents/MacOS/Pythonthroughposix_spawnwithPOSIX_SPAWN_SETEXEC, which the shim'sposix_spawnwrapper records only when it returns, so the chain of observation breaks. The refusal stays. On macOS itsmessagenow adds what was read — the image, read before the run; and at the refusal the script and the options on its#!line when that line names the launcher, the framework's interpreter and, when apyvenv.cfgstands in the launcher's directory or the one above, the__PYVENV_LAUNCHER__value the launcher would have set — and the new stepname_framework_interpretersays to make that interpreter the operation's first word, followed by those options and the script, and to set the variable in the environment Sideeye runs in, since anenv:apparatus entry is checked, not applied. The reason, the verdict, the exit code andcontract_versionare unchanged, and every other broken chain keeps the wrapper step — a--copiesvirtual environment'spythonamong them, which is a copy outside the framework'sbin. Two other ways in are not this refusal and keep their steps:/usr/bin/python3on a Mac with only the Command Line Tools, and a script whose#!line is/usr/bin/env python3, are Apple-shipped binaries the shim never reaches (no_shim_marker). The layout is the one the launcher relies on, read rather than listed: a Mach-O in abindirectory beside the framework's library, which is named after the framework (Python, the Command Line Tools'Python3, python.org's free-threadedPythonT), and an interpreter named after the framework orPython. Measured on macOS arm64 with Homebrew's python@3.14 and @3.12 and the Command Line Tools' 3.9: each ofpython3 script, a virtual environment'spython scriptand a console script was refused with the interpreter named, and the define rewritten as the refusal said was accepted, and reached PASS under--allow-unverifiedfor Homebrew's, the virtual environments' runs writing their ownsys.prefix. python.org's 3.13.7 installer was inspected and not installed: its launcher and interpreter sit in the same layout, and their signatures carry the entitlements that let the shim load. A new macOS CI leg,spike/check-framework-launcher-macos.shwith a stand-in launcherspike/toys/pythonw-launcher.c, holds the shipped engine to it: the launcher refused with the interpreter named, the interpreter judged, and the wrapper step kept once the framework library is gone. The README's target limits and DESIGN.md's known constraints say it, andspike/check-readme-shape.sh's bound moves by the line's 414 bytes.