Repository navigation
v1.10.0 — Pasteable FAIL commands, refusals name next steps
Added
-
Eighteen tools whose state git does not hold, in four waves on the released v1.9.0: 11 FAIL, 5 PASS, 2 UNKNOWN, and nothing filed upstream (
spike/dogfood/2026-10-07-user-data-3/;docs/target-classes.md,spike/dogfood/RUNS.md,spike/outcome-funnel.tsv, the B2 exclusions). Nine FAILs are the truncating open and the kill before the write; ezdxf's.bakrotation and libsndfile's in-place WAV lose nothing a rename or a reader does not restore. No FAIL loses data its user cannot get back from the tool's own documented command, so none was reported. roswell, a static image starting a dynamic SBCL, is turned away underunresolvable_pathas refused by design although--observe supervisedjudges it — the shape #685 records underoracle_missed_operation. -
The four targets whose result moved between runs of one define now have a measured cause (#690;
spike/followup-690/,docs/target-classes.md,spike/fsevents/trace-ops.zig,build.zig). Re-run with every trace decoded (trace-ops --recordsnow prints each record's sequence number, process and thread) andtally.pyrecomputing the tally block inNOTES.md: tombi's refusal is a write landing on another thread of its pool, the order supervised does not record (#687); mogrify writes the time into its PNGs and copies its inputs' change and modification times into them as well, which a restore that rewrites the inputs moves — leaving the date and time chunks out removes every baseline refusal without moving its FAIL 6 of 13; isort's count does not move between runs (8 for both recorded defines); dotter'skill_did_not_landwas its define, which left the deploy cache outside the state — with it inside, FAIL 12 of 19. -
The JSON report carries the figures its
oracle,checkerandprocessessentences state, as eight optional fields (#711, ADR 0096;docs/report-schema.md, "The account as data").oracle_witnessandoracle_operations_agreed;checker_declaredandchecker_worlds;processes_children_admitted,processes_image_changes,processes_threads_createdandprocesses_writer_threads. Each comes from the variable its sentence prints, and is present only where it was measured: a run that never reached the comparison has no agreement count rather than 0, and the process figures — the recording run's — appear only once every check on the recording's trace has held, so a run refused on a trace cut short or never announced has none. None is a closed set, the sentences do not change, andoracle_operations_agreedis not a gate —oracle_verifiedstays that. Additive under surface 2 ofdocs/contract-freeze.md, which records it.
Changed
- When two clean runs leave different bytes, the refusal and
preflight --twicenow say where and what kind (#688;src/report.zig,src/defang.zig,src/engine/judge.zig,src/main.zig,DESIGN.md,docs/cli.md,docs/report-schema.md,docs/apparatus.md,docs/checker-cookbook.md,docs/ci-quickstart.md,spike/dogfood/README.md,spike/toys/toy.c,spike/acceptance.sh). A byte-layerbaseline_violates_invariantnames the byte offset where the recording's and the re-run's bytes first differ, how long the differing stretch is in each and what kind of bytes it holds — decimal digits, hex digits, printable text, or how many bytes fall outside printable text — and no byte of either, since its message reaches the JSON report and MCP.preflight --twiceadds abytesline under each path it names whose content differs, up to eight, quoting both stretches clipped at 48 bytes, in its own output only — a CI log or a transcript that keeps that output keeps those bytes. Neither names a cause;docs/apparatus.mdgains a table mapping the common shapes toscratchandapparatus. - The refusals the dogfood records met most say what to do, where they said "Change the define" or "refused by design" (#710, ADR 0097;
docs/report-schema.md). Counted from the 73 readable reports the outcome funnel points at, and taken back to the site that raises each on this build.recording_run_failedon an exit status nobody declared says to run the operation by hand and then--expect-statusif that status is the tool's success; on a run that ended on a signal it names the signal inmessageand says to look at what stopped it, which on macOS can be the system refusing an image's signature.preflight --twice's second run ending differently says the restore rebuilds the names, kinds and bytes under--stateand not their modes — the cause in both records, upx and argocd. The baseline's byte layer namesscratchandpreflight --twice.kill_did_not_landnamespreflight --twicein its two shapes.multiple_threads_detectednames the README's threads limit, except under--observe supervised.no_shim_markeron a Mach-O whose code directory names a platform —/bin/cp, measured — names a build of the tool that is not part of macOS. Under--observe syscallsand for a toml with nocwd, the recording run's two new steps give way asfix_definedid.unresolvable_path,unsupported_syscall_observed, the supervised threads refusal andoracle_missed_operationunder--observe syscallskeep the class wall; for a static parent whose writer is a dynamic child, which reaches the last andunresolvable_path,--observe supervisedis the way past, and #685 is to name it. - PASS, FAIL and UNKNOWN print the account lines they share under one key, in one column and one order (#711;
src/report.zig'ssayAccount,docs/cli.md). The three text blocks were three format strings: a FAIL's twelve-columnkey value, a PASS's indentedkey: valuein another order (expected status: 0,processesaftercase), an UNKNOWN's columns in a third. One function prints them now. The order opens with the lines that say where the commands ran and what the define declared —cwd,apparatus,warning,recovery— so that on an UNKNOWNcwdstays undernext, belowdivergencewhen there is one (ADR 0086), then a blank line, thenexplored,expected,atomicity,oracle,metadata,checker,l1,case,replay,evidence,processes,not tested. Each verdict prints a fixed set of them: a FAIL all, a PASS all butreplayandevidence, an UNKNOWN the set it printed before. What moves on screen: a PASS takes a FAIL's columns and order; a FAIL'scwdgroup, and itschecker redblock when the two exhibits differ, move up under the counterexample; an UNKNOWN'sexpectedmoves aboveatomicity; a PASS's one-crash-point note sits underexploredinstead of undernot tested. The text report is not a frozen surface (docs/contract-freeze.md), and the JSON report does not change. A script that read a PASS'skey: valuelines readskeypadded to twelve columns now, as a FAIL's always were. - A FAIL's three command lines run as printed:
replay,evidenceandreproduce(#709, #711;src/report.zig,docs/cli.md,docs/evidence.md,README.md). Theevidenceline is the command that renders the bundle,sideeye evidence <path>, where it was the path alone, so a reader of the report learns the command exists; the README names it under "After the first find", held by check 10 ofspike/check-readme-shape.sh. Thereproduceline was a run of environment variables ending in the placeholder<operation>. It is a whole command now, in a subshell:cdto the directory the commands ran in, the variables Sideeye hands a world —SIDEEYE_OBSERVEwith the run's mode among them, without which a run under--observe syscallsis counted the default way and the kill lands on another operation — withSIDEEYE_KILL_GROUPunset (the shim reads its presence, not its value, and the group kill would take the reader's shell with it) and the cgroup variables pinned empty, then the operation's own argv, reading standard input from/dev/nullas a world does. It runs in a POSIX shell once the setup has left the state it starts from, as often as it is pasted: it empties its trace file first, because the shim numbers on from the highest number a trace holds, and a second paste of the old line counted past the crash point and ran to completion without the kill. Thereplayline under--observe syscallsgains--observe syscalls, which it lacked: a case does not record its mode (ADR 0052), so the printed line answeredcase_no_longer_appliesinstead of the FAIL. Every value in the three lines is single-quoted where a shell would split or expand it, so a path holding a space pastes as one word, and an ordinary path prints exactly as it did; the JSONreplayfield carries the same quoting, and the JSONevidencefield stays the bundle's path. When the operation or its directory holds a byte the report defangs, when Sideeye could not name the directory, or when the command would not fit the report's output buffer, the line keeps<operation>. The demo toy readsSIDEEYE_STATE_DIRwhenTOY_STATEis unset, which is what lets the line find the state without the toy's own variable. - The CI quickstart pins v1.9.0 (
docs/ci-quickstart.md,docs/ci-quickstart/release/install-sideeye.sh,.github/workflows/quickstart-release.yml). The release lane installs the published v1.9.0 asset instead of v1.8.0's. The pin moves after the release, as on v1.8.0, because the lane downloads the asset and the asset exists only once the release is published. Nothing else in the lane changes.
Fixed
-
measure.py --selftest's record-stream case no longer fails by a race of its own (spike/loop-closure-timew/measure.py). The case's subject appended a line to the record by name right after streaming three; when the recorder had not yet written those three, its write landed on the append and erased it, and the case failed though the recorder was right — 10 of 60 runs idle and 46 of 60 under CPU load in a Linux container, and once onmain's CI. When its stdout is a pipe the subject now waits for the record to hold the three lines first: 0 of 60 either way, and a recorder that hashes the file afterwards still fails the case. -
A state file written through a shared memory mapping is refused on every path that could judge it, and the stores are declined rather than judged (#689, ADR 0098; trace contract v19;
src/oracle.zig,src/main.zig,src/contract.zig,shim/src/ops.zig,shim/src/macos.zig,shim/src/darwin_libc.zig,shim/src/common.zig,src/fsusage.zig,spike/toys/toy.c,spike/acceptance.sh,spike/check-macos-coverage.py,DESIGN.md,docs/report-schema.md,docs/target-classes.md). A store into aMAP_SHAREDmapping has no call behind it, so no crash point can be placed between two stores. Linux's oracle already refused the subject mapping a state file writable; three paths reached a verdict anyway. On macOS neither the shim nor fs_usage's reader saw a mapping at all, and a target that stored into two pages of one passed 3/3. On Linux a child's writable shared mapping counted only as a touch, which the v15 rule admits when the child's other writes are recorded. And on both, a read-only shared mapping givenPROT_WRITEbymprotectwas seen by nobody — strace did not tracemprotect. Now the strace oracle tracesmprotect,pkey_mprotectandmremap, remembers each read-only shared mapping of a state file the run opened for writing, and refuses a writable one or aPROT_WRITEmprotectover a remembered one from any process — unless something else refuses the run first, as a child the v15 rule does not admit is; and the macOS shim interposesmmapandmprotectto the same effect, with or without an oracle. A read-only shared mapping written throughpwrite(bbolt's shape) is judged as before, and a read-only file mapped and unmapped (memmap2's) leaves nothing that a latermprotectin the same run is refused against — on Linux while the run has renamed and linked nothing, since the oracle matches the open and the mapping by name. On macOS this newly refuses what Linux already refused: SQLite in WAL mode (its-shm) and LMDB (itslock.mdb) under the state directory. Contract v19: a case saved under v18 replayscase_no_longer_applies, and strace's capture — and the line count the report prints — grows by the protection changes a target makes.