Skip to content

v1.10.0 — Pasteable FAIL commands, refusals name next steps

Choose a tag to compare

@yottayoshida yottayoshida released this 08 Oct 05:33
· 373 commits to main since this release
37a0588

Added

  • Eighteen tools whose state git does not hold, in four waves on the released v1.9.0: 11 FAIL, 5 PASS, 2 UNKNOWN, and nothing filed upstream (spike/dogfood/2026-10-07-user-data-3/; docs/target-classes.md, spike/dogfood/RUNS.md, spike/outcome-funnel.tsv, the B2 exclusions). Nine FAILs are the truncating open and the kill before the write; ezdxf's .bak rotation and libsndfile's in-place WAV lose nothing a rename or a reader does not restore. No FAIL loses data its user cannot get back from the tool's own documented command, so none was reported. roswell, a static image starting a dynamic SBCL, is turned away under unresolvable_path as refused by design although --observe supervised judges it — the shape #685 records under oracle_missed_operation.

  • The four targets whose result moved between runs of one define now have a measured cause (#690; spike/followup-690/, docs/target-classes.md, spike/fsevents/trace-ops.zig, build.zig). Re-run with every trace decoded (trace-ops --records now prints each record's sequence number, process and thread) and tally.py recomputing the tally block in NOTES.md: tombi's refusal is a write landing on another thread of its pool, the order supervised does not record (#687); mogrify writes the time into its PNGs and copies its inputs' change and modification times into them as well, which a restore that rewrites the inputs moves — leaving the date and time chunks out removes every baseline refusal without moving its FAIL 6 of 13; isort's count does not move between runs (8 for both recorded defines); dotter's kill_did_not_land was its define, which left the deploy cache outside the state — with it inside, FAIL 12 of 19.

  • The JSON report carries the figures its oracle, checker and processes sentences state, as eight optional fields (#711, ADR 0096; docs/report-schema.md, "The account as data"). oracle_witness and oracle_operations_agreed; checker_declared and checker_worlds; processes_children_admitted, processes_image_changes, processes_threads_created and processes_writer_threads. Each comes from the variable its sentence prints, and is present only where it was measured: a run that never reached the comparison has no agreement count rather than 0, and the process figures — the recording run's — appear only once every check on the recording's trace has held, so a run refused on a trace cut short or never announced has none. None is a closed set, the sentences do not change, and oracle_operations_agreed is not a gate — oracle_verified stays that. Additive under surface 2 of docs/contract-freeze.md, which records it.

Changed

  • When two clean runs leave different bytes, the refusal and preflight --twice now say where and what kind (#688; src/report.zig, src/defang.zig, src/engine/judge.zig, src/main.zig, DESIGN.md, docs/cli.md, docs/report-schema.md, docs/apparatus.md, docs/checker-cookbook.md, docs/ci-quickstart.md, spike/dogfood/README.md, spike/toys/toy.c, spike/acceptance.sh). A byte-layer baseline_violates_invariant names the byte offset where the recording's and the re-run's bytes first differ, how long the differing stretch is in each and what kind of bytes it holds — decimal digits, hex digits, printable text, or how many bytes fall outside printable text — and no byte of either, since its message reaches the JSON report and MCP. preflight --twice adds a bytes line under each path it names whose content differs, up to eight, quoting both stretches clipped at 48 bytes, in its own output only — a CI log or a transcript that keeps that output keeps those bytes. Neither names a cause; docs/apparatus.md gains a table mapping the common shapes to scratch and apparatus.
  • The refusals the dogfood records met most say what to do, where they said "Change the define" or "refused by design" (#710, ADR 0097; docs/report-schema.md). Counted from the 73 readable reports the outcome funnel points at, and taken back to the site that raises each on this build. recording_run_failed on an exit status nobody declared says to run the operation by hand and then --expect-status if that status is the tool's success; on a run that ended on a signal it names the signal in message and says to look at what stopped it, which on macOS can be the system refusing an image's signature. preflight --twice's second run ending differently says the restore rebuilds the names, kinds and bytes under --state and not their modes — the cause in both records, upx and argocd. The baseline's byte layer names scratch and preflight --twice. kill_did_not_land names preflight --twice in its two shapes. multiple_threads_detected names the README's threads limit, except under --observe supervised. no_shim_marker on a Mach-O whose code directory names a platform — /bin/cp, measured — names a build of the tool that is not part of macOS. Under --observe syscalls and for a toml with no cwd, the recording run's two new steps give way as fix_define did. unresolvable_path, unsupported_syscall_observed, the supervised threads refusal and oracle_missed_operation under --observe syscalls keep the class wall; for a static parent whose writer is a dynamic child, which reaches the last and unresolvable_path, --observe supervised is the way past, and #685 is to name it.
  • PASS, FAIL and UNKNOWN print the account lines they share under one key, in one column and one order (#711; src/report.zig's sayAccount, docs/cli.md). The three text blocks were three format strings: a FAIL's twelve-column key value, a PASS's indented key: value in another order (expected status: 0, processes after case), an UNKNOWN's columns in a third. One function prints them now. The order opens with the lines that say where the commands ran and what the define declared — cwd, apparatus, warning, recovery — so that on an UNKNOWN cwd stays under next, below divergence when there is one (ADR 0086), then a blank line, then explored, expected, atomicity, oracle, metadata, checker, l1, case, replay, evidence, processes, not tested. Each verdict prints a fixed set of them: a FAIL all, a PASS all but replay and evidence, an UNKNOWN the set it printed before. What moves on screen: a PASS takes a FAIL's columns and order; a FAIL's cwd group, and its checker red block when the two exhibits differ, move up under the counterexample; an UNKNOWN's expected moves above atomicity; a PASS's one-crash-point note sits under explored instead of under not tested. The text report is not a frozen surface (docs/contract-freeze.md), and the JSON report does not change. A script that read a PASS's key: value lines reads key padded to twelve columns now, as a FAIL's always were.
  • A FAIL's three command lines run as printed: replay, evidence and reproduce (#709, #711; src/report.zig, docs/cli.md, docs/evidence.md, README.md). The evidence line is the command that renders the bundle, sideeye evidence <path>, where it was the path alone, so a reader of the report learns the command exists; the README names it under "After the first find", held by check 10 of spike/check-readme-shape.sh. The reproduce line was a run of environment variables ending in the placeholder <operation>. It is a whole command now, in a subshell: cd to the directory the commands ran in, the variables Sideeye hands a world — SIDEEYE_OBSERVE with the run's mode among them, without which a run under --observe syscalls is counted the default way and the kill lands on another operation — with SIDEEYE_KILL_GROUP unset (the shim reads its presence, not its value, and the group kill would take the reader's shell with it) and the cgroup variables pinned empty, then the operation's own argv, reading standard input from /dev/null as a world does. It runs in a POSIX shell once the setup has left the state it starts from, as often as it is pasted: it empties its trace file first, because the shim numbers on from the highest number a trace holds, and a second paste of the old line counted past the crash point and ran to completion without the kill. The replay line under --observe syscalls gains --observe syscalls, which it lacked: a case does not record its mode (ADR 0052), so the printed line answered case_no_longer_applies instead of the FAIL. Every value in the three lines is single-quoted where a shell would split or expand it, so a path holding a space pastes as one word, and an ordinary path prints exactly as it did; the JSON replay field carries the same quoting, and the JSON evidence field stays the bundle's path. When the operation or its directory holds a byte the report defangs, when Sideeye could not name the directory, or when the command would not fit the report's output buffer, the line keeps <operation>. The demo toy reads SIDEEYE_STATE_DIR when TOY_STATE is unset, which is what lets the line find the state without the toy's own variable.
  • The CI quickstart pins v1.9.0 (docs/ci-quickstart.md, docs/ci-quickstart/release/install-sideeye.sh, .github/workflows/quickstart-release.yml). The release lane installs the published v1.9.0 asset instead of v1.8.0's. The pin moves after the release, as on v1.8.0, because the lane downloads the asset and the asset exists only once the release is published. Nothing else in the lane changes.

Fixed

  • measure.py --selftest's record-stream case no longer fails by a race of its own (spike/loop-closure-timew/measure.py). The case's subject appended a line to the record by name right after streaming three; when the recorder had not yet written those three, its write landed on the append and erased it, and the case failed though the recorder was right — 10 of 60 runs idle and 46 of 60 under CPU load in a Linux container, and once on main's CI. When its stdout is a pipe the subject now waits for the record to hold the three lines first: 0 of 60 either way, and a recorder that hashes the file afterwards still fails the case.

  • A state file written through a shared memory mapping is refused on every path that could judge it, and the stores are declined rather than judged (#689, ADR 0098; trace contract v19; src/oracle.zig, src/main.zig, src/contract.zig, shim/src/ops.zig, shim/src/macos.zig, shim/src/darwin_libc.zig, shim/src/common.zig, src/fsusage.zig, spike/toys/toy.c, spike/acceptance.sh, spike/check-macos-coverage.py, DESIGN.md, docs/report-schema.md, docs/target-classes.md). A store into a MAP_SHARED mapping has no call behind it, so no crash point can be placed between two stores. Linux's oracle already refused the subject mapping a state file writable; three paths reached a verdict anyway. On macOS neither the shim nor fs_usage's reader saw a mapping at all, and a target that stored into two pages of one passed 3/3. On Linux a child's writable shared mapping counted only as a touch, which the v15 rule admits when the child's other writes are recorded. And on both, a read-only shared mapping given PROT_WRITE by mprotect was seen by nobody — strace did not trace mprotect. Now the strace oracle traces mprotect, pkey_mprotect and mremap, remembers each read-only shared mapping of a state file the run opened for writing, and refuses a writable one or a PROT_WRITE mprotect over a remembered one from any process — unless something else refuses the run first, as a child the v15 rule does not admit is; and the macOS shim interposes mmap and mprotect to the same effect, with or without an oracle. A read-only shared mapping written through pwrite (bbolt's shape) is judged as before, and a read-only file mapped and unmapped (memmap2's) leaves nothing that a later mprotect in the same run is refused against — on Linux while the run has renamed and linked nothing, since the oracle matches the open and the mapping by name. On macOS this newly refuses what Linux already refused: SQLite in WAL mode (its -shm) and LMDB (its lock.mdb) under the state directory. Contract v19: a case saved under v18 replays case_no_longer_applies, and strace's capture — and the line count the report prints — grows by the protection changes a target makes.