v0.14.1

Searching fault timing, and fixing what that search exposed.

v0.14.0 shipped packet faults and a Raft harness that could express a partition but not vary when it lands. Closing that gap took one new primitive — and turned up four defects, one of them a documented fault that silently did something else.

Added

sleep(duration, clock="wall") — a wall-clock wait that is indifferent to event traffic.

Faultbox had two ways to wait and both were conditional on the SUT: await_stable() returns on quiescence, await_event() on a matching event. Neither can hold a fault open for a fixed time — and await_stable fails at it in exactly the situation the wait is for, because an active fault emits the events that prevent quiescence. Measured on a 3-node hashicorp/raft cluster under partition: 6681 events in three minutes, longest quiet gap 338 ms, so every window above ~340 ms blocked until the per-test deadline and reported INCONCLUSIVE.

Verified on the case that motivated it — same 18-leaf timing search, only await_stable(gap) → sleep(gap):

BEFORE   6 passed, 0 failed, 12 inconclusive     36 minutes
AFTER   18 passed, 0 failed                      ~2 minutes

bandwidth(rate, dir="both", queue="250ms") and mtu(size) — the link shapers deferred from v0.14.0. They take no matcher: a packet_* rule says what happens to packets that look a certain way, a shaper says what kind of link this is.

queue= bounds the backlog in time, so the shaper drops when saturated the way a real bottleneck does — a rate limiter with an unbounded queue is a memory leak with latency, and the sender never observes the congestion a congestion-control bug needs to surface. Measured: 64kbit → 38 ms peak backlog, 256kbit → 9 ms.

mtu() overrides the link MTU, so netstack advertises a smaller MSS and fragments. v0.14.0 had to approximate this with packet_drop(len_gt=576), which drops oversized packets — that looks like a black hole and behaves like nothing real.

Fixed

packet_delay(dir="s2c") was a silent drop, not a delay. Egress builds a batch list, hands release a closure that appends to it, writes the batch when the loop ends and frees it on return — so a packet released later, by the defer queue's timer, was appended to a list nobody would ever write. packet_reorder on egress had the same fate. Both existing delay tests drove the ingress path, so nothing caught it. A fault that silently becomes a different fault is the exact failure this release line exists to prevent.

The TUN device no longer bricks a host. It was the shared constant faultbox0 and it is persistent — it survives the process. Concurrent runs collided, and any run killed without teardown left a device that failed every later packet-fault run with TUNSETIFF faultbox0: device or resource busy, recoverable only by an undocumented ip link delete. Now: per-process names (fbox<pid>), SIGTERM handled alongside SIGINT, and a startup sweep of orphans whose owning process is gone — which is what recovers a machine after a SIGKILL no handler can catch.

An interrupted suite no longer invents verdicts. RunAll had no cancellation check anywhere in its loop, so Ctrl-C cancelled the context and the walk continued, recording each un-started test as INCONCLUSIVE. An 18-leaf search interrupted after one leaf reported 1 passed, 17 inconclusive. It now says 2 passed, 0 failed, 1 inconclusive, 15 not run (interrupted).

faultbox plan was blind to choose() — it reported Total: 2 plan instances for a spec that runs 24 leaves, so --check-cost --max-instances under-reported by 12×. A cost gate that under-reports is worse than none, because it is trusted. Axes are now read statically from the spec AST; a computed option list reports at least N rather than rounding down.

Inconclusive verdicts print their reason. TestResult.Reason always carried it and the summary discarded it.

Full detail in CHANGELOG.md.

Verification: go build + go vet + go test -race -count=1 ./... green (22 packages, 0 failures); cross-compile clean on linux/amd64, linux/arm64, darwin/arm64; packet-fault and shaper corpora plus the 5-test Raft suite pass end-to-end in Lima on kernel 6.8.