Skip to content

v0.14.0

Choose a tag to compare

@github-actions github-actions released this 28 Jul 16:10
· 78 commits to main since this release
19604c5

Packet-level network faults. Faultbox mediated at two layers — individual syscalls and parsed L7 protocol messages — with nothing in between. A packet was not an object anywhere in the codebase, so "drop this TCP segment", "delay every ACK from the server" and "advertise a zero receive window" were inexpressible. RFC-054 adds that layer using gVisor's userspace TCP/IP stack (gvisor.dev/gvisor/pkg/tcpip) as a plain Go dependency — no fork, no runsc, no new container runtime.

The capability that motivates it: a dropped packet sends no RST. The existing drop() closes the connection, and well-written clients handle ECONNRESET correctly on the first try. A socket stuck in ESTABLISHED writing into a void until a keepalive fires — the failure that actually takes production down — was unreachable. It is now one line.

determinism(runtime = "gvisor")

packet_drop(dir = "to_service", flags = "PSH,ACK", after = 3)
packet_delay(delay = "200ms", where = lambda p: p.payload.startswith("SELECT"))
packet_window(size = 0, dir = "from_service")     # zero-window stall

Added

  • Packet faults: packet_drop, packet_delay, packet_reorder, packet_duplicate, packet_corrupt, packet_reset, packet_window, packet_pass. Opt in with determinism(runtime = "gvisor"); Linux with CAP_NET_ADMIN required (use the Lima VM on macOS).
  • Declarative matcher: dir, proto, flags ("PSH,ACK", "!RST"), port, len/len_gt/len_lt, payload_prefix, payload_contains, plus nth/after/every and the RFC-042 §8.9 probability/max_fires/mode semantics — identical to syscall faults.
  • where= lambda escape hatch with a read-only Packet value. Declarative kwargs evaluate first, so a lambda refines a cheap filter rather than replacing it.
  • partition_start() / partition_stop() and direction= for one-way cuts.
  • packet trace events; a 12-scenario corpus at poc/gvisor-rfc054/; a new tutorial chapter.

Fixed — three bugs that predate this release

Each was user-visible, each silently passed tests, each now has a guard test:

  • ** path globs did not cross directories. op(path = "/data/**") matched nothingMatchPath used filepath.Match, where * and ** are identical. A rule targeting a database that nests its files never fired: no fault, no diagnostic, test green. New internal/pathmatch; the change is a widening, cross-checked by two back-compat tables.
  • events(where = ...) was blind to most event types. It filtered to {syscall, stdout, topic, wal} before invoking the lambda, so the documented events(where = lambda e: e.type == "proxy" ...) could never have worked.
  • The documented source= example does not parse — it puts source= ahead of the positional fault rules, which Starlark forbids.

Fixed — peer-mesh topologies

Gateway address allocation was gated on a proxy address existing, which a dependency cycle can never satisfy — so for at least one link in any mesh, packet rules installed into a link no traffic crossed. Measured on a 3-node hashicorp/raft cluster, isolating the leader from both followers:

BEFORE  applies during full isolation = 88   ← leader with no quorum kept committing
AFTER   applies while leader had no quorum = 0

source= now reaches rule installation (it was parsed, stored, traced — then dropped), and partition() is rebuilt on the packet gateway. The old implementation denied connect(), which only blocks connection setup; against any service that pools connections it silently did nothing. Under runtime="default" it now errors rather than downgrading.

Known limitations

  • watch() (filesystem observation) is deferred to v0.14.1. The sink and DSL are complete, but runsc trace create instruments only tasks created after the session starts. Measured: 2 trace points versus 1054 for the same work. A watch() would observe almost nothing while every assertion under it still passed, so it fails at spec load rather than shipping with a caveat.
  • determinism(runtime = "gvisor") does not raise the determinism ceiling. Both runtimes cap at L1. What widens is the mediated surface, not the promise.
  • --runs N is a flake check, not a search. It varies a seed, but a spec with no probability= and no choose() consumes no seed — every run executes the same fault schedule. Fault-timing exploration is the first item in v0.14.1.

Full detail in CHANGELOG.md and RFC-054.

Verification: go build + go vet + go test -race -count=1 ./... green, diffed against a captured pre-work baseline; testops goldens green; cross-compile green on linux/amd64, linux/arm64, darwin/arm64; CI gains a pinned-gVisor build check on all three targets. 12-scenario packet corpus and the 5-test Raft suite pass end-to-end in Lima on kernel 6.8.