Styx v0.5.0 — one field, one payload story

Adds MaxPayload: one field that gives a plugin a large-payload story for
both unary calls and streamed messages, derived rather than hand-computed.

Added

• Stream-chunking for oversize STREAM_MSG messages. A streamed message
  larger than the sending direction's shared-memory inline limit used to be
  a hard rejection, with no oversize route the way unary calls got from the
  burst path. On a connection where the negotiated stream-chunking
  feature is active, such a message is now split into ladder-sized
  fragments that ride the same shared-memory ring in order and are
  reassembled on the receiving side into one logical message, delivered
  whole and exactly once. Chunking is shm-only and stream-local: it never
  routes over the burst socket, and the single server-streaming request on
  STREAM_OPEN and the single client-streaming response on STREAM_CLOSE
  are not chunked — each stays bounded by its direction's inline limit and
  fails with ErrPayloadTooLarge when exceeded, feature or no feature.

• PluginSpec.MaxPayload. One field states a capacity guarantee — styx
  will carry a call or a streamed message up to this many bytes — and
  derives the stock shared-memory geometry, the burst-path ceiling, and the
  new stream-chunking ceiling from it, so nobody hand-computes a burst
  ceiling or a per-direction slab ladder anymore. A value at or below the
  stock ladder's certain-fit bound derives the stock geometry alone, with
  burst and chunking left off; a larger value derives all three together.
  It is mutually exclusive with a non-zero Geometry or BurstMaxPayload
  on the same spec: hand-set geometry and burst ceilings keep their exact
  v0.4.0 semantics, unchanged — this field is not a second way to reach the
  same knobs.

  The guarantee is checked against the transport that actually has to carry
  it, twice. Before spawning the plugin, Transport pinned to
  TransportUDS with MaxPayload above the uds transport's fixed frame
  cap is refused with a *ConfigError. After a shared-memory attach
  negotiates — once the checksum choice, and with it the connection's exact
  per-direction inline limits, is known — the same requirement is checked
  again against those exact limits: an old plugin that offers shared memory
  but leaves burst or chunking unresolved is accepted as long as
  MaxPayload fits what the connection can actually carry, and refused
  with a typed *IncompatibleError naming MaxPayload and the missing
  capability only once it genuinely does not. The error states both
  remedies directly: upgrade the plugin, or lower MaxPayload.

Changed

• The stream spec's provisioning text now matches the implementation
  (docs/specs/stream-protocol.md §4.2). The section's sizing rules are
  documented as provisioning guidance enforced through typed backpressure at
  admission time, not as startup refusals; its per-side maxima are stated to
  be compiled-in constants rather than operator knobs; and the one structural
  check that does refuse an attach — a non-positive lifecycle queue depth —
  is documented with its real ordering: validated after the region is mapped,
  before anything is constructed on top of it. No wire behavior changed.

Fixed

• A stream's terminal outcome is fully visible the moment its terminal
  phase is. The engine used to publish the terminal phase first and store
  the outcome detail a few instructions later, so a caller observing the
  phase at exactly the wrong moment could read an incomplete outcome. The
  winning terminal transition now records the whole outcome inside the same
  critical section that publishes the phase, closing the window.

• Sends that lose a race with a failing terminal report the winner, not a
  symptom. SendMsg or CloseSend on a stream that already terminated
  with an error — or during whose send such a terminal won — used to surface
  a generic cancellation. Both now return the recorded terminal outcome (the
  peer status, the deadline, the crash cause), so the error a caller gets
  from a send agrees with what Err and RecvMsg report for the same
  stream. Normal completion records no terminal error, so a send after it
  keeps reporting the closed-direction refusal it always did.

• Transport-shutdown errors during stream sends map to the documented
  sentinels. A send that hit a closed transport used to leak an internal
  error. It now maps to ErrOutcomeUnknown — the bytes may or may not have
  reached the peer, the same meaning that sentinel carries everywhere else —
  and a send on a direction the caller already half-closed maps to
  ErrStreamAlreadyClosed. The ErrStreamAlreadyClosed documentation now
  walks every path that returns it: the open-side guard, an already-closed
  direction (including normal completion), and a concurrent CloseSend
  loser, each with what a caller may conclude from it.

Performance

• Measured on one machine at the release commit (bench/stream,
  -benchtime=1000x, plus repeated -count=3 runs at -benchtime=500x;
  advisory numbers, not a gate). Below the per-direction inline limit, a
  send on a connection with MaxPayload set (which derives burst and
  chunking together) shows no latency regression against the same size on a
  connection with MaxPayload unset: 4KiB 4.70µs vs. 4.32µs, 64KiB 38.3µs
  vs. 36.2µs, 1MiB− 490µs vs. 490µs. Across the repeated runs, the paired
  differences were smaller than the run-to-run variance measured on the
  same cell, which is the basis for calling them noise rather than a
  regression. Above the inline limit, where a send can only go through
  chunking:
  2MiB costs 991µs and about 2.10MB/op across 34 allocations; 8MiB costs
  4.14ms and about 8.40MB/op across 88 allocations. Both oversize figures
  are the total cost of the derived configuration end to end —
  fragmentation, the repeated underlying sends, credit and arena
  bookkeeping, and the one train-owned copy together — not the cost of any
  single one of those isolated.

Styx v0.4.0 — the burst path

Adds the burst path: oversize unary payloads stop being a hard limit.

Added

Size-based transport routing for oversize unary payloads. A message
larger than the shared-memory limit was rejected outright, and the only
remedy — sizing a slab class for rare giants — converted the reservation
into permanently resident memory and let one parked giant head-of-line
block its whole direction. Set the new PluginSpec.BurstMaxPayload
ceiling and such a message now travels a second, generation-scoped
Unix-socket byte stream whose memory cost is transient: allocated per
transfer, garbage on completion, region RSS untouched. Zero (the default)
keeps today's behavior exactly; a message above the ceiling is still
rejected with ErrPayloadTooLarge before any byte moves.

The shared-memory ABI is unchanged — no layout field, no descriptor flag,
no layout_version bump. The path is negotiated per plugin, so a peer
built before this release simply keeps today's behavior. Only unary
request and response frames are eligible; streaming, lifecycle, and
status frames always travel shared memory, which is what keeps every
ordering guarantee intact. The receiver enforces the routing rule:
nonconforming traffic on the burst socket condemns the connection before
any call or stream state can observe it.

Every burst failure lands in an existing error class with its existing
meaning. A transfer a peer stalls is bounded by a receive-completion
budget (30-second slack plus a 1 MiB/s rate floor; the documented
defaults may only loosen in future releases) and restarts the instance
instead of parking it, and a slow-but-conforming giant no longer risks a
false transport-wedge restart.

styx.burst.count metric — messages routed to the burst path,
counted at the routing decision. A rising rate is the signal that giants
are frequent enough to deserve a larger shared-memory geometry instead
of routing.

Changed

Cross-call unary ordering is now explicitly documented as not a
contract (migration guide).
The only order a caller can construct is causal — await one call's
response before sending the next — and causal order is preserved
regardless of which transport each call takes. The already-best-effort
nature of cancellation gains one documented window: a cancel can arrive
before the routed request it names, in which case the handler runs to
completion while the caller keeps its local cancelled outcome.

Full details in CHANGELOG.md.

v0.3.1

Closes the stderr-loss window v0.3.0
shipped with and named in its own changelog.

Fixed

The last stderr a crashing plugin writes is no longer lost when the plugin
exits in the same instant it prints. PluginCrashError.StderrTail, and the
stderr suffix inside Reason, could come back empty for a plugin that had in
fact printed — indistinguishable from one that printed nothing, which is the
exact ambiguity the tail exists to remove.

v0.3.0 closed the case where the delivery queue discarded the tail. This closes
the one it named as remaining: the stdio read ends were closed while the readers
could still be draining them, discarding whatever the kernel still held.

The close was racing a reader that was about to finish anyway — the reap
immediately before it produces the EOF the reader stops at, because the host
closes its copy of each pipe write end at spawn. Both paths that end an instance
now wait for the readers before closing what they read.

The ordinary case costs nothing: the readers have already finished by the time
the close is reached. The wait is bounded, because a descendant that escaped the
process-group kill can hold a write end open forever and that pipe never reaches
EOF — losing a tail beats never completing a teardown.

lifecycle.Process.Kill no longer closes stdio (internal API): only its caller
knows whether anything is still reading, and it returns the moment the child is
reaped, which is precisely when a reader still has work left.

Compatibility

No public API change. A consumer on v0.3.0 upgrades by bumping the version.

The workaround v0.3.0 documented for this gap — configuring a PluginSpec.Stdio
sink and logging from there as well — is no longer needed for it, though it
remains useful for observing live output.

Validation

make ci green: lint (plain and failpoint-tagged), vet, full -race suite,
ringhook/eventhook/failpoint tag builds, allocation gate, integration, soak, and
the separate benchmark module.

The window reproduced about once in two hundred runs before this change; 400
iterations of the affected tests now pass with zero failures. A deterministic
regression test pins the ordering itself: build-tag-gated seams park the stderr
reader before it reads a byte and release it from the drain's own entry, so the
reader is provably still holding unread output when teardown begins. The seams
are compile-time eliminated in normal builds.

v0.3.0

Observability and lifecycle reporting: a host can now say which plugin a
failure came from, order the transitions it observes, account for what it
drops, and size its own shared memory. Plus the teardown and crash-reporting
fixes that writing those answers surfaced.

Full detail in CHANGELOG.md.

Added

• Typed lifecycle verdicts. EventUnhealthy carries a typed error
  (MissedHeartbeatsError, WedgedError) instead of prose, so a consumer can
  branch on which check failed rather than matching message text. Sentinels
  ErrHeartbeatsMissed and ErrWedged work with errors.Is.
• Panic identity. PluginPanicError names the plugin, service, and method.
  Calls through generated stubs address methods by numeric ID and had no name
  to report at all; the generator now registers each method's name via
  RegisterIdentityName, resolved only when a panic actually happens, with an
  allocation gate holding the ordinary call path to its measured cost.
  Regenerate stubs with the v0.3.0 generator to get named panics.
• Drop accounting. observe.MetricStdioDropped, MetricObserveDropped,
  and MetricStdioSinkPanic count stdio lines and observability records a host
  discards. Reported as per-interval deltas rather than one event per drop, so
  a plugin spraying output cannot turn a counter into its own flood, with a
  final delta when an instance ends so the interval a crash cuts short is still
  accounted for.
• Health transition revisions. Event.Revision and
  HealthSnapshot.Revision let a consumer seed from Health() and fold
  Events() with one comparison. Kind and Time cannot do this: the bus
  delivers a critical EventCrashed ahead of an informational EventStarting
  published before it, and two events in the same tick carry equal Time.
  Revisions are comparable only within one Event.Plugin.
• ShmGeometry.RegionBytes(). Reports the exact per-plugin region size,
  derived by the same code that lays out a real region so the two cannot drift.
  Default geometry reserves 65,994,752 bytes per plugin — the number to size a
  container against.

Changed

• Host.Stop honors its context on every path it previously outran: a
  supervisor a parked Start had not handed over, pinned-binary hashing under
  the host lock, and a worker release owned by another caller. Concurrent
  Stop callers are now linearized. A caller passing a tight deadline will see
  context errors where it previously blocked and eventually succeeded —
  teardown continues in the background; the error says the wait ended.
• Host.Start after a Stop has begun reports ErrHostStopped.

Fixed

• A crash's stderr is no longer lost to the sink queue. The tail a crash
  reason is built from was reachable only through the same bounded queue that
  feeds a PluginSpec.Stdio sink, so a sink falling behind, or the
  cancellation that precedes reporting a crash, could empty
  PluginCrashError.StderrTail and the stderr suffix inside Reason. A plugin
  that sprayed output before dying was then indistinguishable from one that
  printed nothing. The tail is now written by the goroutine that reads the
  pipe, before the queue.

Compatibility

Source-compatible with v0.2.0 except for positional composite literals of
styx.Event and styx.HealthSnapshot, which gained a Revision field. Keyed
literals and reads are unaffected:

grep -rn 'styx\.Event{[^K]' --include='*.go' .
grep -rn 'styx\.HealthSnapshot{[^A-Z]' --include='*.go' .

Pre-1.0: the public Go API may still move between minor versions. The wire
contracts (shm-abi.md, stream-protocol.md) are frozen.

Known gap

A narrower stderr-loss window remains: the stdio pipes are closed during
process teardown while the reader may still be draining them, so a plugin that
writes to stderr and exits in the same instant can still lose its tail. Closing
it needs a bounded join before the close, since a plugin leaking a grandchild
that holds the stderr write end would otherwise hang teardown — a
teardown-ordering change deliberately left out of this release. Where stderr is
the whole diagnosis, configure a PluginSpec.Stdio sink and log from there as
well; that path does not share this window.

Styx v0.2.0 — the wishlist crossing

The wishlist release: everything the first external integration of the shared-memory transport — a device gateway embedding Styx as its device-plugin transport — had to work around, closed. Six public-API additions, four fixes the work surfaced, every feature commit externally reviewed to a clean verdict before landing.

Stability: pre-1.0 — the public Go API may still move between minor versions. The wire contracts (shm-abi.md, stream-protocol.md) are frozen and change only by explicit, versioned amendment.

Added

• ErrPayloadTooLarge — an oversize payload is rejected before any byte is published, and that rejection is now errors.Is-matchable at the public API on unary calls, stream opens, and stream sends. A host's error-translation table can classify it instead of default-denying a deterministic condition.
• Live stdio observation — PluginSpec.Stdio takes a StdioSink receiving every stdout/stderr line a plugin writes, live; a slow or panicking sink never blocks or crashes the plugin. PluginCrashError.StderrTail carries the crash tail structured, so log pipelines stop parsing it out of Reason.
• Host.Health(name) — a pull-based, level-triggered health snapshot: most recent lifecycle state, last transition time, last error, and the current consecutive missed-heartbeat count. Built for synchronous probes (a Kubernetes liveness handler, an embedding supervisor's Ping()) that previously had to rebuild retained state from the edge-triggered Events() channel. Unknown names answer ErrUnknownPlugin.
• Protobuf editions support in protoc-gen-go-styx — edition = "2023" contract files generate; the declared range mirrors the pinned protobuf runtime, with golden and response-contract tests keeping it honest.
• NoRestart — a named supervision policy for hosts whose embedding supervisor owns restart decisions, plus the documented host-owned-restart pattern (fresh Host per recreation) in docs/plugin-lifecycle.md.
• IncompatibleError.Kind — a tampered or wrong binary on disk (IncompatibleBinaryIdentity) is now distinguishable from ordinary version skew (IncompatibleHandshake) without parsing Reason.

Fixed

• An oversize STREAM_OPEN surfaced wrapped in the retryable ErrPluginUnavailable, mislabeling a deterministic rejection as transient.
• The shared-memory writer's arena oversize backstop reported an error outside the never-published classification, which would have misclassified a provably-unpublished send as outcome-unknown.
• Host.Start silently attached a second supervisor to an already-started name, overwriting its routing; it now refuses with ErrPluginAlreadyStarted.
• make lint never linted the failpoint-tagged files that make test-failpoint tests; the ci gate now runs both passes.

Full details in CHANGELOG.md.

Styx v0.1.0 — one river, two banks

Styx is a Go plugin framework for local, same-machine, process-isolated plugin communication: a shared-memory data plane (memfd descriptor rings, slab arena, eventfd wakeups) in place of gRPC-over-UDS, behind ordinary protobuf services and gRPC-style generated stubs.

Stability: pre-1.0 — the public Go API may still move between minor versions. The wire contracts (shm-abi.md, stream-protocol.md) are frozen and change only by explicit, versioned amendment.

Why "Styx"?

In the old maps of the underworld, the Styx is the river between two worlds — the boundary itself. That's a plugin framework: your host lives in one process, your plugin in another world entirely, with its own runtime and its own crash domain. When it dies, it dies over there. The boundary is the point — but a boundary you can't cross efficiently is just a wall, so everything depends on the ferry.

Styx is the ferry, and the fare is nearly nothing. Both banks touch the same water: a sealed shared-memory region — descriptor rings, a slab arena, eventfd wakeups — carries a unary round trip in ~2.4 µs where gRPC-over-UDS takes ~16. One fixed-size memfd, resident memory pay-as-you-touch. No daemons, no sidecars — one river, two banks.

The mythology holds up under load. The gods swore unbreakable oaths on the Styx — ours are the frozen shm-abi.md and stream-protocol.md. Achilles was dipped in it and came out nearly invulnerable — this transport was dipped in a chaos suite, a differential oracle, and a leak soak. And with state-preserving hot reload, Styx is the rare river you can cross back over: a plugin goes down, its state ferries home, its successor picks up where it left off.

Highlights

• Shared-memory data plane — p50 2.4 µs unary round trip at a 64-byte payload, vs 7.7 µs over Unix domain sockets and 15.9 µs over gRPC-over-UDS. Faster than hashicorp/go-plugin in all 24 cells of the comparison matrix: 1.65×–4.72× on throughput, payloads 64 B–1 MiB, concurrency 1–64.
• uds fallback + TransportAuto — same API, negotiated per plugin.
• Protobuf IDL, gRPC-style stubs via protoc-gen-go-styx — unary and all three streaming shapes; callers never see shared-memory details.
• Supervised lifecycle — crash isolation with restart policies, health from heartbeat progress counters, subscription-based supervisor events, and state-preserving hot reload.
• Default arena geometry — a seven-rung ladder from 256 B to 1 MiB with headroom-aligned slabs, plus a worked sizing guide in docs/configuration.md.
• Typed error surface — handler errors as *styx.Status, panic/crash isolation as typed errors, explicit retryability, and ErrOutcomeUnknown reserved for genuine ambiguity only.
• Validated — differential suite against the UDS oracle, fault-injection (chaos) suite, long-running leak soak, and failpoint suite, all CI-gated.
• Runnable examples — echo, streaming, hot-reload, a backpressuring slow handler, and a real consumer's device-plugin lifecycle contract (examples/device-gateway/).
• Migrating from hashicorp/go-plugin? See docs/migration-from-go-plugin.md.

Docs

Design of record · Configuration · Plugin lifecycle · Supervisor events · Benchmarks · Performance headroom · CHANGELOG