Skip to content

v0.22.0 — two tunnel sessions per agent

Choose a tag to compare

@github-actions github-actions released this 05 Sep 19:22
· 12 commits to main since this release
0bd4375

Two tunnel sessions per agent: a relay restart is no longer an outage

The fourth child of the scale-out epic (#524). Every agent now holds two tunnel sessions on two different relays, so a relay going away is never an agent's only link and the edge keeps routing through the survivor. This completes the zero-drop rollout story that #523's graceful drain started.

What's new

  • Two sessions per agent (#530, PR #539) — piperd runs two dial loops. The second starts only after the first has connected, so the edge already knows which relay holds the agent and places the second dial on a different one. If a relay drains or dies, the other session keeps serving while the lost slot redials. Not a knob: a supported deployment has at least two relays behind piper-edge.
  • Handshake split into a routing preface and a credential frame. The agent sends its base domain first, then its token. piper-edge peeks only the preface on :7000 and places the dial on the relay with the fewest sessions among those that do not already own that agent, falling back to the whole pool when every relay does (so a dial with a made-up base learns nothing). The relay refuses a second session for a base it already holds, but only after the token checked out, so a bad token gets the same undifferentiated rejection as before.
  • Ownership is a set. agent_owners is keyed on (agent_name, instance_id). The edge picks among an agent's live owners (non-draining first, fewest sessions, earliest started) for :443/:80 and retries once on the next owner if a dial fails. Relays re-record every base they hold on each heartbeat, so a stale row never outlives its relay.
  • Every relay serves every route. Hostnames and custom domains are derived from Postgres when a session registers and re-derived on piper_hostnames NOTIFY, so both of an agent's relays route the same hostnames regardless of which session the agent registered them over.
  • piper status names a duplicate refusal. A slot refused as a duplicate reports the reason instead of a blank "retrying".

Upgrading

This release changes the relay schema in place and the agent↔relay wire protocol (pre-1.x, no migrations). Agents on v0.21.0 or older cannot connect to a v0.22.0 relay, so roll the relay pool first and upgrade agents the same day.

  1. Drop the two pool tables before starting the new binaries — mandatory:
    DROP TABLE agent_owners, relay_instances;
    Both are ephemeral (heartbeats and ownership; refilled within seconds). agent_owners changed its primary key, and both piper-relay and piper-edge apply the same schema.sql with CREATE TABLE IF NOT EXISTS, so whichever new process starts first re-creates them. Unlike v0.21.0, skipping the drop fails silently: the old table keeps one row per agent, the edge never sees an agent's second session, and the pool runs single-owner.
  2. Relays and edge together, then agents. Old relays cannot read the new preface and an old edge forwards it blind, so bring both up on v0.22.0 in one step; the pool is unreachable to agents until the agents follow:
    docker compose up -d --no-deps --scale relay=2 relay
    docker compose up -d --no-deps edge
    Compose recreates both relay replicas together (#535), so every agent's sessions drop once here anyway.
  3. Upgrade every agent. Each piperd logs slot 0: connected and slot 1: connected; agent_owners shows two rows per agent once both are up. A one-relay pool logs slot 1: every relay the edge can offer already holds <base> once and retries every minute: the deployment is mis-sized and both sides say so.

Verified on the hosted relay on 2026-09-05.

Runbook: docs/runbooks/relay-deploy.md — "How it routes", "At least two relays", "What still drops", "Upgrade across a schema change".

Known limits

  • After an unclean agent stop (crash, power loss) both relays keep the dead sessions until the yamux keepalive reaps them (about 30–40 s); the first slot retries every 5 s in that window, so the box's apps are unreachable for about 40 s. A clean stop or upgrade reconnects at once. #538 tracks a relay-side ping that would cut the window to about 10 s.
  • With in-place replacement (no surge), the second step of a rolling relay restart finds each agent on one session for up to a minute. Kubernetes' and ECS' surge defaults avoid this.
  • piper-edge itself still stops abruptly on SIGTERM (#534). Zone-aware placement of the second session is #531; the e2e harness still runs one relay (#537).

Spec: docs/superpowers/specs/2026-09-05-two-sessions-per-agent-design.md. Prior art: docs/superpowers/specs/2026-09-05-tunnel-placement-prior-art.md.