Skip to content

v2.0.0 — connection parking: 0.68 KiB/conn idle

Choose a tag to compare

@iamaliybi iamaliybi released this 06 Jul 21:40
dda5402

The parked-connection idle path: idle connections now cost 0.68 KiB of live heap instead of 3.8 KiB — the connection-density gap identified in the audits, closed. A major version because the default runtime behaviour changes (parking is on by default); every v1.x config file loads unchanged, and parking.enabled = false restores the v1.x path verbatim.

Connection parking ([parking], on by default)

A plain-TCP connection that stays fully idle (no in-flight QoS state, no buffered bytes, no partial frame, everything flushed) for idle_grace_secs (default 30) has its per-connection task and io_uring read source torn down — only its fd, armed as a oneshot POLL_ADD on a small per-shard raw io_uring readiness ring, plus a boxed resume record remain. Any inbound byte (ingress) or a delivery routed to the parked client (egress) resurrects it transparently; the client cannot tell the difference.

Semantics preserved and tested across the park: QoS 0 and QoS 1/2 deliveries queue and wake; keep-alive is enforced while parked (Will included); takeover / Clean Start / cross-shard claim close the dormant fd with takeover semantics; topic aliases, packet-id allocator, and negotiated limits survive; graceful shutdown drains parked fds (DISCONNECT 0x8B) and converts sessions to suspended before the final snapshot. Every removal is generation-guarded twice (ring token + session generation). Gauge: $SYS/broker/parked-connections.

Performance: the hot-path logging fix

The per-PUBLISH debug! event cost a measured ~38 µs/msg under the default log filter — every previously published benchmark carried it. Demoted to trace!. Post-fix, identical harnesses, same box, Mosquitto 2.0.18 with set_tcp_nodelay true:

Benchmark rusquitto v2.0.0 mosquitto
200-conn publisher→ack, QoS 0 328k msg/s 149k (2.2×)
200-conn publisher→ack, QoS 1 36.1k 32.7k (+11%)
200-conn publisher→ack, QoS 2 21.6k 17.4k (+24%)
Saturating QoS 1, 1 shard 77.6k 87.1k (−11%, runtime wake floor — tracked)
Saturating QoS 1, 3 shards 328.9k 87.1k ceiling (3.8×)
Idle connection (parked) 0.68 KiB/conn 1.18 KiB/conn

The remaining per-core saturating gap is attributed (new examples/wake_probe.rs measures the runtime's per-wake floor; the MQTT layer adds only ~5 µs CPU over it) and tracked with the full Mosquitto-parity program in .agents/next-steps.md.

Verification

111 unit + 22 integration tests (7 end-to-end parking flows over real sockets); adversarial battery 12/12 including four new parking attacks (park-herd 500/500 thundering-herd wakes, park-thrash 1500/1500 cycles, park-takeover 500/500, park-dribble 200/200); clippy --all-targets -D warnings clean.

Assets: rusquitto-x86_64-unknown-linux-gnu, rusquitto-aarch64-unknown-linux-gnu (zig-built against glibc 2.31), and the annotated rusquitto.config.toml. Full details in CHANGELOG.md.