v2.2.1 — route() allocation cleanup

Internal hot-path cleanup. No wire, config, or behavioural change over 2.2.0.

Changed

• route() fan-out no longer clones a client_id per matched subscriber. The per-publish best/groups maps now key on the subscriber's borrowed &str (from the topic trie) instead of an owned String, via a disjoint-field-borrow of ShardState (the trie stays borrowed across the whole fan-out while delivery reaches sessions/wal/unpark_tx through a free deliver_to). This removes one heap allocation per matched subscriber per message — ~15M allocations/second at a 1000-subscriber fan-out.

Honest note on impact: this is an allocation reduction and a borrow-structure simplification, not a measured throughput win. On a CPU-saturated 1000-subscriber fan-out the publish rate is unchanged within run-to-run noise (~64 µs/publish before and after) — the per-delivery cost is dominated by the mailbox try_send and Delivery construction, not the short-string allocation. Shipped for the cleaner structure and reduced allocator traffic (which can still matter under memory pressure or a different allocator), with no performance claim attached. Full suite green (120 unit + 23 integration, including shared-subscription exactly-once, cross-shard, No Local, and Subscription Identifier echo).

Assets: rusquitto-x86_64-unknown-linux-gnu, rusquitto-aarch64-unknown-linux-gnu (zig-built, glibc 2.31), and the annotated rusquitto.config.toml. Full history in CHANGELOG.md.

v2.2.0 — 2.x consolidation

Consolidation release. No functional change over 2.1.2 — this marks the 2.x hardening line as a stable minor and finishes the repository housekeeping.

The 2.x line, rolled up:

• Connection parking (2.0.0) — idle connections at 0.68 KiB/conn, below Mosquitto's floor.
• Runtime tuning (2.1.0) — [runtime] io_memory_kib (empty-broker RSS to Mosquitto parity) and spin_before_park_us (single-message latency below Mosquitto's).
• Property-based parser fuzzing (2.1.1) — continuous in cargo test.
• Security & memory audit (2.1.2) — fixed a will-topic $SYS/wildcard bypass and a subscribe-ACL wildcard escalation; bounded topic-trie / interner / shared-cursor growth.

Changed

• Removed the superseded top-level docs/ guides (ARCHITECTURE.md, MQTT_IMPLEMENTATION.md, MQTT_PACKETS.md); maintained notes live in .agents/ and README.md.
• Documented the task-per-connection memory/CPU deficits as accepted architectural bounds of the glommio 0.9 runtime (prototyped and proven non-viable to remove), leaving the roadmap clean.

Assets: rusquitto-x86_64-unknown-linux-gnu, rusquitto-aarch64-unknown-linux-gnu (zig-built, glibc 2.31), and the annotated rusquitto.config.toml. Full history in CHANGELOG.md.

v2.1.2 — security & memory hardening

Security & memory hardening from a comprehensive optimization/security audit. Two genuine authorization vulnerabilities fixed, two unbounded-growth vectors bounded. No wire or config compatibility change — a recommended upgrade for any deployment using ACLs or with churning topic namespaces.

Security fixes

• Will-topic validation (fixes a $SYS / wildcard / NUL bypass). The Will Message topic was checked against ACLs but never against the concrete-topic validator that every live PUBLISH passes through. A client could set a retained will on the broker-reserved $SYS/... namespace (or a wildcard / NUL-bearing topic), then disconnect abnormally, and the broker would publish and retain the forged message — poisoning $SYS/# telemetry for every current and future subscriber. The will is now validated exactly like a live publish at CONNECT and dropped if invalid.
• Subscribe-ACL wildcard-subsumption (fixes a privilege escalation). SUBSCRIBE authorization compared the requested filter with a matcher that treats its argument as a concrete topic — so a client granted home/+ could subscribe to home/# and receive the entire subtree. A new correct filter-subset test allows a request only if every topic it could match is also covered by an allow rule. Publish ACLs are unchanged.

Memory fixes

• Topic trie prunes dead nodes; interned segments are reclaimed. UNSUBSCRIBE / disconnect emptied a node's subscriber list but never removed the empty nodes, so the trie grew with every distinct filter ever seen (common when filters embed per-client/correlation ids) and the segment interner held each segment forever. Removal now prunes empty nodes, and a periodic per-shard sweep reclaims unreferenced interned segments — both bounded to the live subscription set under churn.
• Stale shared-subscription round-robin cursors are reclaimed by the same sweep.

Notes

The audit confirmed the message hot path is already well-optimized; the one remaining allocation (a per-subscriber client_id clone in route()) is documented but deliberately left unchanged pending a wide-fan-out benchmark, as eliminating it needs a delivery-path borrow-restructure not worth the regression risk without proof of the win.

Tests: +7 (120 unit + 23 integration). Adversarial battery still 12/12; clippy -D clean.

Assets: rusquitto-x86_64-unknown-linux-gnu, rusquitto-aarch64-unknown-linux-gnu (zig-built, glibc 2.31), and the annotated rusquitto.config.toml. Full details in CHANGELOG.md.

v2.1.1 — property-based parser fuzzing

Hardening release: property-based fuzzing of the MQTT parser and packet handlers, wired into cargo test. Closes the last testing gap named in the audits. No runtime behaviour change (test-only + a dev-dependency).

Parser fuzzing

A proptest harness (src/server/connection/tests.rs::fuzz) that runs inside cargo test, so the parser is fuzzed continuously in CI rather than only spot-checked against the hand-curated malformed battery. Three properties over an adversarial input distribution — pure random bytes, single plausible-but-malformed packets, and concatenations (framing-boundary fuzzing):

• the frame parser never panics and every parse loop terminates;
• a fully-connected connection fed arbitrary bytes drives every handler (publish, subscribe, the QoS ack flows, ping) through the real dispatch seam without panicking;
• a pre-CONNECT arbitrary packet is rejected cleanly by the CONNECT-first guard.

Deep-validated at PROPTEST_CASES=50000 (50 k parser + 3 k dispatch cases) with no findings.

On the remaining audit deficits (measured, documented)

• Active-connection memory, CPU-per-message, and the −7% saturating throughput share one root cause — the task-per-connection model and the io_uring per-wake floor. This release measured that runtime tuning cannot close them (a spin_before_park sweep left saturating QoS 1 flat at ~81k). The only lever is dispatcher mode + multishot RECV, gated behind a prototype (next-steps.md §1).
• The cross-shard delivery tax is structural, not a defect: one mandatory cross-thread reactor wake over the mesh, inherent to shared-nothing. Recorded in scope.md.

Assets: rusquitto-x86_64-unknown-linux-gnu, rusquitto-aarch64-unknown-linux-gnu (zig-built against glibc 2.31), and the annotated rusquitto.config.toml. Full details in CHANGELOG.md.

v2.1.0 — memory & latency: 8 MiB baseline, sub-Mosquitto latency

Memory & latency optimization, aligned with thread-per-core: the empty-broker footprint is more than halved and single-message latency can now beat Mosquitto's — both from tuning the glommio runtime, with zero throughput regression and no architectural change. Two new [runtime] knobs.

[runtime] io_memory_kib (default 512)

glommio pre-allocates and pins a 10 MiB io_uring registered-buffer pool per core at startup — resident for the shard's whole life, and the dominant term in the empty-broker footprint. The network fast path (recv/send) never draws from it; only DMA file I/O (persistence snapshots) does, and that falls back to the heap when exhausted, so shrinking it is safe (glommio's 64 KiB floor enforced).

Empty-broker RSS	v2.0.0	v2.1.0	Mosquitto 2.0.18
1 shard	17.6 MiB	8.1 MiB	7.6
4 shards	51.7 MiB	13.1 MiB	—

Verified with zero regression: saturating QoS 1 unchanged (79.4k/core, 359k on 3 shards), parked-idle floor unchanged (0.68 KiB/conn), adversarial battery still 12/12. Bonus: the small pinned pool keeps multi-shard within a tight RLIMIT_MEMLOCK, so multi-shard now starts on memlock-constrained hosts that the 10 MiB×N default could push into io_uring ENOMEM.

[runtime] spin_before_park_us (default 0 = off)

Busy-polls io_uring completions for this many microseconds before parking the reactor, removing the park/unpark round-trip from single-message latency. Measured at 50 µs: PUBLISH→PUBACK RTT p50 37 → 27 µs — below Mosquitto's 31.9 µs. Opt-in because spinning trades idle CPU for the latency; suited to latency-critical, steadily-busy shards. Effective only under max-spread/max-pack placement.

Notes

Per-connection cost is unchanged (idle 6.24, active 7.56 KiB/conn) — this release lowers the fixed baseline and unlocks cheap multi-shard, not the per-connection constant. Total footprint still improves markedly: a 500-connection active broker drops 21.4 → 11.8 MiB (−45%) from the baseline win alone. Closing the per-connection constant is the dispatcher-mode program (.agents/next-steps.md §1), whose A0 design study landed this cycle; the rewrite is gated to the next release behind a prototype.

Assets: rusquitto-x86_64-unknown-linux-gnu, rusquitto-aarch64-unknown-linux-gnu (zig-built against glibc 2.31), and the annotated rusquitto.config.toml. Full details in CHANGELOG.md.

v2.0.0 — connection parking: 0.68 KiB/conn idle

The parked-connection idle path: idle connections now cost 0.68 KiB of live heap instead of 3.8 KiB — the connection-density gap identified in the audits, closed. A major version because the default runtime behaviour changes (parking is on by default); every v1.x config file loads unchanged, and parking.enabled = false restores the v1.x path verbatim.

Connection parking ([parking], on by default)

A plain-TCP connection that stays fully idle (no in-flight QoS state, no buffered bytes, no partial frame, everything flushed) for idle_grace_secs (default 30) has its per-connection task and io_uring read source torn down — only its fd, armed as a oneshot POLL_ADD on a small per-shard raw io_uring readiness ring, plus a boxed resume record remain. Any inbound byte (ingress) or a delivery routed to the parked client (egress) resurrects it transparently; the client cannot tell the difference.

Semantics preserved and tested across the park: QoS 0 and QoS 1/2 deliveries queue and wake; keep-alive is enforced while parked (Will included); takeover / Clean Start / cross-shard claim close the dormant fd with takeover semantics; topic aliases, packet-id allocator, and negotiated limits survive; graceful shutdown drains parked fds (DISCONNECT 0x8B) and converts sessions to suspended before the final snapshot. Every removal is generation-guarded twice (ring token + session generation). Gauge: $SYS/broker/parked-connections.

Performance: the hot-path logging fix

The per-PUBLISH debug! event cost a measured ~38 µs/msg under the default log filter — every previously published benchmark carried it. Demoted to trace!. Post-fix, identical harnesses, same box, Mosquitto 2.0.18 with set_tcp_nodelay true:

Benchmark	rusquitto v2.0.0	mosquitto
200-conn publisher→ack, QoS 0	328k msg/s	149k (2.2×)
200-conn publisher→ack, QoS 1	36.1k	32.7k (+11%)
200-conn publisher→ack, QoS 2	21.6k	17.4k (+24%)
Saturating QoS 1, 1 shard	77.6k	87.1k (−11%, runtime wake floor — tracked)
Saturating QoS 1, 3 shards	328.9k	87.1k ceiling (3.8×)
Idle connection (parked)	0.68 KiB/conn	1.18 KiB/conn

The remaining per-core saturating gap is attributed (new examples/wake_probe.rs measures the runtime's per-wake floor; the MQTT layer adds only ~5 µs CPU over it) and tracked with the full Mosquitto-parity program in .agents/next-steps.md.

Verification

111 unit + 22 integration tests (7 end-to-end parking flows over real sockets); adversarial battery 12/12 including four new parking attacks (park-herd 500/500 thundering-herd wakes, park-thrash 1500/1500 cycles, park-takeover 500/500, park-dribble 200/200); clippy --all-targets -D warnings clean.

Assets: rusquitto-x86_64-unknown-linux-gnu, rusquitto-aarch64-unknown-linux-gnu (zig-built against glibc 2.31), and the annotated rusquitto.config.toml. Full details in CHANGELOG.md.

rusquitto 1.10.0

Mutual-TLS identity: a verified client certificate can now carry a per-device MQTT identity for ACLs — the last deferred item from the roadmap.

[tls] cert_cn_as_username

When mutual TLS is enabled and this is set (default false), a verified client certificate's subject Common Name becomes the MQTT username, so [[auth.users]] ACLs apply per device.

• A cert-verified client that sends no username is authenticated by the certificate and keyed on its CN.
• A client that also sends an explicit username is checked against [auth] the usual way (the certificate only gated the transport).
• Default false preserves the prior behaviour (verified cert with no username → anonymous ACLs).

Internally: transport::tls gained a TlsIdentity enum and a resolver that extracts the leaf certificate's CN with x509-parser (rustls verifies the chain but does not expose the parsed subject).

Verified end-to-end

Live harness — client cert CN=sensor-01 mapped to a [[auth.users]] entry whose publish ACL is sensors/01/#, under allow_anonymous = false: the in-scope publish is delivered and the out-of-scope one is refused. Plus two CI unit tests.

96 unit + 15 integration tests, clippy -D warnings clean, fmt clean.

Assets

• rusquitto-x86_64-unknown-linux-gnu — x86-64 Linux binary
• rusquitto-aarch64-unknown-linux-gnu — ARM64 Linux binary
• rusquitto.config.toml — annotated reference configuration

rusquitto 1.9.2

Test coverage: a real end-to-end integration suite, plus documentation of the whole testing strategy. Addresses the audit's "Test Coverage & Reliability 7/10".

tests/integration.rs — 15 end-to-end tests

Each boots a real broker in-process and drives it over real TCP sockets with a minimal MQTT 5 client (built on mqttbytes). Unlike the unit tests (which run the connection state machine over an in-memory mock), these exercise the whole stack — accept loop, transport, routing, sessions, auth, and the cross-shard mesh — the way a client does:

CONNACK · QoS 0/1/2 full handshakes · downgrade-to-granted · retained replay + clear · +/# wildcards · unsubscribe · persistent-session offline-queue replay · will-on-abrupt-disconnect · malformed-frame survival · auth (bad password / anonymous rejection / success) · ACL enforcement · cross-shard delivery · shared-subscription exactly-once across shards.

Brokers are lazily started and shared per configuration, so the suite adds ~2 s to cargo test and runs in CI.

TESTING.md

The full strategy, layer by layer: unit (mock-stream state machine), integration (this suite), the adversarial battery, the crash-recovery and mTLS harnesses, soak, and the memory/throughput probes — with how to run each and the known gaps.

Changed

• Logging init is now idempotent (try_init instead of init), so starting more than one broker in a single process — embedding, or the integration suite — is a no-op on the second call rather than a panic.

Verification

94 unit + 15 integration tests, clippy -D warnings clean, fmt clean.

Assets

• rusquitto-x86_64-unknown-linux-gnu — x86-64 Linux binary
• rusquitto-aarch64-unknown-linux-gnu — ARM64 Linux binary
• rusquitto.config.toml — annotated reference configuration

rusquitto 1.9.1

Robustness plus a measurement. The audit's ack-bound-throughput and cross-shard single-message-latency items were investigated and found to be at their floor — this release ships the two genuine changes that surfaced, and is not a throughput or latency win.

Measured, not guessed

Single-shard QoS 1 request-response is 55 µs p50 / 76 µs p90 / 128 µs p99 with no Nagle artifacts, so TCP_NODELAY was already effective and the per-request cost is the mqttbytes parse plus the socket round-trip — at parity with a mature C broker. The cross-shard single-message tax is one cross-thread reactor wake, internal to glommio. Neither has application-level headroom.

Changed

• TCP_NODELAY is now set explicitly on every accepted socket, rather than relying on the listener's option being inherited by the kernel (which happens on Linux but isn't contractual across platforms). A portability guarantee for the latency-critical option — MQTT is request/response, where a coalesced small PUBACK/PUBLISH would cost a round-trip.
• Single-shard brokers skip the mesh fan-out path — fan_out no longer clones the (absent) peer-senders handle or runs the self-only loop when there are no peers.

These are robustness/cleanup changes. Going below the measured floor is the below-glommio work (parked connections / faster mesh wakeup) tracked in next-steps.md.

Assets

• rusquitto-x86_64-unknown-linux-gnu — x86-64 Linux binary
• rusquitto-aarch64-unknown-linux-gnu — ARM64 Linux binary
• rusquitto.config.toml — annotated reference configuration

rusquitto 1.9.0

Cross-shard reliability and hot-path efficiency. Three audit follow-ups, each verified on a multi-shard broker.

Mesh control plane is now reliable under overload

Session Claim/Handoff (migration) and shared-subscription Join/Leave (membership) were best-effort (try_send_to, drop-on-full). A drop under sustained saturation could desync the replicated shared-subscription membership view (transient double- or zero-delivery) or silently lose a migrating client's session. They now go through a per-shard reliable outbox: enqueuing is synchronous and never drops, and a foreground task drains it with the awaiting send_to (mesh backpressure), FIFO so a Join can't be reordered past a later Leave. The best-effort data plane ($SYS, QoS 0 publishes) is unchanged; control volume is low, so the outbox stays small.

Cross-shard delivery: less CPU per message

The inbound mesh receiver batch-drains — after a blocking recv wakes, it drains every already-queued message without yielding, so a peer's forwarded burst is handled in one wake instead of one reactor reschedule per message.

QoS 1/2 delivery: one PUBLISH clone, not two

send_publish now takes the in-flight retransmit copy only when outbound topic-aliasing could clear the topic; on the common non-aliasing path the message is moved into the in-flight table after a successful write — one fewer Publish clone per QoS 1/2 delivery, which scales with fan-out.

Verification

• Multi-shard (3 shards): cross-shard delivery 40/40; shared-subscription exactly-once 60/60 across members on different shards (no loss, no duplicates).
• No regression: 3-shard QoS 0 throughput 380k msg/s; adversarial battery 8/8 healthy. 94 tests, clippy -D warnings clean.

Assets

• rusquitto-x86_64-unknown-linux-gnu — x86-64 Linux binary
• rusquitto-aarch64-unknown-linux-gnu — ARM64 Linux binary
• rusquitto.config.toml — annotated reference configuration