RFC: one wire protocol — retiring aimdb-ws-protocol so every transport speaks AimX

[lxsaah]

AimDB currently carries two wire protocols for one feature: AimX (tagged NDJSON frames) on UDS/serial/TCP and a separate WebSocket protocol for the WS connector and the browser bridge. Both already run on the same session engine, the fork is only the envelope.

What could change:

• The WebSocket wire becomes AimX, one tagged frame per WS text message. aimdb-ws-protocol,
  the WS codec and the browser bridge's hand-rolled demux (~1,700 lines total) go away;
  correlation, reconnect, keepalive and the offline queue would exist exactly once, in core.
• Wildcard subscribe on every transport: one sub with an MQTT-style pattern (#, *)
  fans in all matching records — events tagged with the record that fired, one late-join
  snapshot per match. aimdb-cli watch '#' would work against any AimDB instance.
• Shared query/list shapes: record.query → {records: [{topic, payload, ts}], total};
  record.list rows carry name / schema_type / entity.
• The JS WsBridge API stays unchanged, only the wire underneath moves. Raw (non-bridge) WS
  clients are the surface that breaks.

The architecture this would leave us with:

flowchart TB
    subgraph S ["server"]
        DB["AimDb — typed records · buffers"]
        AX["AimxDispatch (UDS · serial · TCP)"]
        WS["WsDispatch (HTTP auth · fan-out bus)"]
        ENG["serve / run_session — subs · acks · snapshots"]
        C1["AimxCodec"]
        DB --> AX --> ENG
        DB --> WS --> ENG
        ENG --> C1
    end

    W(["AimX frames — req · reply · sub · subscribed · unsub · event · snap · write · ping · pong"])

    subgraph C ["clients"]
        C2["AimxCodec"]
        RC["run_client — demux · reconnect · keepalive · offline queue"]
        CLI["aimdb-cli / aimdb-mcp"]
        MIR["AimDB ↔ AimDB mirrors"]
        BR["WsBridge (browser, WASM)"]
        C2 --> RC
        RC --> CLI
        RC --> MIR
        RC --> BR
    end

    C1 === W === C2

Loading

What a wildcard subscription would look like on the wire:

→ {"t":"sub","id":1,"topic":"temp/#"}
← {"t":"subscribed","sub":"1"}                                // WS only
← {"t":"snap","sub":"1","topic":"temp/vienna","data":{...}}   // one per matched record
← {"t":"event","sub":"1","seq":1,"topic":"temp/berlin","data":{...}}

The payoff would be: AimX is how you talk to AimDB, over UDS, serial, TCP or WebSocket and cheap future transports (WebTransport, Web Serial, …): only a Connection/Dialer shim per transport. Protocol, codec and engines come for free.

@thaodt what do you think?

[thaodt]

I'm in favor of the architectural direction here.

AimX should be the default JSON protocol over every current transport. Removing the second WebSocket envelope, codec and browser-side demux gives us one place for correlation, subscriptions, reconnect behavior and future transport adapters.

I also compared your feat/retire-aimdb-ws-protocol branch with my #156 representation benchmark branch. Those results do not justify keeping a separate WS protocol or changing the current wire to CBOR. They point toward optimizing the direct JSON path now, while preserving an explicitly negotiated native-binary edge for a future binary-heavy consumer. In other words: one protocol today should not mean one serialization forever.

Before treating the migration as ready, I think we should close a few contracts:

1. Version and migration

This changes more than raw WS framing: query results move from values/count to records/total and old WS clients may not fail clearly. Keeping PROTOCOL_VERSION = "2.0" makes the break hard to detect. I would prefer AimX v3, a versioned WS subprotocol/endpoint or explicit capability negotiation. The auto-subscribe limitation for run_client-based clients should also be part of the migration contract.

2. Truly shared result types

The current branch still has different record.list schemas between WS and AimX. The WS path returns {name, schema_type, entity}, while AimX returns full RecordMetadata, where name also has different semantics. We should define canonical query/list parameter and result types and use them from both dispatch paths.

3. Wildcard and loss semantics

The wildcard grammar needs to be specified and validated. It is described as MQTT-style, but currently uses *, splits only on / and accepts patterns such as temp.* without matching temp.vienna.

The end-to-end loss contract also needs work: events can be dropped before a useful sequence number is assigned, run_client discards the sequence number and client subscription delivery is unbounded. A slow wildcard consumer should either receive bounded backpressure or an observable gap/drop indication.

4. Fan-out performance

The existing WS path serializes a shared frame once. The AimX path in this branch appears to serialize the full envelope once per subscriber. I would like a 1/10/100/500-client fan-out benchmark before we claim the new path preserves O(1)-style sharing.

I ran the focused core, client, persistence, WebSocket and wasm-target checks successfully. There are still a few branch blockers: make fmt-check currently fails because of duplicated Makefile loops, the new design document collides with the existing design 045 and the native unsafe impl Send justification should be replaced with strict wasm32 target gating.

With those boundaries addressed, I think the desired end state is right: AimX as the default JSON protocol over every transport, with future binary representations introduced through explicit negotiation rather than another parallel protocol stack.

[lxsaah]

@thaodt, thanks for this useful pass. Let me walk through what I found digging into each of your points, because a couple of them turned out deeper than they looked.

On versioning. You're right and it's actually worse, neither side checks the version at all today. The server's hello handler is just "hello" => Ok(self.welcome()), it never looks at the client's version and run_client never reads welcome.version. PROTOCOL_VERSION gets serialized and then ignored. My suggestion is to go "2.0" → "3.0" and have both sides share one version_compatible() predicate, with the server refusing at hello (a new RpcError::VersionMismatch) if the client's version is incompatible or missing, before it ever builds welcome. Migration shall rather be a follow-up issue, since it is quite self contained.

On the two record.list shapes. Confirmed the split you spotted, WS replied from Arc<Vec<TopicInfo>> ({name, schema_type, entity}), core from the bigger RecordMetadata, so a generic AimX client had to special-case the transport. WS now goes through db.list_records(), the same call core's AimX/UDS path makes and TopicInfo is gone.

On the wildcard grammar. Good catch and it was a real bug: topic_matches("temp.*", "temp.vienna") returned false, because the matcher split on / while nearly all our keys use dots. I went with . as the single separator, topic_matches splits on . only now.

On loss/backpressure — this is the one I want your read on. I don't think there's a single "is subscribe lossy?" switch to pick, because each record's buffer type already is its loss contract: SingleLatest is latest-wins, SPMC Ring(N) is bounded and reports an exact BufferLagged { lag_count }, Mailbox overwrites. So the job isn't to invent a policy, it's to carry the one each record already has out to the remote client and right now that's dropped on the floor (BufferLagged gets swallowed at the session edge and the client throws away the wire seq).

Your worry was that one socket muxing every record means per-record loss can't survive. I think it can, because every frame is tagged (sub, seq) and seq is per-subscription, a gap in one sub's seq is that sub's loss no matter which shared queue actually dropped it. So the fix is small: SubUpdate gets a skipped: u64 (0 = no loss), the server folds buffer lag into the per-sub seq (seq += skipped + 1) so buffer loss and queue loss land on the same counter and the client derives skipped from the per-sub gap. I made its channel bounded too. The bit I'd like a second opinion on: I made skipped a field on a successful Ok(SubUpdate) rather than a Gap error variant, since loss riding alongside a good delivery matches how BufferLagged lets a reader keep going and it keeps the item type unchanged, so WsBridge and aimdb-cli watch just read update.skipped. The known downside is that skipped rides the next update, so "lost N then silence" won't surface until traffic resumes; it's a best-effort live signal and record.query stays the complete path.

On fan-out cost. I want to push back on the framing a little first. "Old path serializes once, AimX serializes once per subscriber" is true, but the old frame could reuse finished bytes precisely because it carried no per-subscriber fields. The new event frame carries sub (demux) and seq (drop detection), which is the whole point of converging and main is going away, so "serialize once" isn't really a baseline we can fall back to. So I benchmarked the branch against itself (isolated encode loop + a real socket harness), 1/10/100/500 subscribers, small and large payloads. This is a noisy virtualized host, so I'm treating absolutes as ballpark and leaning on the relative signals. Scaling is clean, linear in the encode loop and end-to-end through the real ClientManager/DashMap bus, no O(N²) knee. As a baseline, one broadcast reaches all 500 clients in the ~17–23 ms range across runs (~36–46 µs/delivery), and that's socket-write-and-scheduling dominated — byte-heavy and small land within noise of each other (in one run byte-heavy even came out slightly faster). The one thing that did look expensive in isolation was the encode: byte-heavy hit ~7.6 ms/broadcast at 500 subscribers, because the naive codec re-validated the whole record via as_raw once per subscriber (O(payload) × N).

So I fixed that: splicing the trusted payload verbatim instead is roughly an order of magnitude faster to encode and byte-identical (~14× in-run, byte-heavy ~7.6 ms → ~0.5 ms/broadcast at 500). End-to-end it's within this host's run-to-run noise, though the socket write dominates, so the residual cost is bytes on the wire, not encode. I shipped it in AimxCodec since it's free cleanup, but I wouldn't sell it as a hot-path fix.

So net: per-subscriber encode scales cleanly, 500-client fan-out is comfortable and the one thing that looked expensive is gone.