Skip to content

feat: unify all links on WebSocket+Protobuf (v2), deprecate v1#144

Merged
su-fen merged 10 commits into
Stack-Cairn:mainfrom
yyg-max:feat/protocol-v2
Jul 19, 2026
Merged

feat: unify all links on WebSocket+Protobuf (v2), deprecate v1#144
su-fen merged 10 commits into
Stack-Cairn:mainfrom
yyg-max:feat/protocol-v2

Conversation

@yyg-max

@yyg-max yyg-max commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Summary

Unifies every realtime link on a single WebSocket+Protobuf protocol (v2) and deprecates the dual v1 stack (browser JSON WebSocket + desktop gRPC). Business messages in proto/v1/gateway.proto remain the single source of truth
for all three ends; v2 only adds frame shells around them (proto/v2/gateway_ws.proto).

Problem

  • The browser link hand-translated every request through four layers (JSON decode → build proto → relay → proto → hand-built map → JSON encode): ~16 handler files, a protojson int64/casing patch layer, and hand-mirrored Go/TS
    types that had already drifted (inconsistent key casing, dual-case compat fields).
  • Two transports (gRPC + WS) meant two connection/auth/heartbeat stacks and an extra port to expose on self-hosted deployments.
  • No Go linter, stdlib log, unpinned codegen toolchain.

Protocol v2

  • Three routes on the existing HTTP listener (no new port): /ws/v2 (browser), /ws/v2/agent (desktop, replaces Authenticate+AgentConnect), /ws/v2/terminal (both roles, replaces the custom binary framing and
    AgentTerminalConnect). Subprotocol liveagent.v2.pb; one proto message per binary WS frame; first frame is ClientHello (constant-time token check, close 4401 on reject).
  • Browser requests become near-verbatim GatewayEnvelope passthrough behind a whitelist guard (pbws/guard.go, inheriting the v1 route table's security role) with per-connection request-id namespacing; nine broadcast fanouts
    forward session-layer seam messages with zero shaping.
  • The write pump (priority dual queue, shedding, close-on-full for correlated responses, dual-channel heartbeat, idle eviction) is extracted to internal/transport/wscore byte-identical and shared by v1/v2. Reconnect/replay
    semantics (after_seq/stream_epoch, snapshot replay, command dedupe) are unchanged.
  • Desktop tries v2 first every attempt; falls back to v1 gRPC only on handshake-level failure (old gateway 404), never on auth rejection; fallback sessions redial v2 after 10min so a transient blip can't pin a client to v1.
    Settings page badge shows Connected (v2)/(v1).

Compatibility & removal path

  • Old browser tabs keep working on /ws; old desktops keep working on gRPC; new desktop ↔ old gateway falls back automatically. Embedded WebUI is lockstep with the gateway.
  • All v1 surface carries Deprecated: markers (Go/Rust/TS/proto) and runtime counters: /api/status → protocol_usage plus WARN logs on v1 connections. Removal checklist and zero-traffic criteria:
    docs/architecture/protocol-v2-migration.md.

Toolchain

  • buf replaces raw protoc (Go+TS generation, make proto-check = lint + breaking vs origin/main, CI drift gate widened to all generated dirs); golangci-lint baseline fixed and gated in CI; stdlib log → log/slog; tracked
    mise.toml pins go/buf/protoc/golangci-lint/node/pnpm to the exact CI versions.

Performance (measured, websocket_payload_bench_test.go, 50-conversation history.list)

v1 JSON shaping v2 proto encode
encode time 609 µs/op 12.9 µs/op (~47x)
wire size 18,214 B 8,834 B (-52%)
allocations 5,114/op 5/op

Desktop link performance is unchanged by design (it was already protobuf over gRPC); its win is one transport/port and simpler self-hosting.

Test plan

  • Go: go test ./... all green, incl. new v2 integration suite (hello auth, passthrough roundtrip + guard, chat command orchestration, terminal gating, full binary browser↔gateway↔agent path) and -race -count=3 on
    v2/transport packages
  • golangci-lint run ./... — 0 issues; buf lint + buf breaking + generated-code drift check clean
  • Rust: cargo check --tests clean; gateway suite 53 passed (incl. 9 ws_transport tests: URL/port derivation, handshake-vs-auth classification, mock-server hello+envelope roundtrip); chat_history 43 passed
  • WebUI: build + biome clean; 351 tests passed (binary-frame socket suite + adapter unit tests: bigint boundary, oneof dispatch, payload_json roundtrip); mirror check 75 files OK
  • Manual e2e on a live gateway: WebUI chat/terminal over v2 (protocol_usage: v2_browser active, 1200+ requests), desktop connects via /ws/v2/agent (v2_agent_active=1, v1 counters flat at 0), v1 fallback exercised
    against the pre-v2 gateway build and auto-upgraded back to v2 after the retry window

yyg-max added 10 commits July 19, 2026 00:18
@yyg-max
chore(toolchain): adopt buf codegen, golangci-lint gate, slog, and pi…
920ca2b 
…nned versions

- buf.yaml/buf.gen.yaml replace raw protoc; make proto-check adds lint+breaking gates
- .golangci.yml baseline; CI gains lint step and wider proto drift gate
- stdlib log -> log/slog; internal/observability adds logging setup and
  v1/v2 protocol usage counters (exposed later via /api/status)
- mise.toml (tracked) pins go/buf/protoc/golangci-lint/node/pnpm; CI matches
@yyg-max
fix(gateway): resolve golangci-lint baseline findings
7fa7adf 
unchecked defers, unused symbols, ineffectual assignment, staticcheck
simplifications; no behavior change
@yyg-max
feat(proto): add v2 WebSocket frame schema reusing v1 business messages
3317cd9 
- proto/v2/gateway_ws.proto: hello handshake, browser/agent/terminal frame
  shells, gateway-local payloads; v1 messages stay the three-end source of truth
- deprecate service AgentGateway (option deprecated); CancelChatRequest gains run_id
- Rust build.rs compiles both packages as gateway_proto::{v1,v2} with proto alias
@yyg-max
refactor(gateway): extract protocol-agnostic runtime and domain logic
3d6fc5c 
- internal/transport/wscore: priority dual-queue write pump, shedding,
  retries, heartbeat, idle eviction (behavior-identical port, tests moved)
- internal/protocol/shared: origin check, terminal gating/finalize, interest tracker
- internal/chatcmd: chat command orchestration (normalize/probe/dispatch/watchdog)
- session.AwaitUnaryResponse: shared unary request/response primitive
- v1 server rewired onto the shared pieces; behavior unchanged
@yyg-max
feat(gateway): serve v2 WebSocket+Protobuf protocol on /ws/v2*
d6abce7 
- internal/protocol/pbws: browser link (envelope passthrough with whitelist
  guard and per-connection request-id namespacing, local chat/status/workspace
  ops, nine broadcast fanouts, snapshot replay), agent link (replaces gRPC
  Authenticate+AgentConnect), terminal link (proto frames for both roles)
- routes mounted beside v1; /api/status exposes protocol_usage counters
- integration suite incl. full binary browser<->gateway<->agent path;
  encode benchmark: ~47x faster, 52% smaller than v1 JSON shaping
@yyg-max
feat(webui): switch gateway client to v2 binary protocol
af95c05 
- /ws/v2 with liveagent.v2.pb subprotocol; protoc-gen-es generated codecs
  (web/src/lib/proto/gen) plus adapters restoring v1-shaped app payloads
  (single bigint boundary, oneof dispatch)
- terminal client moves to /ws/v2/terminal proto frames, replacing the
  custom binary framing; public client API unchanged
- v1 wire internals deleted (embedded WebUI is lockstep with the gateway);
  tests rewritten for binary frames + new adapter unit tests
@yyg-max
feat(desktop): connect via v2 WebSocket with gRPC fallback and protoc…
da4f07e 
…ol badge

- ws_transport: URL derivation (UI port box applied), subprotocol handshake,
  handshake-vs-auth error classification; serve loop keeps dual-lane merge,
  reconciliation and backoff; liveness watchdog replaces h2 keepalive
- fallback to deprecated v1 gRPC only on handshake-level failure; fallback
  sessions redial v2 after 10min so transient blips never pin v1
- terminal stream rides /ws/v2/terminal; tokio-tungstenite gains rustls TLS
- status snapshot carries protocol; settings page shows Connected (v2)/(v1)
@yyg-max
refactor(gateway): mark v1 protocol surface deprecated with runtime c…
c712907 
…ounters

- Go 'Deprecated:' markers on the JSON WebSocket server, route table, all
  domain handlers, payload shaping, terminal stream, and gRPC service
- v1 connections/requests increment protocol_usage counters and log WARN,
  so operators can watch v1 traffic reach zero before removal
- command-queue-timeout flag becomes a documented no-op
@yyg-max
refactor(session): delete production-dead send/queue APIs
bb5e315 
SendToAgent, RegisterStream, SendToAgentOrQueue and the offline command
queue had no production callers (queue enqueue was unreachable); tests
migrate to the context-aware variants
@yyg-max
docs: document v2 protocol as primary contract with v1 removal plan
95cec27 
- protocols.md rewritten around the unified WebSocket+Protobuf links,
  v1 kept as a deprecated appendix; gateway/webui/gui/overview updated
- new protocol-v2-migration.md: skew matrix, zero-traffic criteria,
  step-by-step v1 removal checklist
- development.md: gateway layering map, capability-addition walkthrough,
  deprecation conventions, lint/proto-check commands
@su-fen
su-fen merged commit 7c0d1a6 into Stack-Cairn:main Jul 19, 2026
7 checks passed
@su-fen su-fen mentioned this pull request Jul 19, 2026
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@yyg-max @su-fen