Clustering

Clustering

Run several tyo-mq nodes (typically one per VM) against one shared Redis. All nodes are equal peers — there is no master node; Redis is the shared source of truth.

{
  "storage": "redis",
  "storage_options": { "url": "redis://:<password>@10.0.0.5:6379/0", "prefix": "tyo-mq" },
  "cluster": { "enabled": true },
  "auth": { "enabled": true }
}

cluster.redis_url defaults to storage_options.url, so most deployments configure Redis exactly once. Optional: prefix (key/channel namespace, default tyo-mq:cluster), node_id. Cluster config is read at startup.

What you get

Capability	How
Settings sync	realms, keys, acceptance flags, approved/revoked tokens, persistence config applied on any node propagate to every node (revisioned document + pub/sub channel in Redis)
Replay protection	signed manager proofs are single-use cluster-wide (SET NX PX nonce claims)
Live cross-node routing	produced messages (plain, topic, broadcast) relay over a Redis channel; a producer on node A reaches subscribers on node B
Durable replay across nodes	with shared storage: "redis", a durable consumer can reconnect to a different node and still get its queue
Shared authorization requests	submit on any node, poll/decide from any node; the requester is notified on whichever node holds its socket

Delivery semantics

• The origin node delivers with full semantics: live sockets, durable enqueue, group selection.
• Peer nodes deliver relayed messages to their live local subscribers only — durable enqueueing happens exactly once, on the origin node.
• Durable + ACK remains at-least-once: a consumer that switches nodes may see one duplicate after the switch. Keep handlers idempotent.
• Consumer groups are per-node: relayed messages skip group subscriptions (prevents double delivery), so one group's members should connect to the same node.

Setup checklist

1. Redis on one VM / managed service: bind to the private address, requirepass, maxmemory 256mb + noeviction, appendonly yes, firewall 6379 to the node VMs only. Add a replica + Sentinel (or use a managed Redis) for HA.
2. Identical settings.json on every node (above), plus an identical .env: set TYO_MQ_ADMIN_TOKEN explicitly — if every node auto-generates its own on first boot, the last to publish wins and the others' tokens die.
3. Load balancer with sticky sessions (socket.io's handshake needs them). nginx: ip_hash upstream + WebSocket upgrade headers. Producers and consumers may land on different nodes; only group members must stay together.
4. Verify: change a setting through node 1 (TYO_MQ_HOST=… npm run manager), read it from node 2 — both must agree; a consumer key must be enforced by every node.

Full guide with configs: docs/CLUSTERING.md.

Failure behavior

If Redis goes briefly unavailable: nodes keep serving with their last-known settings, management commands still apply locally, and nonce replay protection degrades to per-node (fail-open). State re-syncs on the next successful publish. Live cross-node relay pauses until Redis returns.

Sizing

Settings sync is a few Redis commands per management action — idle otherwise. The relay adds one publish per produced message; durable storage memory grows with the offline backlog (~payload + 0.5 KB per queued message). A 1-core / 512 MB Redis is comfortable for typical fleets.