Skip to content

feat(cluster): Add Phase 3 cluster routing and WAL replication#111

Merged
xe-nvdk merged 1 commit intomainfrom
feat/phase3-cluster-routing
Jan 13, 2026
Merged

feat(cluster): Add Phase 3 cluster routing and WAL replication#111
xe-nvdk merged 1 commit intomainfrom
feat/phase3-cluster-routing

Conversation

@xe-nvdk
Copy link
Member

@xe-nvdk xe-nvdk commented Jan 13, 2026

Phase 3.1 - Raft Foundation:

  • Add hashicorp/raft-based consensus for leader election
  • Implement ClusterFSM for replicated cluster state
  • Add zerolog adapter for Raft logging

Phase 3.2 - Request Routing:

  • Add Router with HTTP forwarding to appropriate nodes
  • Support round-robin and least-connections load balancing
  • Integrate router into API handlers (LineProtocol, MsgPack, Query)

Phase 3.3 - WAL Replication:

  • Add entry-level streaming from writer to readers
  • Implement Sender (writer-side) with fan-out to multiple readers
  • Implement Receiver (reader-side) with auto-reconnect
  • Add ReplicationHook to WAL for real-time entry interception

Phase 3.4 - Peer Discovery:

  • Add TCP-based peer discovery protocol
  • Implement JoinRequest/JoinResponse handshake
  • Support seed-based cluster joining with leader redirect

Performance: Writer forwards achieve 8.8M rps (vs 11.4M direct)

Phase 3.1 - Raft Foundation:
- Add hashicorp/raft-based consensus for leader election
- Implement ClusterFSM for replicated cluster state
- Add zerolog adapter for Raft logging

Phase 3.2 - Request Routing:
- Add Router with HTTP forwarding to appropriate nodes
- Support round-robin and least-connections load balancing
- Integrate router into API handlers (LineProtocol, MsgPack, Query)

Phase 3.3 - WAL Replication:
- Add entry-level streaming from writer to readers
- Implement Sender (writer-side) with fan-out to multiple readers
- Implement Receiver (reader-side) with auto-reconnect
- Add ReplicationHook to WAL for real-time entry interception

Phase 3.4 - Peer Discovery:
- Add TCP-based peer discovery protocol
- Implement JoinRequest/JoinResponse handshake
- Support seed-based cluster joining with leader redirect

Performance: Writer forwards achieve 8.8M rps (vs 11.4M direct)
@xe-nvdk xe-nvdk merged commit 50472a3 into main Jan 13, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant