Skip to content

fix(raft): fix committed entries never applied due to advance() ordering#124

Merged
slow2342 merged 1 commit into
mainfrom
fix/raft-restart-hang
Jun 13, 2026
Merged

fix(raft): fix committed entries never applied due to advance() ordering#124
slow2342 merged 1 commit into
mainfrom
fix/raft-restart-hang

Conversation

@slow2342

Copy link
Copy Markdown
Owner

Summary

  • Fix Raft event loop where committed entries were never applied to the state machine, causing all write and linearizable read requests to hang indefinitely.
  • Root cause: raft-rs 0.7 delivers committed entries via LightReady returned by advance(), not in the original Ready. The old code called advance() after take_committed_entries(), which always returned empty.

Changes

  • src/raft/node.rs — Restructured the Ready processing loop to follow the raft-rs 0.7 API contract: call advance() before processing committed entries from LightReady.

Testing

  • All 184 unit tests pass (cargo test)
  • cargo clippy -- -D warnings — 0 warnings
  • cargo fmt -- --check — OK
  • Manual E2E testing with grpcurl: Put/Get/Delete/Range/Txn/Lease/Session/Lock/Election/Barrier/Queue/Cluster/Maintenance/Shard/Auth/Watch all functional
  • Server restart: data persistence verified, writes work after restart

raft-rs 0.7 delivers committed entries via the LightReady returned by
advance(), not in the original Ready. The old code called advance() at
the end of the Ready processing loop after take_committed_entries(),
which always returned empty. This caused all Raft proposals (writes,
linearizable reads) to hang indefinitely.

Restructure the Ready processing loop to call advance() before
processing committed entries, following the raft-rs 0.7 API contract:
1. Persist entries and hard state
2. Send messages, handle snapshots, process ReadIndex
3. Call advance() to get LightReady
4. Process committed entries from LightReady
5. Call advance_apply() to signal completion
@slow2342 slow2342 merged commit beed7c9 into main Jun 13, 2026
7 of 9 checks passed
@slow2342 slow2342 deleted the fix/raft-restart-hang branch June 13, 2026 14:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant