Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Invalid entries commited on followers with a differing log #4

Open
4e554c4c opened this issue Oct 27, 2020 · 0 comments
Open

Invalid entries commited on followers with a differing log #4

4e554c4c opened this issue Oct 27, 2020 · 0 comments

Comments

@4e554c4c
Copy link

This leads to the CommittedEntriesAgreement invariant to fail.
Here is a picture of the situation on a modified fork with 3 servers and network partitions
heartbeat sent to follower with commitIndex > matchIndex
heartbeat
Raft in invalid state
raft

This is easy to simulate on my fork with disconnection #3 but reproduction can be done with vanilla runway raft with the following steps:

  1. Let S1 become a leader and commit (1,1) into the log.
  2. Disconnect leader (do not allow any further messages to s1)
  3. Append (2,1) to S1's log
  4. Let S2 become leader (with votes from all others) and commit (2,2) into log (with quorum of all except S1)
  5. Reconnect S1 (this will trigger an election, but eventually some node except S1 will be a leader. wlog let it be S2)
    Then, S2 sends a heartbeat eventually to S1 with commitindex=2, but where the matchIndex of S1 is just 0. Then S1 thinks it is fine to commit (2,1), which is different from the committed (2,2) of all other nodes. This causes the invariant to fail.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant