Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable slaves to switch masters when masters are temporarily unavailable #82

Closed
Tracked by #84
krishan1390 opened this issue Jul 15, 2021 · 0 comments · Fixed by #119
Closed
Tracked by #84

Enable slaves to switch masters when masters are temporarily unavailable #82

krishan1390 opened this issue Jul 15, 2021 · 0 comments · Fixed by #119
Assignees
Labels
bug Something isn't working

Comments

@krishan1390
Copy link
Contributor

Due to #72 and after changes in #73 , the following scenario breaks

    • Start 1 slave. Auto Service discovery enables it to replicate from any master
    • Do some writes
    • Shut down the master that the slave is replicating from. Can get the master from - /bin/dkvctl -dkvAddr 127.0.0.1:8001 -getClusterInfo <"dc id">
    • Wait for slave to replicate from another master
    • Do some more writes
    • Shut down the new master that the slave is replicating from. Can get the master from - /bin/dkvctl -dkvAddr 127.0.0.1:8001 -getClusterInfo <"dc id">
    • Do steps 6 and 7 until slave starts replicating from a previous master again. Once it gets to this stage, replication will fail due to slave having higher change number that master

Specifically because the change numbers aren't consistent across raft nodes (when one of them restarts it applies all changes to its own state), this trickles to slaves too which can thus have a higher change number than another node and is thus now not sure what change number to replicate from

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
No open projects
Development

Successfully merging a pull request may close this issue.

3 participants