-
Notifications
You must be signed in to change notification settings - Fork 391
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
RFC: Follower Snapshot #135
Comments
We can name this feature Follower snapshot :-) Maybe we can even support Follower replication 🤔 |
We've previously talked about how this feature would be a desirable feature for our use case in TiKV, and I think it can be useful for others as well. I'd definitely love to do this, but if anyone else wants to tackle it please feel encouraged to let us know and start it! It sounds like the generalized, simplest definition of the feature is:
I think doing this would open the door for a second, future feature to support follower replication. However this feature will require some more consideration. Let's plan for that after this one? I can open another issue for it, so we can separate the discussion |
I'd like to work on this and I came up with a few questions which may need to be discussed probably:
|
Thanks @Fullstop000
|
@Fullstop000 |
@siddontang Thanks for the quick reply. Considering about the point of efficiency @BusyJay mentioned here, in the situation of 2 IDCs, once leader has applied the We can save 1.5 roundtrip and crossing IDC data sending from leader if node ignores the msgs but it may introduce some extra complexity into raft algorithm. I prefer to keep raft layer clean and let third party ( such as pd ? ) to do the control generally. |
I agree with siddontang, it is better to keep raft layer clean. raft is complicated and is critical to protect data consistency across nodes. one new mechanism introduced should be ensured not to break raft protocol. |
For now snapshot is always sent from leader to follower, which is not always sufficient. For example, consider there are 5 nodes in two data center, (1, 2) and (3, 4, 5). If 5 is leader and 1 needs a snapshot, then data have to be transferred across data center.
But in fact any nodes in cluster can send a snapshot once requested logs are applied. So it's possible that 1 requests a snapshot aggressively from 2, so that data can be transferred internally.
Support requesting snapshot aggressively is also useful for recovery from snapshot files corruption.
The text was updated successfully, but these errors were encountered: