-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
check PD list validation #1186
Milestone
Comments
Maybe PD can panic too if it finds the initialize list has different clusters.
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
pd1 is a standalone cluster(a), pd2 and pd3 is the other cluster(b). We have many TiKVs, when the TiKV bootstraps, it randomly selects a PD to connect, so we may have two TiKVs which have same store ID (1) but belong to different clusters (a and b). We use 1a and 1b to distinguish them.
When 1b is down and 1a restarts, it may connect to cluster b because it chooses the PD in PD list randomly. Badly, 1a can join the cluster b, can communicate with other TiKVs in the cluster b too and we will meet data corruption here.
When we start tikv with a pd list like
pd1:2379,pd2:2379,pd3:2379
, we must check these PDs are all in the same cluster. E,g, we can usemembers
API to get all PD members. If we get pd1 and pd2 only in the response, we can think PD3 is not a valid PD. But here we should also consider the condition that PD3 has already been removed./cc @huachaohuang
The text was updated successfully, but these errors were encountered: