Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

domain, store/tikv: enable etcd client auto sync #9575

merged 2 commits into from Mar 7, 2019


None yet
6 participants
Copy link

disksing commented Mar 6, 2019

Signed-off-by: disksing

What problem does this PR solve?

When all PD nodes are replaced by new PD nodes, etcd clients (used by DDL worker and safepoint syncer) in tidb will lost connection with PD cluster.

What is changed and how it works?

Enable etcd's AutoSync feature so it will automatically update member list.

Check List


  • Manual test (add detailed scripts or steps below)
  1. start a cluster with 3 PDs, 1 TiKV, 1TiDB
  2. Join 3 more PDs then member delete original 3 PDs.
  3. Wait 100s and check if there are any 'safePoint out of sync' errors (TRUE for master, FALSE for auto-sync)
  4. Run some DDL and check if it succeds (FALSE for master, TRUE for auto-sync)

Note: No unit test case included, it will be more suitable to add a integration test in schrodinger later. This bug is kind urgent and the fix is straightforward so I guess we can merge it ahead.

Related changes

  • Need to cherry-pick to the release branch
  • Need to be included in the release note
domain, store/tikv: enable etcd client auto sync
Signed-off-by: disksing <>

@disksing disksing requested review from tiancaiamao , zimulala and nolouch Mar 6, 2019


This comment has been minimized.

Copy link

codecov-io commented Mar 6, 2019

Codecov Report

Merging #9575 into master will decrease coverage by <.01%.
The diff coverage is 0%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #9575      +/-   ##
- Coverage    67.3%   67.29%   -0.01%     
  Files         376      376              
  Lines       78961    78963       +2     
- Hits        53144    53139       -5     
- Misses      21058    21061       +3     
- Partials     4759     4763       +4
Impacted Files Coverage Δ
store/tikv/kv.go 53.52% <0%> (-0.26%) ⬇️
domain/domain.go 40.24% <0%> (-0.08%) ⬇️
ddl/delete_range.go 75.13% <0%> (-4.24%) ⬇️
store/tikv/gcworker/gc_worker.go 41.57% <0%> (+0.42%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f8d9020...4f02e67. Read the comment docs.

Copy link

zimulala left a comment


@zimulala zimulala added the status/LGT1 label Mar 6, 2019


nolouch approved these changes Mar 7, 2019

Copy link

nolouch left a comment


@zz-jason zz-jason merged commit 108c3d2 into pingcap:master Mar 7, 2019

5 checks passed

ci/circleci Your tests passed on CircleCI!
continuous-integration/travis-ci/pr The Travis CI build passed
idc-jenkins-ci-tidb/build Jenkins job succeeded.
idc-jenkins-ci-tidb/check_dev Jenkins job succeeded.
license/cla Contributor License Agreement is signed.

@disksing disksing deleted the disksing:auto-sync branch Mar 7, 2019

disksing added a commit to disksing/tidb that referenced this pull request Mar 7, 2019

disksing added a commit to disksing/tidb that referenced this pull request Mar 7, 2019

zz-jason added a commit that referenced this pull request Mar 7, 2019

zz-jason added a commit that referenced this pull request Mar 7, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.