Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

domain, store/tikv: enable etcd client auto sync #9575

Merged
merged 2 commits into from Mar 7, 2019

Conversation

Projects
None yet
6 participants
@disksing
Copy link
Member

disksing commented Mar 6, 2019

Signed-off-by: disksing i@disksing.com

What problem does this PR solve?

When all PD nodes are replaced by new PD nodes, etcd clients (used by DDL worker and safepoint syncer) in tidb will lost connection with PD cluster.

What is changed and how it works?

Enable etcd's AutoSync feature so it will automatically update member list.

Check List

Tests

  • Manual test (add detailed scripts or steps below)
  1. start a cluster with 3 PDs, 1 TiKV, 1TiDB
  2. Join 3 more PDs then member delete original 3 PDs.
  3. Wait 100s and check if there are any 'safePoint out of sync' errors (TRUE for master, FALSE for auto-sync)
  4. Run some DDL and check if it succeds (FALSE for master, TRUE for auto-sync)

Note: No unit test case included, it will be more suitable to add a integration test in schrodinger later. This bug is kind urgent and the fix is straightforward so I guess we can merge it ahead.

Related changes

  • Need to cherry-pick to the release branch
  • Need to be included in the release note
domain, store/tikv: enable etcd client auto sync
Signed-off-by: disksing <i@disksing.com>

@disksing disksing requested review from tiancaiamao , zimulala and nolouch Mar 6, 2019

@codecov-io

This comment has been minimized.

Copy link

codecov-io commented Mar 6, 2019

Codecov Report

Merging #9575 into master will decrease coverage by <.01%.
The diff coverage is 0%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #9575      +/-   ##
==========================================
- Coverage    67.3%   67.29%   -0.01%     
==========================================
  Files         376      376              
  Lines       78961    78963       +2     
==========================================
- Hits        53144    53139       -5     
- Misses      21058    21061       +3     
- Partials     4759     4763       +4
Impacted Files Coverage Δ
store/tikv/kv.go 53.52% <0%> (-0.26%) ⬇️
domain/domain.go 40.24% <0%> (-0.08%) ⬇️
ddl/delete_range.go 75.13% <0%> (-4.24%) ⬇️
store/tikv/gcworker/gc_worker.go 41.57% <0%> (+0.42%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f8d9020...4f02e67. Read the comment docs.

@zimulala
Copy link
Member

zimulala left a comment

LGTM

@zimulala zimulala added the status/LGT1 label Mar 6, 2019

@nolouch

nolouch approved these changes Mar 7, 2019

Copy link
Member

nolouch left a comment

LGTM

@zz-jason zz-jason merged commit 108c3d2 into pingcap:master Mar 7, 2019

5 checks passed

ci/circleci Your tests passed on CircleCI!
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
idc-jenkins-ci-tidb/build Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/check_dev Jenkins job succeeded.
Details
license/cla Contributor License Agreement is signed.
Details

@disksing disksing deleted the disksing:auto-sync branch Mar 7, 2019

disksing added a commit to disksing/tidb that referenced this pull request Mar 7, 2019

disksing added a commit to disksing/tidb that referenced this pull request Mar 7, 2019

zz-jason added a commit that referenced this pull request Mar 7, 2019

zz-jason added a commit that referenced this pull request Mar 7, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.