Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tikvclient: add metrics for gRPC connection transient failure (#12084) #12092

Merged
merged 1 commit into from Sep 9, 2019

Conversation

@lonng
Copy link
Member

lonng commented Sep 9, 2019

Signed-off-by: Lonng heng@lonng.org

Cherry-pick #12084 for release-2.1


What problem does this PR solve?

TiDB access to TiKV through gRPC requests. If the underlying socket is disconnected, gRPC will try to reconnect to the underlying socket, which may cause the request delay to jitter. We need to way to monitor the low-level socket state change.

What is changed and how it works?

This PR adds a metric to monitor the gRPC connection state, the metric will record the connection state before sending the request to TiKV. We can diagnose the delay jitter by rate(tidb_grpc_connection_state) after this PR merged.

Check List

Tests

  • No code
  • Manual test
    image

Related changes

  • Need to cherry-pick to the release branch

Release note

  • Write release note for bug-fix or new feature.
Signed-off-by: Lonng <heng@lonng.org>
Copy link
Member

crazycs520 left a comment

LGTM

@winkyao
winkyao approved these changes Sep 9, 2019
Copy link
Member

winkyao left a comment

LGTM

@sre-bot

This comment has been minimized.

Copy link

sre-bot commented Sep 9, 2019

/run-all-tests

@sre-bot sre-bot merged commit ebf7d8f into pingcap:release-2.1 Sep 9, 2019
13 checks passed
13 checks passed
idc-jenkins-ci-tidb/build Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/build_check_race Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/check_dev Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/check_dev_2 Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/common-test job succeeded
Details
idc-jenkins-ci-tidb/integration-common-test Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/integration-compatibility-test Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/integration-ddl-test Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/mybatis-test job succeeded
Details
idc-jenkins-ci-tidb/sqllogic-test-1 Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/sqllogic-test-2 Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/unit-test Jenkins job succeeded.
Details
license/cla Contributor License Agreement is signed.
Details
@lysu

This comment has been minimized.

Copy link
Member

lysu commented Sep 9, 2019

/bench

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.