-
Notifications
You must be signed in to change notification settings - Fork 282
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add retry to gRPC stream client getter #390
Conversation
/run-integration-tests |
83ae4b8
to
ff737e7
Compare
Codecov Report
@@ Coverage Diff @@
## master #390 +/- ##
================================================
+ Coverage 29.2417% 31.2643% +2.0226%
================================================
Files 59 60 +1
Lines 5328 5655 +327
================================================
+ Hits 1558 1768 +210
- Misses 3629 3730 +101
- Partials 141 157 +16 |
/run-integration-tests |
log.Warn("get grpc stream client failed", zap.Error(err)) | ||
bo := tikv.NewBackoffer(ctx, tikvRequestMaxBackoff) | ||
c.regionCache.OnSendFail(bo, rpcCtx, needReloadRegion(sri.failStoreIDs, rpcCtx), err) | ||
continue MainLoop |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if tikv is not active, will we fall into an endless loop?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
c.regionCache.OnSendFail(bo, rpcCtx, needReloadRegion(sri.failStoreIDs, rpcCtx), err)
has set this tikv as failed store
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What happens if the server is all down?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cdc retries to connect to tikv until some TiKVs are up and the cluster is available
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In fact this error only happens when gRPC connection is not established, which means at the CDC startup procedure.
What problem does this PR solve?
Fix #387
What is changed and how it works?
dispatchRequest
when getting stream client failed.Check List
Tests