New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
client: Implement batch client #308
Conversation
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Co-Authored-By: amyangfei <amyangfei@gmail.com>
/run-all-tests |
2 similar comments
/run-all-tests |
/run-all-tests |
Codecov Report
@@ Coverage Diff @@
## master #308 +/- ##
==========================================
+ Coverage 28.39% 29.69% +1.29%
==========================================
Files 59 55 -4
Lines 5558 5261 -297
==========================================
- Hits 1578 1562 -16
+ Misses 3843 3560 -283
- Partials 137 139 +2
Continue to review full report at Codecov.
|
What problem does this PR solve?
This PR implements batch client to reuse one stream for multiple regions.
Fixes #292
Also fixes #303
What is changed and how it works?
This PR switches kvproto to https://github.com/5kbpers/kvproto/tree/cdc-batch-service . For each invoking to
EventFeed
, at most one stream will be connected to each TiKV instance.There will be one sender goroutine that dispatches requests to each stream, one receiver goroutine for each stream receives the events from it. However, different from the original design, there are still one goroutine for each region. The receiver goroutines doesn't process the events by themselves, instead, they dispatches the events to the goroutines for each region. It's because to maintain the region's states in a single goroutine is complicated, and to make it work as soon as possible, currently these goroutines are still kept. I'll continue working on further refactoring and optimizing.
In this PR, error handling still needs polishing.
Check List
Tests
Side effects