Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tracking Issue for Load Base Split Improvement #12063

Closed
11 of 13 tasks
JmPotato opened this issue Mar 2, 2022 · 1 comment
Closed
11 of 13 tasks

Tracking Issue for Load Base Split Improvement #12063

JmPotato opened this issue Mar 2, 2022 · 1 comment
Assignees
@JmPotato
Labels
type/enhancement Type: Issue - Enhancement

Comments

@JmPotato
Copy link
Member

JmPotato commented Mar 2, 2022 •
edited

As mentioned in tikv/pd#4588, Load Base Split can not work well in most or pure scan scenarios. We need to introduce more statistical dimensions to the Load Base Split decision process, such as statistics related to the read CPU usage. This is the tracking issue for some related work.

Steps

Unresolved Questions

  • Observer & Metrics: We need to settle a way to observe the Load Base Split behavior to help us debug and check the split results.
  • Cost Model: A cost model is needed to evaluate and determine the split effect between the read pool and gRPC poll CPU.
  • Switch & Mechanism: What kind of switch or mechanism it should be like to control the CPU usage collecting?
  • Feedback Functionality: Provide feedback functionality for TiKV pd#5267
@JmPotato JmPotato added the type/enhancement Type: Issue - Enhancement label Mar 2, 2022
@JmPotato
Copy link
Member Author

JmPotato commented Mar 2, 2022 •
edited

/assign JmPotato

PTAL @lhy1024 @nolouch

@JmPotato JmPotato mentioned this issue Mar 9, 2022
Open
This was referenced May 20, 2022
Closed
Merged
ti-chi-bot added a commit that referenced this issue May 25, 2022
@JmPotato @ti-chi-bot
raftstore: refine some code of the PD and SplitController (#12639)
2ffb7cf 
ref #12063

raftstore: refine some code of the PD and SplitController

Signed-off-by: JmPotato <ghzpotato@gmail.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
YuJuncen pushed a commit to YuJuncen/tikv that referenced this issue May 26, 2022
@JmPotato @ti-chi-bot @YuJuncen
raftstore: refine some code of the PD and SplitController (tikv#12639)
f7657e7 
ref tikv#12063

raftstore: refine some code of the PD and SplitController

Signed-off-by: JmPotato <ghzpotato@gmail.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
Signed-off-by: Yu Juncen <yujuncen@pingcap.com>
This was referenced Jun 30, 2022
Merged
Merged
ti-chi-bot pushed a commit that referenced this issue Jul 5, 2022
@JmPotato
pd_worker, split_controller: introduce the new config and CPU collect…
16589c0 
…or registration mechanism (#12942)

ref #12063, ref #12593

Introduce the new split config and CPU collector registration mechanism.

Signed-off-by: JmPotato <ghzpotato@gmail.com>
ti-chi-bot added a commit that referenced this issue Jul 6, 2022
@JmPotato @ti-chi-bot
raftstore: introduce the CPU-based Load Base Split strategy (#12955)
07e7cd4 
ref #12063, ref #12593, ref #12942

Introduce the CPU-based Load Base Split strategy.

Signed-off-by: JmPotato <ghzpotato@gmail.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
@JmPotato JmPotato mentioned this issue Jul 6, 2022
Merged
ti-chi-bot added a commit that referenced this issue Jul 7, 2022
@JmPotato @ti-chi-bot
raftstore: add some test cases for the CPU-based Load Base Split stra…
2ca69a5 
…tegy (#12969)

ref #12063

Add some test cases for the CPU-based Load Base Split strategy.

Signed-off-by: JmPotato <ghzpotato@gmail.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
@JmPotato JmPotato mentioned this issue Jul 11, 2022
Merged
ti-chi-bot pushed a commit that referenced this issue Jul 11, 2022
@JmPotato
raftstore: update the Load Base Split metrics event (#12992)
5c94158 
ref #12063

Update the Load Base Split metrics event to distinguish more cases.

Signed-off-by: JmPotato <ghzpotato@gmail.com>
This was referenced Jul 12, 2022
Merged
Merged
ti-chi-bot added a commit that referenced this issue Jul 13, 2022
@JmPotato @ti-chi-bot
raftstore: fix the building warning caused by the feature cfg (#13004)
2edf25d 
ref #12063

Fix the building warning caused by the feature cfg.

Signed-off-by: JmPotato <ghzpotato@gmail.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
ti-chi-bot pushed a commit that referenced this issue Jul 13, 2022
@JmPotato
raftstore: make the gRPC poll busy check to consider the average value (
4f9a528 
#13005)

ref #12063

Make the gRPC poll busy check to consider the average value to make sure the check is accurate.

Signed-off-by: JmPotato <ghzpotato@gmail.com>
ekexium pushed a commit to ekexium/tikv that referenced this issue Jul 13, 2022
@JmPotato @ekexium
pd_worker, split_controller: introduce the new config and CPU collect…
5fd9234 
…or registration mechanism (tikv#12942)

ref tikv#12063, ref tikv#12593

Introduce the new split config and CPU collector registration mechanism.

Signed-off-by: JmPotato <ghzpotato@gmail.com>
ekexium pushed a commit to ekexium/tikv that referenced this issue Jul 13, 2022
@JmPotato @ti-chi-bot @ekexium
raftstore: introduce the CPU-based Load Base Split strategy (tikv#12955)
6b20561 
ref tikv#12063, ref tikv#12593, ref tikv#12942

Introduce the CPU-based Load Base Split strategy.

Signed-off-by: JmPotato <ghzpotato@gmail.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
ekexium pushed a commit to ekexium/tikv that referenced this issue Jul 13, 2022
@JmPotato @ti-chi-bot @ekexium
raftstore: add some test cases for the CPU-based Load Base Split stra…
3b041c3 
…tegy (tikv#12969)

ref tikv#12063

Add some test cases for the CPU-based Load Base Split strategy.

Signed-off-by: JmPotato <ghzpotato@gmail.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
This was referenced Jul 14, 2022
Merged
Merged
ti-chi-bot added a commit that referenced this issue Jul 15, 2022
@JmPotato @ti-chi-bot
raftstore: only record UNABLE_TO_SPLIT_CPU_TOP when the top_cpu_usage…
5ae20e2 
… is not empty (#13016)

ref #12063

Only record `UNABLE_TO_SPLIT_CPU_TOP` when the `top_cpu_usage` is not empty.

Signed-off-by: JmPotato <ghzpotato@gmail.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
@JmPotato JmPotato mentioned this issue Nov 9, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@JmPotato JmPotato

Labels
type/enhancement Type: Issue - Enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@JmPotato