-
Notifications
You must be signed in to change notification settings - Fork 709
tutorial: add load-base-split #2933
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
d90f002
tutorial: add load-base-split
ran-huang b5d70e8
Update configure-load-base-split.md
ran-huang 7564efa
replace traffic with load
ran-huang c63dcc4
Apply suggestions from code review
ran-huang 0837f2c
Merge branch 'master' into load-base-split
lilin90 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,74 @@ | ||
| --- | ||
| title: Load Base Split | ||
| summary: Learn the feature of Load Base Split. | ||
| category: tutorials | ||
| --- | ||
|
|
||
| # Load Base Split | ||
|
|
||
| Load Base Split is a new feature introduced in TiDB 4.0. It aims to solve the hotspot issue caused by unbalanced access between Regions, such as full table scans for small tables. | ||
|
|
||
| ## Scenarios | ||
|
|
||
| In TiDB, it is easy to generate hotspots when the load is concentrated on certain nodes. PD tries to schedule the hot Regions so that they are distributed as evenly as possible across all nodes for better performance. | ||
|
|
||
| However, the minimum unit for PD scheduling is Region. If the number of hotspots in a cluster is smaller than the number of nodes, or if a few hotspots have far more load than other Regions, PD can only move the hotspot from one node to another, but not make the entire cluster share the load. | ||
|
|
||
| This scenario is especially common with workloads that are mostly read requests, such as full table scans and index lookups for small tables, or frequent access to some fields. | ||
|
|
||
| Previously, the solution to this problem was to manually execute a command to split one or more hotspot Regions, but this approach has two problems: | ||
|
|
||
| - Evenly splitting a Region is not always the best choice, because requests might be concentrated on a few keys. In such cases, hotspots might still be on one of the Regions after evenly splitting, and it might take multiple even splits to realize the goal. | ||
| - Human intervention is not timely or simple. | ||
|
|
||
| ## Implementation principles | ||
|
|
||
| Load Base Split automatically splits the Region based on statistics. It identifies the Regions whose read load consistently exceeds the threshold for 10 seconds, and splits these Regions at a proper position. When choosing the split position, Load Base Split tries to balance the access load of both Regions after the split and avoid access across Regions. | ||
|
|
||
| The Region split by Load Base Split will not be merged quickly. On the one hand, PD's `MergeChecker` skips the hot Regions; on the other hand, PD also determines whether to merge two Regions according to `QPS` in the heartbeat information, to avoid the merging of two Regions with high `QPS`. | ||
|
|
||
| ## Usage | ||
|
|
||
| The Load Base Split feature is currently controlled by the `split.qps-threshold` parameter. If the sum of all types of read requests per second for a Region exceeds the value of `split.qps-threshold` for 10 seconds on end, split the Region. | ||
|
|
||
| Load Base Split is enabled by default, but the parameter is set to a rather high value, defaulting to `3000`. If you want to disable this feature, set the threshold high enough. | ||
|
|
||
| To modify the parameter, take either of the following two methods: | ||
|
|
||
| - Use a SQL statement: | ||
|
|
||
| {{< copyable "sql" >}} | ||
|
|
||
| ```sql | ||
| set config tikv split.qps-threshold=3000 | ||
| ``` | ||
|
|
||
| - Use TiKV: | ||
|
|
||
| {{< copyable "shell-regular" >}} | ||
|
|
||
| ```shell | ||
| curl -X POST "http://ip:status_port/config" -H "accept: application/json" -d '{"split.qps-threshold":"3000"}' | ||
| ``` | ||
|
|
||
| Accordingly, you can view the configuration by either of the following two methods: | ||
|
|
||
| - Use a SQL statement: | ||
|
|
||
| {{< copyable "sql" >}} | ||
|
|
||
| ```sql | ||
| show config where type='tikv' and name like '%split.qps-threshold%' | ||
| ``` | ||
|
|
||
| - Use TiKV: | ||
|
|
||
| {{< copyable "shell-regular" >}} | ||
|
|
||
| ```shell | ||
| curl "http://ip:status_port/config" | ||
| ``` | ||
|
|
||
| > **Note:** | ||
| > | ||
| > Starting from v4.0.0-rc.2, you can modify and view the configuration using SQL statements. | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.