Do not merge until v8.5.4: distribute-table: support to scatter the region distribution of the given table and engine (#20345) #21883
+302
−0
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| --- | ||
| title: CANCEL DISTRIBUTION JOB | ||
| summary: An overview of the usage of CANCEL DISTRIBUTION JOB in TiDB. | ||
| --- | ||
|
|
||
| # CANCEL DISTRIBUTION JOB <span class="version-mark">New in v8.5.4</span> | ||
|
|
||
| The `CANCEL DISTRIBUTION JOB` statement is used to cancel a Region scheduling task created using the [`DISTRIBUTE TABLE`](/sql-statements/sql-statement-distribute-table.md) statement in TiDB. | ||
|
|
||
| <CustomContent platform="tidb-cloud"> | ||
|
|
||
| > **Note:** | ||
| > | ||
| > This feature is not available on [{{{ .starter }}}](https://docs.pingcap.com/tidbcloud/select-cluster-tier#tidb-cloud-serverless) and [{{{ .essential }}}](https://docs.pingcap.com/tidbcloud/select-cluster-tier#essential) clusters. | ||
|
|
||
| </CustomContent> | ||
|
|
||
| ## Synopsis | ||
|
|
||
| ```ebnf+diagram | ||
| CancelDistributionJobsStmt ::= | ||
| 'CANCEL' 'DISTRIBUTION' 'JOB' JobID | ||
| ``` | ||
|
|
||
| ## Examples | ||
|
|
||
| The following example cancels the distribution job with ID `1`: | ||
|
|
||
| ```sql | ||
| CANCEL DISTRIBUTION JOB 1; | ||
| ``` | ||
|
|
||
| The output is as follows: | ||
|
|
||
| ``` | ||
| Query OK, 0 rows affected (0.01 sec) | ||
| ``` | ||
|
|
||
| ## MySQL compatibility | ||
|
|
||
| This statement is a TiDB extension to MySQL syntax. | ||
|
|
||
| ## See also | ||
|
|
||
| * [`DISTRIBUTE TABLE`](/sql-statements/sql-statement-distribute-table.md) | ||
| * [`SHOW DISTRIBUTION JOBS`](/sql-statements/sql-statement-show-distribution-jobs.md) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,128 @@ | ||
| --- | ||
| title: DISTRIBUTE TABLE | ||
| summary: An overview of the usage of DISTRIBUTE TABLE for the TiDB database. | ||
| --- | ||
|
|
||
| # DISTRIBUTE TABLE <span class="version-mark">New in v8.5.4</span> | ||
|
|
||
| > **Warning:** | ||
| > | ||
| > This feature is experimental. It is not recommended that you use it in the production environment. This feature might be changed or removed without prior notice. If you find a bug, you can report an [issue](https://github.com/pingcap/tidb/issues) on GitHub. | ||
|
|
||
| <CustomContent platform="tidb-cloud"> | ||
|
|
||
| > **Note:** | ||
| > | ||
| > This feature is not available on [{{{ .starter }}}](https://docs.pingcap.com/tidbcloud/select-cluster-tier#tidb-cloud-serverless) and [{{{ .essential }}}](https://docs.pingcap.com/tidbcloud/select-cluster-tier#essential) clusters. | ||
|
|
||
| </CustomContent> | ||
|
|
||
| The `DISTRIBUTE TABLE` statement redistributes and reschedules Regions of a specified table to achieve a balanced distribution at the table level. Executing this statement helps prevent Regions from being concentrated on a few TiFlash or TiKV nodes, addressing the issue of uneven region distribution in the table. | ||
qiancai marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ## Synopsis | ||
|
|
||
| ```ebnf+diagram | ||
| DistributeTableStmt ::= | ||
| "DISTRIBUTE" "TABLE" TableName PartitionNameListOpt "RULE" EqOrAssignmentEq Identifier "ENGINE" EqOrAssignmentEq Identifier "TIMEOUT" EqOrAssignmentEq Identifier | ||
|
|
||
| TableName ::= | ||
| (SchemaName ".")? Identifier | ||
|
|
||
| PartitionNameList ::= | ||
| "PARTITION" "(" PartitionName ("," PartitionName)* ")" | ||
| ``` | ||
|
|
||
| ## Parameter description | ||
|
|
||
| When redistributing Regions in a table using the `DISTRIBUTE TABLE` statement, you can specify the storage engine (such as TiFlash or TiKV) and different Raft roles (such as Leader, Learner, or Voter) for balanced distribution. | ||
|
|
||
| - `RULE`: specifies which Raft role's Region to balance and schedule. Optional values are `"leader-scatter"`, `"peer-scatter"`, and `"learner-scatter"`. | ||
qiancai marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| - `ENGINE`: specifies the storage engine. Optional values are `"tikv"` and `"tiflash"`. | ||
| - `TIMEOUT`: specifies the timeout limit for the scatter operation. If PD does not complete the scatter within this time, the scatter task will automatically exit. When this parameter is not specified, the default value is `"30m"`. | ||
qiancai marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ## Examples | ||
|
|
||
| Redistribute the Regions of the Leaders in the table `t1` on TiKV: | ||
qiancai marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ```sql | ||
| CREATE TABLE t1 (a INT); | ||
| ... | ||
| DISTRIBUTE TABLE t1 RULE = "leader-scatter" ENGINE = "tikv" TIMEOUT = "1h"; | ||
| ``` | ||
|
|
||
| ``` | ||
| +--------+ | ||
| | JOB_ID | | ||
| +--------+ | ||
| | 100 | | ||
| +--------+ | ||
| ``` | ||
|
|
||
| Redistribute the Regions of the Learners in the table `t2` on TiFlash: | ||
|
|
||
| ```sql | ||
| CREATE TABLE t2 (a INT); | ||
| ... | ||
| DISTRIBUTE TABLE t2 RULE = "learner-scatter" ENGINE = "tiflash"; | ||
| ``` | ||
|
|
||
| ``` | ||
| +--------+ | ||
| | JOB_ID | | ||
| +--------+ | ||
| | 101 | | ||
| +--------+ | ||
| ``` | ||
|
|
||
| Redistribute the Regions of the Peers in the table `t3`'s `p1` and `p2` partitions on TiKV: | ||
|
|
||
| ```sql | ||
| CREATE TABLE t3 ( a INT, b INT, INDEX idx(b)) PARTITION BY RANGE( a ) ( | ||
| PARTITION p1 VALUES LESS THAN (10000), | ||
| PARTITION p2 VALUES LESS THAN (20000), | ||
| PARTITION p3 VALUES LESS THAN (MAXVALUE) ); | ||
| ... | ||
| DISTRIBUTE TABLE t3 PARTITION (p1, p2) RULE = "peer-scatter" ENGINE = "tikv"; | ||
| ``` | ||
|
|
||
| ``` | ||
| +--------+ | ||
| | JOB_ID | | ||
| +--------+ | ||
| | 102 | | ||
| +--------+ | ||
| ``` | ||
|
|
||
| Redistribute the Regions of the Learner in the table `t4`'s `p1` and `p2` partitions on TiFlash: | ||
|
|
||
| ```sql | ||
| CREATE TABLE t4 ( a INT, b INT, INDEX idx(b)) PARTITION BY RANGE( a ) ( | ||
| PARTITION p1 VALUES LESS THAN (10000), | ||
| PARTITION p2 VALUES LESS THAN (20000), | ||
| PARTITION p3 VALUES LESS THAN (MAXVALUE) ); | ||
| ... | ||
| DISTRIBUTE TABLE t4 PARTITION (p1, p2) RULE = "learner-scatter" ENGINE="tiflash"; | ||
| ``` | ||
|
|
||
| ``` | ||
| +--------+ | ||
| | JOB_ID | | ||
| +--------+ | ||
| | 103 | | ||
| +--------+ | ||
| ``` | ||
|
|
||
| ## Notes | ||
|
|
||
| When you execute the `DISTRIBUTE TABLE` statement to redistribute Regions of a table, the Region distribution result might be affected by the PD hotspot scheduler. After the redistribution, the Region distribution of this table might become imbalanced again over time. | ||
qiancai marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ## MySQL compatibility | ||
|
|
||
| This statement is a TiDB extension to MySQL syntax. | ||
|
|
||
| ## See also | ||
|
|
||
| - [`SHOW DISTRIBUTION JOBS`](/sql-statements/sql-statement-show-distribution-jobs.md) | ||
| - [`SHOW TABLE DISTRIBUTION`](/sql-statements/sql-statement-show-table-distribution.md) | ||
| - [`SHOW TABLE REGIONS`](/sql-statements/sql-statement-show-table-regions.md) | ||
| - [`CANCEL DISTRIBUTION JOB`](/sql-statements/sql-statement-cancel-distribution-job.md) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,51 @@ | ||||||
| --- | ||||||
| title: SHOW DISTRIBUTION JOBS | ||||||
| summary: An overview of the usage of SHOW DISTRIBUTION JOBS for the TiDB database. | ||||||
| --- | ||||||
|
|
||||||
| # SHOW DISTRIBUTION JOBS <span class="version-mark">New in v8.5.4</span> | ||||||
|
|
||||||
| The `SHOW DISTRIBUTION JOBS` statement shows all current Region distribution jobs. | ||||||
|
There was a problem hiding this comment. The description is slightly inaccurate. The statement shows all created jobs, including
Suggested change
Style Guide ReferencesFootnotes
|
||||||
|
|
||||||
| <CustomContent platform="tidb-cloud"> | ||||||
|
|
||||||
| > **Note:** | ||||||
| > | ||||||
| > This feature is not available on [{{{ .starter }}}](https://docs.pingcap.com/tidbcloud/select-cluster-tier#tidb-cloud-serverless) and [{{{ .essential }}}](https://docs.pingcap.com/tidbcloud/select-cluster-tier#essential) clusters. | ||||||
|
|
||||||
| </CustomContent> | ||||||
|
|
||||||
| ## Synopsis | ||||||
|
|
||||||
| ```ebnf+diagram | ||||||
| ShowDistributionJobsStmt ::= | ||||||
| "SHOW" "DISTRIBUTION" "JOBS" | ||||||
| ``` | ||||||
|
|
||||||
| ## Examples | ||||||
|
|
||||||
| Show all current Region distribution jobs: | ||||||
|
There was a problem hiding this comment. The word "current" is inaccurate here, as the command shows historical jobs as well. It's better to remove it.1
Suggested change
Style Guide ReferencesFootnotes
|
||||||
|
|
||||||
| ```sql | ||||||
| SHOW DISTRIBUTION JOBS; | ||||||
| ``` | ||||||
|
|
||||||
| ``` | ||||||
| +--------+----------+-------+----------------+--------+----------------+-----------+---------------------+---------------------+---------------------+ | ||||||
| | Job_ID | Database | Table | Partition_List | Engine | Rule | Status | Create_Time | Start_Time | Finish_Time | | ||||||
| +--------+----------+-------+----------------+--------+----------------+-----------+---------------------+---------------------+---------------------+ | ||||||
| | 100 | test | t1 | NULL | tikv | leader-scatter | finished | 2025-04-24 16:09:55 | 2025-04-24 16:09:55 | 2025-04-24 17:09:59 | | ||||||
| | 101 | test | t2 | NULL | tikv | learner-scatter| cancelled | 2025-05-08 15:33:29 | 2025-05-08 15:33:29 | 2025-05-08 15:33:37 | | ||||||
| | 102 | test | t5 | p1,p2 | tikv | peer-scatter | cancelled | 2025-05-21 15:32:44 | 2025-05-21 15:32:47 | 2025-05-21 15:32:47 | | ||||||
| +--------+----------+-------+----------------+--------+----------------+-----------+---------------------+---------------------+---------------------+ | ||||||
qiancai marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
| ``` | ||||||
|
|
||||||
| ## MySQL compatibility | ||||||
|
|
||||||
| This statement is a TiDB extension to MySQL syntax. | ||||||
|
|
||||||
| ## See also | ||||||
|
|
||||||
| - [`DISTRIBUTE TABLE`](/sql-statements/sql-statement-distribute-table.md) | ||||||
| - [`SHOW TABLE DISTRIBUTION`](/sql-statements/sql-statement-show-table-distribution.md) | ||||||
| - [`CANCEL DISTRIBUTION JOB`](/sql-statements/sql-statement-cancel-distribution-job.md) | ||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| --- | ||
| title: SHOW TABLE DISTRIBUTION | ||
| summary: An overview of the usage of SHOW TABLE DISTRIBUTION for the TiDB database. | ||
| --- | ||
|
|
||
| # SHOW TABLE DISTRIBUTION <span class="version-mark">New in v8.5.4</span> | ||
|
|
||
| The `SHOW TABLE DISTRIBUTION` statement shows the Region distribution information for a specified table. | ||
|
|
||
| <CustomContent platform="tidb-cloud"> | ||
|
|
||
| > **Note:** | ||
| > | ||
| > This feature is not available on [{{{ .starter }}}](https://docs.pingcap.com/tidbcloud/select-cluster-tier#tidb-cloud-serverless) and [{{{ .essential }}}](https://docs.pingcap.com/tidbcloud/select-cluster-tier#essential) clusters. | ||
|
|
||
| </CustomContent> | ||
|
|
||
| ## Synopsis | ||
|
|
||
| ```ebnf+diagram | ||
| ShowTableDistributionStmt ::= | ||
| "SHOW" "TABLE" TableName "DISTRIBUTIONS" | ||
|
|
||
| TableName ::= | ||
| (SchemaName ".")? Identifier | ||
| ``` | ||
|
|
||
| ## Examples | ||
|
|
||
| Show the Region distribution of the table `t`: | ||
|
|
||
| ```sql | ||
| CREATE TABLE `t` ( | ||
| `a` int DEFAULT NULL, | ||
| `b` int DEFAULT NULL, | ||
| KEY `idx` (`b`) | ||
| ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin | ||
| PARTITION BY RANGE (`a`) | ||
| (PARTITION `p1` VALUES LESS THAN (10000), | ||
| PARTITION `p2` VALUES LESS THAN (MAXVALUE)); | ||
| SHOW TABLE t DISTRIBUTIONS; | ||
| ``` | ||
|
|
||
| ``` | ||
| +----------------+----------+------------+---------------------+-------------------+--------------------+-------------------+--------------------+--------------------------+-------------------------+--------------------------+------------------------+-----------------------+------------------------+ | ||
| | PARTITION_NAME | STORE_ID | STORE_TYPE | REGION_LEADER_COUNT | REGION_PEER_COUNT | REGION_WRITE_BYTES | REGION_WRITE_KEYS | REGION_WRITE_QUERY | REGION_LEADER_READ_BYTES | REGION_LEADER_READ_KEYS | REGION_LEADER_READ_QUERY | REGION_PEER_READ_BYTES | REGION_PEER_READ_KEYS | REGION_PEER_READ_QUERY | | ||
| +----------------+----------+------------+---------------------+-------------------+--------------------+-------------------+--------------------+--------------------------+-------------------------+--------------------------+------------------------+-----------------------+------------------------+ | ||
| | p1 | 1 | tikv | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | ||
| | p1 | 15 | tikv | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | ||
| | p1 | 4 | tikv | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | ||
| | p1 | 5 | tikv | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | ||
| | p1 | 6 | tikv | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | ||
| | p2 | 1 | tikv | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | ||
| | p2 | 15 | tikv | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | ||
| | p2 | 4 | tikv | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | ||
| | p2 | 5 | tikv | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | ||
| | p2 | 6 | tikv | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | ||
| +----------------+----------+------------+---------------------+-------------------+--------------------+-------------------+--------------------+--------------------------+-------------------------+--------------------------+------------------------+-----------------------+------------------------+ | ||
qiancai marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| ``` | ||
|
|
||
| ## MySQL compatibility | ||
|
|
||
| This statement is a TiDB extension to MySQL syntax. | ||
|
|
||
| ## See also | ||
|
|
||
| - [`DISTRIBUTE TABLE`](/sql-statements/sql-statement-distribute-table.md) | ||
| - [`SHOW DISTRIBUTION JOBS`](/sql-statements/sql-statement-show-distribution-jobs.md) | ||
| - [`CANCEL DISTRIBUTION JOB`](/sql-statements/sql-statement-cancel-distribution-job.md) | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.