Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion best-practices/tidb-best-practices.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ Lots of MySQL experience is also applicable to TiDB. It is noted that TiDB has i

* Ensure the order of results through indexes

You can use indexes to filter or sort data. Firstly, get row IDs according to the index order. Then, return the row content according to the return order of row IDs. In this way, the returned results are ordered according to the index column. It has been mentioned earlier that the model of scanning index and getting row is parallel + pipeline. If the row is returned according to the index order, a high concurrency between two queries does not reduce latency. Thus, the concurrency is low by default, but it can be modified through the [`tidb_index_serial_scan_concurrency`](/system-variables.md#tidb_index_serial_scan_concurrency) variable.
You can use indexes to filter or sort data. Firstly, get row IDs according to the index order. Then, return the row content according to the return order of row IDs. In this way, the returned results are ordered according to the index column. It has been mentioned earlier that the model of scanning index and getting row is parallel + pipeline. If the row is returned according to the index order, a high concurrency between two queries does not reduce latency. Thus, the concurrency is low by default, but you can increase it by adjusting [`tidb_executor_concurrency`](/system-variables.md#tidb_executor_concurrency-new-in-v50).

* Reverse index scan

Expand Down
4 changes: 2 additions & 2 deletions sql-statements/sql-statement-show-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ mysql> SHOW GLOBAL VARIABLES LIKE 'tidb%';
| tidb_backoff_weight | 2 |
| tidb_batch_commit | 0 |
| tidb_batch_delete | 0 |
| tidb_build_stats_concurrency | 4 |
| tidb_build_stats_concurrency | 2 |
| tidb_capture_plan_baselines | off |
| tidb_check_mb4_value_in_utf8 | 1 |
| tidb_checksum_table_concurrency | 4 |
Expand Down Expand Up @@ -173,4 +173,4 @@ The `SHOW [GLOBAL|SESSION] VARIABLES` statement in TiDB is fully compatible with

## See also

* [`SET [GLOBAL|SESSION]`](/sql-statements/sql-statement-set-variable.md)
* [`SET [GLOBAL|SESSION]`](/sql-statements/sql-statement-set-variable.md)
18 changes: 10 additions & 8 deletions statistics.md
Original file line number Diff line number Diff line change
Expand Up @@ -752,23 +752,25 @@

#### `tidb_build_stats_concurrency`

When you run the `ANALYZE` statement, the task is divided into multiple small tasks. Each task only works on statistics of one column or index. You can use the [`tidb_build_stats_concurrency`](/system-variables.md#tidb_build_stats_concurrency) variable to control the number of simultaneous small tasks. The default value is `2`. The default value is `4` for v7.4.0 and earlier versions.
Used to control the concurrency of table or partition analysis — for example, the number of partitions or table tasks that can be processed simultaneously. The default value is `2`. The default value is `4` for v7.4.0 and earlier versions.

Check failure on line 755 in statistics.md

View workflow job for this annotation

GitHub Actions / vale

[vale] reported by reviewdog 🐶 [PingCAP.EmDash] Don't put a space before or after a dash. Raw Output: {"message": "[PingCAP.EmDash] Don't put a space before or after a dash.", "location": {"path": "statistics.md", "range": {"start": {"line": 755, "column": 63}}}, "severity": "ERROR"}

#### `tidb_build_sampling_stats_concurrency`

When analyzing ordinary columns, you can use [`tidb_build_sampling_stats_concurrency`](/system-variables.md#tidb_build_sampling_stats_concurrency-new-in-v750) to control the concurrency of executing sampling tasks. The default value is `2`.
1. Used to control the concurrency level for merging samples collected from different regions.
2. Used to control the concurrency for special indexes (such as indexes on generated virtual columns), for example, the number of special indexes that can collect statistics simultaneously.

#### `tidb_analyze_partition_concurrency`
The default value is `2`.

When running the `ANALYZE` statement, you can use [`tidb_analyze_partition_concurrency`](/system-variables.md#tidb_analyze_partition_concurrency) to control the concurrency of reading and writing statistics for a partitioned table. The default value is `2`. The default value is `1` for v7.4.0 and earlier versions.
#### `tidb_analyze_partition_concurrency`

#### `tidb_distsql_scan_concurrency`
Used to control the concurrency for saving analyze results (writing TopN and histograms to system tables). The default value is `2`. The default value is `1` for v7.4.0 and earlier versions.

When you analyze regular columns, you can use the [`tidb_distsql_scan_concurrency`](/system-variables.md#tidb_distsql_scan_concurrency) variable to control the number of Regions to be read at one time. The default value is `15`. Note that changing the value will affect query performance. Adjust the value carefully.
#### `tidb_analyze_distsql_scan_concurrency`

#### `tidb_index_serial_scan_concurrency`
1. Used to control the concurrency of scanning TiKV regions.
2. Used to control the concurrency of scanning regions for special indexes (indexes generated from virtual columns).

When you analyze index columns, you can use the [`tidb_index_serial_scan_concurrency`](/system-variables.md#tidb_index_serial_scan_concurrency) variable to control the number of Regions to be read at one time. The default value is `1`. Note that changing the value will affect query performance. Adjust the value carefully.
The default value is `4`.

## See also

Expand Down
30 changes: 19 additions & 11 deletions system-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -1130,13 +1130,19 @@

### tidb_analyze_distsql_scan_concurrency <span class="version-mark">New in v7.6.0</span>

> **Note:**
>
> Before v7.6.0, it was controlled by the variable `tidb_distsql_scan_concurrency`.
> So please consider changing the value of `tidb_distsql_scan_concurrency` if you want to adjust the concurrency of scanning TiKV regions in versions earlier than v7.6.0.

- Scope: SESSION | GLOBAL
- Persists to cluster: Yes
- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No
- Type: Integer
- Default value: `4`
- Range: `[0, 4294967295]`. In versions earlier than v8.2.0, the minimum value is `1`. When you set it to `0`, it adaptively adjusts the concurrency based on the cluster size.
- This variable is used to set the concurrency of the `scan` operation when executing the `ANALYZE` operation.
- Used to control the concurrency of scanning TiKV regions.
- Used to control the concurrency of scanning regions for special indexes (indexes generated from virtual columns).

### tidb_analyze_partition_concurrency

Expand All @@ -1145,7 +1151,7 @@
- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No
- Default value: `2`. The default value is `1` for v7.4.0 and earlier versions.
- Range: `[1, 128]`. Before v8.4.0, the value range is `[1, 18446744073709551615]`.
- This variable specifies the concurrency for writing collected statistics when TiDB analyzes a partitioned table.
- For manual/auto analyze: used to control the concurrency for saving analyze results (writing TopN and histograms to system tables).

### tidb_analyze_version <span class="version-mark">New in v5.1.0</span>

Expand Down Expand Up @@ -1189,7 +1195,7 @@
+----------------------------------+
| @@tidb_analyze_skip_column_types |
+----------------------------------+
| json,blob,mediumblob,longblob |
| json,blob,mediumblob,longblob,text,mediumtext,longtext |
+----------------------------------+
1 row in set (0.00 sec)

Expand Down Expand Up @@ -1287,7 +1293,7 @@
- Type: Integer
- Default value: `1`
- Range: `[1, 256]`
- This variable is used to set the concurrency of executing the automatic update of statistics.
- For auto analyze: used to control the concurrency of table or partition analysis — for example, the number of partitions or table tasks that can be processed simultaneously.

Check failure on line 1296 in system-variables.md

View workflow job for this annotation

GitHub Actions / vale

[vale] reported by reviewdog 🐶 [PingCAP.EmDash] Don't put a space before or after a dash. Raw Output: {"message": "[PingCAP.EmDash] Don't put a space before or after a dash.", "location": {"path": "system-variables.md", "range": {"start": {"line": 1296, "column": 83}}}, "severity": "ERROR"}

### tidb_backoff_lock_fast

Expand Down Expand Up @@ -1400,8 +1406,7 @@
- Default value: `2`. The default value is `4` for v7.4.0 and earlier versions.
- Range: `[1, 256]`
- Unit: Threads
- This variable is used to set the concurrency of executing the `ANALYZE` statement.
- When the variable is set to a larger value, the execution performance of other queries is affected.
- For manual analyze: used to control the concurrency of table or partition analysis — for example, the number of partitions or table tasks that can be processed simultaneously.

Check failure on line 1409 in system-variables.md

View workflow job for this annotation

GitHub Actions / vale

[vale] reported by reviewdog 🐶 [PingCAP.EmDash] Don't put a space before or after a dash. Raw Output: {"message": "[PingCAP.EmDash] Don't put a space before or after a dash.", "location": {"path": "system-variables.md", "range": {"start": {"line": 1409, "column": 85}}}, "severity": "ERROR"}

### tidb_build_sampling_stats_concurrency <span class="version-mark">New in v7.5.0</span>

Expand All @@ -1412,8 +1417,8 @@
- Unit: Threads
- Default value: `2`
- Range: `[1, 256]`
- This variable is used to set the sampling concurrency in the `ANALYZE` process.
- When the variable is set to a larger value, the execution performance of other queries is affected.
- Used to control the concurrency level for merging samples collected from different regions.
- Used to control the concurrency for special indexes (such as indexes on generated virtual columns), for example, the number of indexes which can concurrently collect statistics.

### tidb_capture_plan_baselines <span class="version-mark">New in v4.0</span>

Expand Down Expand Up @@ -3506,15 +3511,18 @@

### tidb_index_serial_scan_concurrency

> **Warning:**
>
> This variable is deprecated and cannot be modified. Sequential index scans now follow [`tidb_executor_concurrency`](#tidb_executor_concurrency-new-in-v50), and [`ANALYZE TABLE`](/sql-statements/sql-statement-analyze-table.md) uses [`tidb_analyze_distsql_scan_concurrency`](#tidb_analyze_distsql_scan_concurrency-new-in-v760) to control index read concurrency.

- Scope: SESSION | GLOBAL
- Persists to cluster: Yes
- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): Yes
- Type: Integer
- Default value: `1`
- Range: `[1, 256]`
- Unit: Threads
- This variable is used to set the concurrency of the `serial scan` operation.
- Use a bigger value in OLAP scenarios, and a smaller value in OLTP scenarios.
- This variable remains only for backward compatibility. Adjust [`tidb_executor_concurrency`](#tidb_executor_concurrency-new-in-v50) to affect sequential index scans, or [`tidb_analyze_distsql_scan_concurrency`](#tidb_analyze_distsql_scan_concurrency-new-in-v760) to tune index statistics collection.

### tidb_init_chunk_size

Expand Down Expand Up @@ -4035,7 +4043,7 @@
- Persists to cluster: Yes
- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No
- Default value: `1`
- This variable specifies the concurrency of merging statistics for a partitioned table when TiDB analyzes the partitioned table.
- Used to control the concurrency for merging TopN results of partitioned tables.

### tidb_enable_async_merge_global_stats <span class="version-mark">New in v7.5.0</span>

Expand Down
4 changes: 2 additions & 2 deletions tidb-cloud/v8.1-performance-benchmarking-with-tpcc.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,8 +96,8 @@ This section introduces how to perform the TPC-C performance test step by step.

```sql
SET tidb_build_stats_concurrency=16;
SET tidb_distsql_scan_concurrency=16;
SET tidb_index_serial_scan_concurrency=16;
SET tidb_build_sampling_stats_concurrency = 16;
SET tidb_analyze_distsql_scan_concurrency=16;
```

5. Run the following `go-tpc tpcc` command to conduct stress tests on the TiDB Cloud Dedicated cluster. For each concurrency, the test takes two hours.
Expand Down
4 changes: 2 additions & 2 deletions tidb-cloud/v8.5-performance-benchmarking-with-tpcc.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,8 +96,8 @@ This section introduces how to perform the TPC-C performance test step by step.

```sql
SET tidb_build_stats_concurrency=16;
SET tidb_distsql_scan_concurrency=16;
SET tidb_index_serial_scan_concurrency=16;
SET tidb_build_sampling_stats_concurrency = 16;
SET tidb_analyze_distsql_scan_concurrency=16;
```

5. Run the following `go-tpc tpcc` command to conduct stress tests on the TiDB Cloud Dedicated cluster. For each concurrency, the test takes two hours.
Expand Down
3 changes: 1 addition & 2 deletions troubleshoot-tidb-oom.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,6 @@ You can take the following measures to reduce the memory usage of SQL statements

- Reduce the number of Regions to be read simultaneously or reduce the concurrency of operators to avoid memory problems caused by high concurrency. The corresponding system variables include:
- [`tidb_distsql_scan_concurrency`](/system-variables.md#tidb_distsql_scan_concurrency)
- [`tidb_index_serial_scan_concurrency`](/system-variables.md#tidb_index_serial_scan_concurrency)
- [`tidb_executor_concurrency`](/system-variables.md#tidb_executor_concurrency-new-in-v50)

- The concurrency of sessions is too high near the time point when the problem occurs. In this case, consider scaling out the TiDB cluster by adding more TiDB nodes.
Expand Down Expand Up @@ -195,4 +194,4 @@ To locate the root cause of an OOM issue, you need to collect the following info
## See also

- [TiDB Memory Control](/configure-memory-usage.md)
- [Tune TiKV Memory Parameter Performance](/tune-tikv-memory-performance.md)
- [Tune TiKV Memory Parameter Performance](/tune-tikv-memory-performance.md)