diff --git a/best-practices/tidb-best-practices.md b/best-practices/tidb-best-practices.md index c7cca03df2d46..5f1a025dd6b31 100644 --- a/best-practices/tidb-best-practices.md +++ b/best-practices/tidb-best-practices.md @@ -130,7 +130,7 @@ Lots of MySQL experience is also applicable to TiDB. It is noted that TiDB has i * Ensure the order of results through indexes - You can use indexes to filter or sort data. Firstly, get row IDs according to the index order. Then, return the row content according to the return order of row IDs. In this way, the returned results are ordered according to the index column. It has been mentioned earlier that the model of scanning index and getting row is parallel + pipeline. If the row is returned according to the index order, a high concurrency between two queries does not reduce latency. Thus, the concurrency is low by default, but it can be modified through the [`tidb_index_serial_scan_concurrency`](/system-variables.md#tidb_index_serial_scan_concurrency) variable. + You can use indexes to filter or sort data. Firstly, get row IDs according to the index order. Then, return the row content according to the return order of row IDs. In this way, the returned results are ordered according to the index column. It has been mentioned earlier that the model of scanning index and getting row is parallel + pipeline. If the row is returned according to the index order, a high concurrency between two queries does not reduce latency. Thus, the concurrency is low by default, but you can increase it by adjusting [`tidb_executor_concurrency`](/system-variables.md#tidb_executor_concurrency-new-in-v50). * Reverse index scan diff --git a/sql-statements/sql-statement-show-variables.md b/sql-statements/sql-statement-show-variables.md index fbf38a26abaaf..6c8e16bf51bac 100644 --- a/sql-statements/sql-statement-show-variables.md +++ b/sql-statements/sql-statement-show-variables.md @@ -37,7 +37,7 @@ mysql> SHOW GLOBAL VARIABLES LIKE 'tidb%'; | tidb_backoff_weight | 2 | | tidb_batch_commit | 0 | | tidb_batch_delete | 0 | -| tidb_build_stats_concurrency | 4 | +| tidb_build_stats_concurrency | 2 | | tidb_capture_plan_baselines | off | | tidb_check_mb4_value_in_utf8 | 1 | | tidb_checksum_table_concurrency | 4 | @@ -173,4 +173,4 @@ The `SHOW [GLOBAL|SESSION] VARIABLES` statement in TiDB is fully compatible with ## See also -* [`SET [GLOBAL|SESSION]`](/sql-statements/sql-statement-set-variable.md) \ No newline at end of file +* [`SET [GLOBAL|SESSION]`](/sql-statements/sql-statement-set-variable.md) diff --git a/statistics.md b/statistics.md index 11c4baee08105..260505f7ee2ca 100644 --- a/statistics.md +++ b/statistics.md @@ -752,23 +752,25 @@ The relationships of the relevant system variables are shown below: #### `tidb_build_stats_concurrency` -When you run the `ANALYZE` statement, the task is divided into multiple small tasks. Each task only works on statistics of one column or index. You can use the [`tidb_build_stats_concurrency`](/system-variables.md#tidb_build_stats_concurrency) variable to control the number of simultaneous small tasks. The default value is `2`. The default value is `4` for v7.4.0 and earlier versions. +Used to control the concurrency of table or partition analysis — for example, the number of partitions or table tasks that can be processed simultaneously. The default value is `2`. The default value is `4` for v7.4.0 and earlier versions. #### `tidb_build_sampling_stats_concurrency` -When analyzing ordinary columns, you can use [`tidb_build_sampling_stats_concurrency`](/system-variables.md#tidb_build_sampling_stats_concurrency-new-in-v750) to control the concurrency of executing sampling tasks. The default value is `2`. +1. Used to control the concurrency level for merging samples collected from different regions. +2. Used to control the concurrency for special indexes (such as indexes on generated virtual columns), for example, the number of special indexes that can collect statistics simultaneously. -#### `tidb_analyze_partition_concurrency` +The default value is `2`. -When running the `ANALYZE` statement, you can use [`tidb_analyze_partition_concurrency`](/system-variables.md#tidb_analyze_partition_concurrency) to control the concurrency of reading and writing statistics for a partitioned table. The default value is `2`. The default value is `1` for v7.4.0 and earlier versions. +#### `tidb_analyze_partition_concurrency` -#### `tidb_distsql_scan_concurrency` +Used to control the concurrency for saving analyze results (writing TopN and histograms to system tables). The default value is `2`. The default value is `1` for v7.4.0 and earlier versions. -When you analyze regular columns, you can use the [`tidb_distsql_scan_concurrency`](/system-variables.md#tidb_distsql_scan_concurrency) variable to control the number of Regions to be read at one time. The default value is `15`. Note that changing the value will affect query performance. Adjust the value carefully. +#### `tidb_analyze_distsql_scan_concurrency` -#### `tidb_index_serial_scan_concurrency` +1. Used to control the concurrency of scanning TiKV regions. +2. Used to control the concurrency of scanning regions for special indexes (indexes generated from virtual columns). -When you analyze index columns, you can use the [`tidb_index_serial_scan_concurrency`](/system-variables.md#tidb_index_serial_scan_concurrency) variable to control the number of Regions to be read at one time. The default value is `1`. Note that changing the value will affect query performance. Adjust the value carefully. +The default value is `4`. ## See also diff --git a/system-variables.md b/system-variables.md index 28a74eff6cb86..bf6b051da0533 100644 --- a/system-variables.md +++ b/system-variables.md @@ -1130,13 +1130,19 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a ### tidb_analyze_distsql_scan_concurrency New in v7.6.0 +> **Note:** +> +> Before v7.6.0, it was controlled by the variable `tidb_distsql_scan_concurrency`. +> So please consider changing the value of `tidb_distsql_scan_concurrency` if you want to adjust the concurrency of scanning TiKV regions in versions earlier than v7.6.0. + - Scope: SESSION | GLOBAL - Persists to cluster: Yes - Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No - Type: Integer - Default value: `4` - Range: `[0, 4294967295]`. In versions earlier than v8.2.0, the minimum value is `1`. When you set it to `0`, it adaptively adjusts the concurrency based on the cluster size. -- This variable is used to set the concurrency of the `scan` operation when executing the `ANALYZE` operation. +- Used to control the concurrency of scanning TiKV regions. +- Used to control the concurrency of scanning regions for special indexes (indexes generated from virtual columns). ### tidb_analyze_partition_concurrency @@ -1145,7 +1151,7 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a - Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No - Default value: `2`. The default value is `1` for v7.4.0 and earlier versions. - Range: `[1, 128]`. Before v8.4.0, the value range is `[1, 18446744073709551615]`. -- This variable specifies the concurrency for writing collected statistics when TiDB analyzes a partitioned table. +- For manual/auto analyze: used to control the concurrency for saving analyze results (writing TopN and histograms to system tables). ### tidb_analyze_version New in v5.1.0 @@ -1189,7 +1195,7 @@ mysql> SELECT @@tidb_analyze_skip_column_types; +----------------------------------+ | @@tidb_analyze_skip_column_types | +----------------------------------+ -| json,blob,mediumblob,longblob | +| json,blob,mediumblob,longblob,text,mediumtext,longtext | +----------------------------------+ 1 row in set (0.00 sec) @@ -1287,7 +1293,7 @@ mysql> SELECT job_info FROM mysql.analyze_jobs ORDER BY end_time DESC LIMIT 1; - Type: Integer - Default value: `1` - Range: `[1, 256]` -- This variable is used to set the concurrency of executing the automatic update of statistics. +- For auto analyze: used to control the concurrency of table or partition analysis — for example, the number of partitions or table tasks that can be processed simultaneously. ### tidb_backoff_lock_fast @@ -1400,8 +1406,7 @@ mysql> SELECT job_info FROM mysql.analyze_jobs ORDER BY end_time DESC LIMIT 1; - Default value: `2`. The default value is `4` for v7.4.0 and earlier versions. - Range: `[1, 256]` - Unit: Threads -- This variable is used to set the concurrency of executing the `ANALYZE` statement. -- When the variable is set to a larger value, the execution performance of other queries is affected. +- For manual analyze: used to control the concurrency of table or partition analysis — for example, the number of partitions or table tasks that can be processed simultaneously. ### tidb_build_sampling_stats_concurrency New in v7.5.0 @@ -1412,8 +1417,8 @@ mysql> SELECT job_info FROM mysql.analyze_jobs ORDER BY end_time DESC LIMIT 1; - Unit: Threads - Default value: `2` - Range: `[1, 256]` -- This variable is used to set the sampling concurrency in the `ANALYZE` process. -- When the variable is set to a larger value, the execution performance of other queries is affected. +- Used to control the concurrency level for merging samples collected from different regions. +- Used to control the concurrency for special indexes (such as indexes on generated virtual columns), for example, the number of indexes which can concurrently collect statistics. ### tidb_capture_plan_baselines New in v4.0 @@ -3506,6 +3511,10 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified ### tidb_index_serial_scan_concurrency +> **Warning:** +> +> This variable is deprecated and cannot be modified. Sequential index scans now follow [`tidb_executor_concurrency`](#tidb_executor_concurrency-new-in-v50), and [`ANALYZE TABLE`](/sql-statements/sql-statement-analyze-table.md) uses [`tidb_analyze_distsql_scan_concurrency`](#tidb_analyze_distsql_scan_concurrency-new-in-v760) to control index read concurrency. + - Scope: SESSION | GLOBAL - Persists to cluster: Yes - Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): Yes @@ -3513,8 +3522,7 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified - Default value: `1` - Range: `[1, 256]` - Unit: Threads -- This variable is used to set the concurrency of the `serial scan` operation. -- Use a bigger value in OLAP scenarios, and a smaller value in OLTP scenarios. +- This variable remains only for backward compatibility. Adjust [`tidb_executor_concurrency`](#tidb_executor_concurrency-new-in-v50) to affect sequential index scans, or [`tidb_analyze_distsql_scan_concurrency`](#tidb_analyze_distsql_scan_concurrency-new-in-v760) to tune index statistics collection. ### tidb_init_chunk_size @@ -4035,7 +4043,7 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified - Persists to cluster: Yes - Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No - Default value: `1` -- This variable specifies the concurrency of merging statistics for a partitioned table when TiDB analyzes the partitioned table. +- Used to control the concurrency for merging TopN results of partitioned tables. ### tidb_enable_async_merge_global_stats New in v7.5.0 diff --git a/tidb-cloud/v8.1-performance-benchmarking-with-tpcc.md b/tidb-cloud/v8.1-performance-benchmarking-with-tpcc.md index 8aad399883823..79e74aec086dd 100644 --- a/tidb-cloud/v8.1-performance-benchmarking-with-tpcc.md +++ b/tidb-cloud/v8.1-performance-benchmarking-with-tpcc.md @@ -96,8 +96,8 @@ This section introduces how to perform the TPC-C performance test step by step. ```sql SET tidb_build_stats_concurrency=16; - SET tidb_distsql_scan_concurrency=16; - SET tidb_index_serial_scan_concurrency=16; + SET tidb_build_sampling_stats_concurrency = 16; + SET tidb_analyze_distsql_scan_concurrency=16; ``` 5. Run the following `go-tpc tpcc` command to conduct stress tests on the TiDB Cloud Dedicated cluster. For each concurrency, the test takes two hours. diff --git a/tidb-cloud/v8.5-performance-benchmarking-with-tpcc.md b/tidb-cloud/v8.5-performance-benchmarking-with-tpcc.md index 53abd9096c0cb..3bd5bbd2a576a 100644 --- a/tidb-cloud/v8.5-performance-benchmarking-with-tpcc.md +++ b/tidb-cloud/v8.5-performance-benchmarking-with-tpcc.md @@ -96,8 +96,8 @@ This section introduces how to perform the TPC-C performance test step by step. ```sql SET tidb_build_stats_concurrency=16; - SET tidb_distsql_scan_concurrency=16; - SET tidb_index_serial_scan_concurrency=16; + SET tidb_build_sampling_stats_concurrency = 16; + SET tidb_analyze_distsql_scan_concurrency=16; ``` 5. Run the following `go-tpc tpcc` command to conduct stress tests on the TiDB Cloud Dedicated cluster. For each concurrency, the test takes two hours. diff --git a/troubleshoot-tidb-oom.md b/troubleshoot-tidb-oom.md index 1c2f102cd0681..c888f278f5e63 100644 --- a/troubleshoot-tidb-oom.md +++ b/troubleshoot-tidb-oom.md @@ -99,7 +99,6 @@ You can take the following measures to reduce the memory usage of SQL statements - Reduce the number of Regions to be read simultaneously or reduce the concurrency of operators to avoid memory problems caused by high concurrency. The corresponding system variables include: - [`tidb_distsql_scan_concurrency`](/system-variables.md#tidb_distsql_scan_concurrency) - - [`tidb_index_serial_scan_concurrency`](/system-variables.md#tidb_index_serial_scan_concurrency) - [`tidb_executor_concurrency`](/system-variables.md#tidb_executor_concurrency-new-in-v50) - The concurrency of sessions is too high near the time point when the problem occurs. In this case, consider scaling out the TiDB cluster by adding more TiDB nodes. @@ -195,4 +194,4 @@ To locate the root cause of an OOM issue, you need to collect the following info ## See also - [TiDB Memory Control](/configure-memory-usage.md) -- [Tune TiKV Memory Parameter Performance](/tune-tikv-memory-performance.md) \ No newline at end of file +- [Tune TiKV Memory Parameter Performance](/tune-tikv-memory-performance.md)