diff --git a/TOC.md b/TOC.md index 170c3135cd29a..b44afe2fd8159 100644 --- a/TOC.md +++ b/TOC.md @@ -280,6 +280,9 @@ - [`cluster_info`](/reference/system-databases/cluster-info.md) - [`cluster_hardware`](/reference/system-databases/cluster-hardware.md) - [`cluster_config`](/reference/system-databases/cluster-config.md) + - [`cluster_load`](/reference/system-databases/cluster-load.md) + - [`cluster_systeminfo`](/reference/system-databases/cluster-systeminfo.md) + - [`cluster_log`](/reference/system-databases/cluster-log.md) - [Errors Codes](/reference/error-codes.md) - [Supported Client Drivers](/reference/supported-clients.md) + Garbage Collection (GC) diff --git a/reference/system-databases/cluster-config.md b/reference/system-databases/cluster-config.md index 67902f44831df..e029923ef6fc3 100644 --- a/reference/system-databases/cluster-config.md +++ b/reference/system-databases/cluster-config.md @@ -6,7 +6,7 @@ category: reference # CLUSTER_CONFIG -You can use the `CLUSTER_CONFIG` cluster configuration table to get the current configuration of all TiDB/PD/TiKV nodes in the cluster. For TiDB versions earlier than 4.0, you need to access the HTTP API of each node one by one to collect all component configurations. +You can use the `CLUSTER_CONFIG` cluster configuration table to get the current configuration of all TiDB/PD/TiKV instances in the cluster. For TiDB versions earlier than 4.0, you need to access the HTTP API of each instance one by one to collect all component configurations. {{< copyable "sql" >}} @@ -14,7 +14,7 @@ You can use the `CLUSTER_CONFIG` cluster configuration table to get the current desc cluster_config; ``` -``` +```sql +----------+--------------+------+------+---------+-------+ | Field | Type | Null | Key | Default | Extra | +----------+--------------+------+------+---------+-------+ @@ -27,12 +27,12 @@ desc cluster_config; Field description: -* `TYPE`: The node type. The optional values are `tidb`, `pd`, and `tikv`. -* `INSTANCE`: The service address of the node. +* `TYPE`: The instance type. The optional values are `tidb`, `pd`, and `tikv`. +* `INSTANCE`: The service address of the instance. * `KEY`: The configuration item name. * `VALUE`: The configuration item value. -The following example shows how to query the `coprocessor` configuration on the TiKV node using the `CLUSTER_CONFIG` table: +The following example shows how to query the `coprocessor` configuration on the TiKV instance using the `CLUSTER_CONFIG` table: {{< copyable "sql" >}} @@ -40,7 +40,7 @@ The following example shows how to query the `coprocessor` configuration on the select * from cluster_config where type='tikv' and `key` like 'coprocessor%'; ``` -``` +```sql +------+-----------------+-----------------------------------+----------+ | TYPE | INSTANCE | KEY | VALUE | +------+-----------------+-----------------------------------+----------+ diff --git a/reference/system-databases/cluster-hardware.md b/reference/system-databases/cluster-hardware.md index 2dacd4ab71e5b..a5542498a849f 100644 --- a/reference/system-databases/cluster-hardware.md +++ b/reference/system-databases/cluster-hardware.md @@ -6,7 +6,7 @@ category: reference # CLUSTER_HARDWARE -The `CLUSTER_HARDWARE` hardware system table provides the hardware information of the server where each node of the cluster is located. +The `CLUSTER_HARDWARE` hardware system table provides the hardware information of the server where each instance of the cluster is located. {{< copyable "sql" >}} @@ -14,7 +14,7 @@ The `CLUSTER_HARDWARE` hardware system table provides the hardware information o desc cluster_hardware; ``` -``` +```sql +-------------+--------------+------+------+---------+-------+ | Field | Type | Null | Key | Default | Extra | +-------------+--------------+------+------+---------+-------+ @@ -30,7 +30,7 @@ desc cluster_hardware; Field description: * `TYPE`: Corresponds to the `TYPE` field in the [`information_schema.cluster_info`](/reference/system-databases/cluster-info.md) table. The optional values are `tidb`, `pd`, and `tikv`. -* `INSTANCE`: Corresponds to the `STATUS_ADDRESS` field in the [`information_schema.cluster_info`](/reference/system-databases/cluster-info.md) cluster information table. +* `INSTANCE`: Corresponds to the `INSTANCE` field in the [`information_schema.cluster_info`](/reference/system-databases/cluster-info.md) cluster information table. * `DEVICE_TYPE`: Hardware type. Currently, you can query the `cpu`, `memory`, `disk`, and `net` types. * `DEVICE_NAME`: Hardware name. The value of `DEVICE_NAME` varies with `DEVICE_TYPE`. * `cpu`: The hardware name is cpu. @@ -48,7 +48,7 @@ The following example shows how to query the CPU information using the `CLUSTER_ select * from cluster_hardware where device_type='cpu' and device_name='cpu' and name like '%cores'; ``` -``` +```sql +------+-----------------+-------------+-------------+--------------------+-------+ | TYPE | INSTANCE | DEVICE_TYPE | DEVICE_NAME | NAME | VALUE | +------+-----------------+-------------+-------------+--------------------+-------+ diff --git a/reference/system-databases/cluster-info.md b/reference/system-databases/cluster-info.md index 720975bec6b4b..7565ba7491762 100644 --- a/reference/system-databases/cluster-info.md +++ b/reference/system-databases/cluster-info.md @@ -6,7 +6,7 @@ category: reference # CLUSTER_INFO -The `CLUSTER_INFO` cluster topology table provides the current topology information of the cluster, the version information of each node, the Git Hash corresponding to the node version, the starting time of each node, and the running time of each node. +The `CLUSTER_INFO` cluster topology table provides the current topology information of the cluster, the version information of each instance, the Git Hash corresponding to the instance version, the starting time of each instance, and the running time of each instance. {{< copyable "sql" >}} @@ -14,7 +14,7 @@ The `CLUSTER_INFO` cluster topology table provides the current topology informat desc cluster_info; ``` -``` +```sql +----------------+-------------+------+------+---------+-------+ | Field | Type | Null | Key | Default | Extra | +----------------+-------------+------+------+---------+-------+ @@ -31,13 +31,13 @@ desc cluster_info; Field description: -* `TYPE`: The node type. The optional values are `tidb`, `pd`, and `tikv`. +* `TYPE`: The instance type. The optional values are `tidb`, `pd`, and `tikv`. * `INSTANCE`: The instance address, which is a string in the format of `IP:PORT`. * `STATUS_ADDRESS`: The service address of HTTP API. Some commands in tikv-ctl, pd-ctl, or tidb-ctl might use this API and this address. You can also get more cluster information via this address. Refer to [TiDB HTTP API document](https://github.com/pingcap/tidb/blob/master/docs/tidb_http_api.md) for details. -* `VERSION`: The semantic version number of the corresponding node. To be compatible with the MySQL version number, the TiDB version is displayed in the format of `${mysql-version}-${tidb-version}`. -* `GIT_HASH`: The Git Commit Hash when compiling the node version, which is used to identify whether two nodes are of the absolutely consistent version. -* `START_TIME`: The starting time of the corresponding node. -* `UPTIME`: The uptime of the corresponding node. +* `VERSION`: The semantic version number of the corresponding instance. To be compatible with the MySQL version number, the TiDB version is displayed in the format of `${mysql-version}-${tidb-version}`. +* `GIT_HASH`: The Git Commit Hash when compiling the instance version, which is used to identify whether two instances are of the absolutely consistent version. +* `START_TIME`: The starting time of the corresponding instance. +* `UPTIME`: The uptime of the corresponding instance. {{< copyable "sql" >}} @@ -45,7 +45,7 @@ Field description: select * from cluster_info; ``` -``` +```sql +------+-----------------+-----------------+----------------------------------------+------------------------------------------+---------------------------+--------------+ | TYPE | INSTANCE | STATUS_ADDRESS | VERSION | GIT_HASH | START_TIME | UPTIME | +------+-----------------+-----------------+----------------------------------------+------------------------------------------+---------------------------+--------------+ diff --git a/reference/system-databases/cluster-load.md b/reference/system-databases/cluster-load.md new file mode 100644 index 0000000000000..fd3bdb308ce01 --- /dev/null +++ b/reference/system-databases/cluster-load.md @@ -0,0 +1,65 @@ +--- +title: CLUSTER_LOAD +summary: Learn the `CLUSTER_LOAD` cluster load table. +category: reference +--- + +# CLUSTER_LOAD + +The `CLUSTER_LOAD` cluster load table provides the current load information of the server where each instance of the TiDB cluster is located. + +{{< copyable "sql" >}} + +```sql +desc cluster_load; +``` + +```sql ++-------------+--------------+------+------+---------+-------+ +| Field | Type | Null | Key | Default | Extra | ++-------------+--------------+------+------+---------+-------+ +| TYPE | varchar(64) | YES | | NULL | | +| INSTANCE | varchar(64) | YES | | NULL | | +| DEVICE_TYPE | varchar(64) | YES | | NULL | | +| DEVICE_NAME | varchar(64) | YES | | NULL | | +| NAME | varchar(256) | YES | | NULL | | +| VALUE | varchar(128) | YES | | NULL | | ++-------------+--------------+------+------+---------+-------+ +``` + +Field description: + +* `TYPE`: Corresponds to the `TYPE` field in the [`information_schema.cluster_info`](/reference/system-databases/cluster-info.md) table. The optional values are `tidb`, `pd`, and `tikv`. +* `INSTANCE`: Corresponds to the `INSTANCE` field in the [`information_schema.cluster_info`](/reference/system-databases/cluster-info.md) cluster information table. +* `DEVICE_TYPE`: Hardware type. Currently, you can query the `cpu`, `memory`, `disk`, and `net` types. +* `DEVICE_NAME`: Hardware name. The value of `DEVICE_NAME` varies with `DEVICE_TYPE`. + * `cpu`: The hardware name is cpu. + * `disk`: The disk name. + * `net`: The network card name. + * `memory`: The hardware name is memory. +* `NAME`: Different load types. For example, cpu has three load types: `load1`, `load5`, and `load15`, which respectively mean the average load of cpu within 1 minute, 5 minutes, and 15 minutes. +* `VALUE`: The value of the hardware load. For example, `1min`, `5min`, and `15min` respectively mean the average load of the hardware within 1 minute, 5 minutes, and 15 minutes. + +The following example shows how to query the current load information of cpu using the `CLUSTER_LOAD` table: + +{{< copyable "sql" >}} + +```sql +select * from cluster_load where device_type='cpu' and device_name='cpu'; +``` + +```sql ++------+-----------------+-------------+-------------+--------+---------------+ +| TYPE | INSTANCE | DEVICE_TYPE | DEVICE_NAME | NAME | VALUE | ++------+-----------------+-------------+-------------+--------+---------------+ +| tidb | 127.0.0.1:10080 | cpu | cpu | load1 | 1.94 | +| tidb | 127.0.0.1:10080 | cpu | cpu | load5 | 2.16 | +| tidb | 127.0.0.1:10080 | cpu | cpu | load15 | 2.24 | +| pd | 127.0.0.1:2379 | cpu | cpu | load1 | 1.94 | +| pd | 127.0.0.1:2379 | cpu | cpu | load5 | 2.16 | +| pd | 127.0.0.1:2379 | cpu | cpu | load15 | 2.24 | +| tikv | 127.0.0.1:20160 | cpu | cpu | load1 | 1.94287109375 | +| tikv | 127.0.0.1:20160 | cpu | cpu | load5 | 2.15576171875 | +| tikv | 127.0.0.1:20160 | cpu | cpu | load15 | 2.2421875 | ++------+-----------------+-------------+-------------+--------+---------------+ +``` diff --git a/reference/system-databases/cluster-log.md b/reference/system-databases/cluster-log.md new file mode 100644 index 0000000000000..1ce32bafeb5c7 --- /dev/null +++ b/reference/system-databases/cluster-log.md @@ -0,0 +1,73 @@ +--- +title: CLUSTER_LOG +summary: Learn the `CLUSTER_LOG` cluster log table. +category: reference +--- + +# CLUSTER_LOG + +You can query cluster logs on the `CLUSTER_LOG` cluster log table. By pushing down query conditions to each instance, the impact of the query on cluster performance is less than that of the `grep` command. + +To get the logs of the TiDB cluster before v4.0, you need to log in to each instance to summarize logs. This cluster log table in 4.0 provides the global and time-ordered log search result, which makes it easier to track full-link events. For example, by searching logs according to the `region id`, you can query all logs in the life cycle of this Region. Similarly, by searching the full link log through the slow log's `txn id`, you can query the flow and the number of keys scanned by this transaction at each instance. + +{{< copyable "sql" >}} + +```sql +desc cluster_log; +``` + +```sql ++----------+---------------------------+------+------+---------+-------+ +| Field | Type | Null | Key | Default | Extra | ++----------+---------------------------+------+------+---------+-------+ +| TIME | varchar(32) | YES | | NULL | | +| TYPE | varchar(64) | YES | | NULL | | +| INSTANCE | varchar(64) | YES | | NULL | | +| LEVEL | varchar(8) | YES | | NULL | | +| MESSAGE | var_string(1024) unsigned | YES | | NULL | | ++----------+---------------------------+------+------+---------+-------+ +5 rows in set (0.00 sec) +``` + +Field description: + +* `TIME`: The time to print the log. +* `TYPE`: The instance type. The optional values are `tidb`, `pd`, and `tikv`. +* `INSTANCE`: The service address of the instance. +* `LEVEL`: The log level. +* `MESSAGE`: The log content. + +> **Note:** +> +> + All fields of the cluster log table are pushed down to the corresponding instance for execution. So to reduce the overhead of using the cluster log table, specify as many conditions as possible. For example, the `select * from cluter_log where instance='tikv-1'` statement only executes the log search on the `tikv-1` instance. +> +> + The `message` field supports the `like` and `regexp` regular expressions, and the corresponding pattern is encoded as `regexp`. Specifying multiple `message` conditions is equivalent to the `pipeline` form of the `grep` command. For example, executing the `select * from cluster_log where message like 'coprocessor%' and message regexp '.*slow.*'` statement is equivalent to executing `grep 'coprocessor' xxx.log | grep -E '.*slow.*'` on all cluster instances. + +The following example shows how to query the execution process of a DDL statement using the `CLUSTER_LOG` table: + +{{< copyable "sql" >}} + +```sql +select * from `CLUSTER_LOG` where message like '%ddl%' and message like '%job%58%' and type='tidb' and time > '2020-03-27 15:39:00'; +``` + +```sql ++-------------------------+------+------------------+-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| TIME | TYPE | INSTANCE | LEVEL | MESSAGE | ++-------------------------+------+------------------+-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| 2020/03/27 15:39:36.140 | tidb | 172.16.5.40:4008 | INFO | [ddl_worker.go:253] ["[ddl] add DDL jobs"] ["batch count"=1] [jobs="ID:58, Type:create table, State:none, SchemaState:none, SchemaID:1, TableID:57, RowCount:0, ArgLen:1, start time: 2020-03-27 15:39:36.129 +0800 CST, Err:, ErrCount:0, SnapshotVersion:0; "] | +| 2020/03/27 15:39:36.140 | tidb | 172.16.5.40:4008 | INFO | [ddl.go:457] ["[ddl] start DDL job"] [job="ID:58, Type:create table, State:none, SchemaState:none, SchemaID:1, TableID:57, RowCount:0, ArgLen:1, start time: 2020-03-27 15:39:36.129 +0800 CST, Err:, ErrCount:0, SnapshotVersion:0"] [query="create table t3 (a int, b int,c int)"] | +| 2020/03/27 15:39:36.879 | tidb | 172.16.5.40:4009 | INFO | [ddl_worker.go:554] ["[ddl] run DDL job"] [worker="worker 1, tp general"] [job="ID:58, Type:create table, State:none, SchemaState:none, SchemaID:1, TableID:57, RowCount:0, ArgLen:0, start time: 2020-03-27 15:39:36.129 +0800 CST, Err:, ErrCount:0, SnapshotVersion:0"] | +| 2020/03/27 15:39:36.936 | tidb | 172.16.5.40:4009 | INFO | [ddl_worker.go:739] ["[ddl] wait latest schema version changed"] [worker="worker 1, tp general"] [ver=35] ["take time"=52.165811ms] [job="ID:58, Type:create table, State:done, SchemaState:public, SchemaID:1, TableID:57, RowCount:0, ArgLen:1, start time: 2020-03-27 15:39:36.129 +0800 CST, Err:, ErrCount:0, SnapshotVersion:0"] | +| 2020/03/27 15:39:36.938 | tidb | 172.16.5.40:4009 | INFO | [ddl_worker.go:359] ["[ddl] finish DDL job"] [worker="worker 1, tp general"] [job="ID:58, Type:create table, State:synced, SchemaState:public, SchemaID:1, TableID:57, RowCount:0, ArgLen:0, start time: 2020-03-27 15:39:36.129 +0800 CST, Err:, ErrCount:0, SnapshotVersion:0"] | +| 2020/03/27 15:39:36.140 | tidb | 172.16.5.40:4009 | INFO | [ddl_worker.go:253] ["[ddl] add DDL jobs"] ["batch count"=1] [jobs="ID:58, Type:create table, State:none, SchemaState:none, SchemaID:1, TableID:57, RowCount:0, ArgLen:1, start time: 2020-03-27 15:39:36.129 +0800 CST, Err:, ErrCount:0, SnapshotVersion:0; "] | +| 2020/03/27 15:39:36.140 | tidb | 172.16.5.40:4009 | INFO | [ddl.go:457] ["[ddl] start DDL job"] [job="ID:58, Type:create table, State:none, SchemaState:none, SchemaID:1, TableID:57, RowCount:0, ArgLen:1, start time: 2020-03-27 15:39:36.129 +0800 CST, Err:, ErrCount:0, SnapshotVersion:0"] [query="create table t3 (a int, b int,c int)"] | +| 2020/03/27 15:39:37.141 | tidb | 172.16.5.40:4008 | INFO | [ddl.go:489] ["[ddl] DDL job is finished"] [jobID=58] | ++-------------------------+------+------------------+-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +``` + +The above query results show the following process: + +1. The request with a DDL JOB ID of `58` is sent to the `172.16.5.40: 4008` TiDB instance. +2. The `172.16.5.40: 4009` TiDB instance processes this DDL request, which indicates that the `172.16.5.40: 4009` instance is the DDL owner at that time. +3. The request with a DDL JOB ID of `58` has been processed. diff --git a/reference/system-databases/cluster-systeminfo.md b/reference/system-databases/cluster-systeminfo.md new file mode 100644 index 0000000000000..d44dcdc540eb5 --- /dev/null +++ b/reference/system-databases/cluster-systeminfo.md @@ -0,0 +1,54 @@ +--- +title: CLUSTER_SYSTEMINFO +summary: Learn the `CLUSTER_SYSTEMINFO` kernel parameter table. +category: reference +--- + +# CLUSTER_SYSTEMINFO + +You can use the `CLUSTER_SYSTEMINFO` kernel parameter table to query the kernel configuration information of the server where all nodes of the cluster are located. Currently, you can query the information of the `sysctl` system. + +{{< copyable "sql" >}} + +```sql +desc cluster_systeminfo; +``` + +```sql ++-------------+--------------+------+------+---------+-------+ +| Field | Type | Null | Key | Default | Extra | ++-------------+--------------+------+------+---------+-------+ +| TYPE | varchar(64) | YES | | NULL | | +| INSTANCE | varchar(64) | YES | | NULL | | +| SYSTEM_TYPE | varchar(64) | YES | | NULL | | +| SYSTEM_NAME | varchar(64) | YES | | NULL | | +| NAME | varchar(256) | YES | | NULL | | +| VALUE | varchar(128) | YES | | NULL | | ++-------------+--------------+------+------+---------+-------+ +6 rows in set (0.00 sec) +``` + +Field description: + +* `TYPE`: Corresponds to the `TYPE` field in the [`information_schema.cluster_info`](/reference/system-databases/cluster-info.md) table. The optional values are `tidb`, `pd`, and `tikv`. +* `INSTANCE`: Corresponds to the `INSTANCE` field in the [`information_schema.cluster_info`](/reference/system-databases/cluster-info.md) cluster information table. +* `SYSTEM_TYPE`: The system type. Currently, you can query the `system` system type. +* `SYSTEM_NAME`: The system name. Currently, you can query the `sysctl` system name. +* `NAME`: The configuration name corresponding to `sysctl`. +* `VALUE`: The value of the configuration item corresponding to `sysctl`. + +The following example shows how to query the kernel version of all servers in the cluster using the `CLUSTER_SYSTEMINFO` system information table. + +```sql +select * from CLUSTER_SYSTEMINFO where name like '%kernel.osrelease%' +``` + +```sql ++------+-------------------+-------------+-------------+------------------+----------------------------+ +| TYPE | INSTANCE | SYSTEM_TYPE | SYSTEM_NAME | NAME | VALUE | ++------+-------------------+-------------+-------------+------------------+----------------------------+ +| tidb | 172.16.5.40:4008 | system | sysctl | kernel.osrelease | 3.10.0-862.14.4.el7.x86_64 | +| pd | 172.16.5.40:20379 | system | sysctl | kernel.osrelease | 3.10.0-862.14.4.el7.x86_64 | +| tikv | 172.16.5.40:21150 | system | sysctl | kernel.osrelease | 3.10.0-862.14.4.el7.x86_64 | ++------+-------------------+-------------+-------------+------------------+----------------------------+ +``` diff --git a/reference/system-databases/sql-diagnosis.md b/reference/system-databases/sql-diagnosis.md index 6765bf556f496..ad97e99453c05 100644 --- a/reference/system-databases/sql-diagnosis.md +++ b/reference/system-databases/sql-diagnosis.md @@ -32,9 +32,9 @@ The cluster information tables bring together the information of all nodes and i + From the cluster topology table [`information_schema.cluster_info`](/reference/system-databases/cluster-info.md), you can get the current topology information of the cluster, the version of each node, the Git Hash corresponding to the version, the starting time of each node, and the running time of each node. + From the cluster configuration table [`information_schema.cluster_config`](/reference/system-databases/cluster-config.md), you can get the configuration of all nodes in the cluster. For versions earlier than 4.0, you need to access the HTTP API of each node one by one to get these configuration information. + On the cluster hardware table [`information_schema.cluster_hardware`](/reference/system-databases/cluster-hardware.md), you can quickly query the cluster hardware information. -+ On the cluster load table `information_schema.cluster_load`, you can query the load information of different nodes and hardware types of the cluster. -+ On the kernel parameter table `information_schema.cluster_systeminfo`, you can query the kernel configuration information of different nodes in the cluster. Currently, TiDB supports querying the sysctl information. -+ On the cluster log table `information_schema.cluster_log`, you can query cluster logs. By pushing down query conditions to each node, the impact of the query on cluster performance is less than that of the `grep` command. ++ On the cluster load table [`information_schema.cluster_load`](/reference/system-databases/cluster-load.md), you can query the load information of different nodes and hardware types of the cluster. ++ On the kernel parameter table [`information_schema.cluster_systeminfo`](/reference/system-databases/cluster-systeminfo.md), you can query the kernel configuration information of different nodes in the cluster. Currently, TiDB supports querying the sysctl information. ++ On the cluster log table [`information_schema.cluster_log`](/reference/system-databases/cluster-log.md), you can query cluster logs. By pushing down query conditions to each node, the impact of the query on cluster performance is less than that of the `grep` command. On the system tables earlier than TiDB v4.0, you can only view the current node. TiDB v4.0 introduces the corresponding cluster tables and you can have a global view of the entire cluster on a single TiDB node. These tables are currently in `information_schema`, and the query method is the same as other `information_schema` system tables.