Skip to content

reference/system-databases: add 3 cluster tables (2) #2212

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Apr 21, 2020
Merged

reference/system-databases: add 3 cluster tables (2) #2212

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
36f0fcd
reference/system-databases: add 3 cluster tables (2)
TomShawn Apr 8, 2020
2df80fa
replace node with instance and address comments
TomShawn Apr 10, 2020
179c778
change field name
TomShawn Apr 10, 2020
9b374d2
Merge branch 'master' into cluster-table2
lilin90 Apr 14, 2020
2bb1c6e
add `sql` to results
TomShawn Apr 21, 2020
0ed1106
Merge branch 'master' into cluster-table2
lilin90 Apr 21, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions TOC.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -280,6 +280,9 @@
- [`cluster_info`](/reference/system-databases/cluster-info.md)
- [`cluster_hardware`](/reference/system-databases/cluster-hardware.md)
- [`cluster_config`](/reference/system-databases/cluster-config.md)
- [`cluster_load`](/reference/system-databases/cluster-load.md)
- [`cluster_systeminfo`](/reference/system-databases/cluster-systeminfo.md)
- [`cluster_log`](/reference/system-databases/cluster-log.md)
- [Errors Codes](/reference/error-codes.md)
- [Supported Client Drivers](/reference/supported-clients.md)
+ Garbage Collection (GC)
Expand Down
12 changes: 6 additions & 6 deletions reference/system-databases/cluster-config.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,15 @@ category: reference

# CLUSTER_CONFIG

You can use the `CLUSTER_CONFIG` cluster configuration table to get the current configuration of all TiDB/PD/TiKV nodes in the cluster. For TiDB versions earlier than 4.0, you need to access the HTTP API of each node one by one to collect all component configurations.
You can use the `CLUSTER_CONFIG` cluster configuration table to get the current configuration of all TiDB/PD/TiKV instances in the cluster. For TiDB versions earlier than 4.0, you need to access the HTTP API of each instance one by one to collect all component configurations.

{{< copyable "sql" >}}

```sql
desc cluster_config;
```

```
```sql
+----------+--------------+------+------+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+----------+--------------+------+------+---------+-------+
Expand All @@ -27,20 +27,20 @@ desc cluster_config;

Field description:

* `TYPE`: The node type. The optional values are `tidb`, `pd`, and `tikv`.
* `INSTANCE`: The service address of the node.
* `TYPE`: The instance type. The optional values are `tidb`, `pd`, and `tikv`.
* `INSTANCE`: The service address of the instance.
* `KEY`: The configuration item name.
* `VALUE`: The configuration item value.

The following example shows how to query the `coprocessor` configuration on the TiKV node using the `CLUSTER_CONFIG` table:
The following example shows how to query the `coprocessor` configuration on the TiKV instance using the `CLUSTER_CONFIG` table:

{{< copyable "sql" >}}

```sql
select * from cluster_config where type='tikv' and `key` like 'coprocessor%';
```

```
```sql
+------+-----------------+-----------------------------------+----------+
| TYPE | INSTANCE | KEY | VALUE |
+------+-----------------+-----------------------------------+----------+
Expand Down
8 changes: 4 additions & 4 deletions reference/system-databases/cluster-hardware.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,15 @@ category: reference

# CLUSTER_HARDWARE

The `CLUSTER_HARDWARE` hardware system table provides the hardware information of the server where each node of the cluster is located.
The `CLUSTER_HARDWARE` hardware system table provides the hardware information of the server where each instance of the cluster is located.

{{< copyable "sql" >}}

```sql
desc cluster_hardware;
```

```
```sql
+-------------+--------------+------+------+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+--------------+------+------+---------+-------+
Expand All @@ -30,7 +30,7 @@ desc cluster_hardware;
Field description:

* `TYPE`: Corresponds to the `TYPE` field in the [`information_schema.cluster_info`](/reference/system-databases/cluster-info.md) table. The optional values are `tidb`, `pd`, and `tikv`.
* `INSTANCE`: Corresponds to the `STATUS_ADDRESS` field in the [`information_schema.cluster_info`](/reference/system-databases/cluster-info.md) cluster information table.
* `INSTANCE`: Corresponds to the `INSTANCE` field in the [`information_schema.cluster_info`](/reference/system-databases/cluster-info.md) cluster information table.
* `DEVICE_TYPE`: Hardware type. Currently, you can query the `cpu`, `memory`, `disk`, and `net` types.
* `DEVICE_NAME`: Hardware name. The value of `DEVICE_NAME` varies with `DEVICE_TYPE`.
* `cpu`: The hardware name is cpu.
Expand All @@ -48,7 +48,7 @@ The following example shows how to query the CPU information using the `CLUSTER_
select * from cluster_hardware where device_type='cpu' and device_name='cpu' and name like '%cores';
```

```
```sql
+------+-----------------+-------------+-------------+--------------------+-------+
| TYPE | INSTANCE | DEVICE_TYPE | DEVICE_NAME | NAME | VALUE |
+------+-----------------+-------------+-------------+--------------------+-------+
Expand Down
16 changes: 8 additions & 8 deletions reference/system-databases/cluster-info.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,15 @@ category: reference

# CLUSTER_INFO

The `CLUSTER_INFO` cluster topology table provides the current topology information of the cluster, the version information of each node, the Git Hash corresponding to the node version, the starting time of each node, and the running time of each node.
The `CLUSTER_INFO` cluster topology table provides the current topology information of the cluster, the version information of each instance, the Git Hash corresponding to the instance version, the starting time of each instance, and the running time of each instance.

{{< copyable "sql" >}}

```sql
desc cluster_info;
```

```
```sql
+----------------+-------------+------+------+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+----------------+-------------+------+------+---------+-------+
Expand All @@ -31,21 +31,21 @@ desc cluster_info;

Field description:

* `TYPE`: The node type. The optional values are `tidb`, `pd`, and `tikv`.
* `TYPE`: The instance type. The optional values are `tidb`, `pd`, and `tikv`.
* `INSTANCE`: The instance address, which is a string in the format of `IP:PORT`.
* `STATUS_ADDRESS`: The service address of HTTP API. Some commands in tikv-ctl, pd-ctl, or tidb-ctl might use this API and this address. You can also get more cluster information via this address. Refer to [TiDB HTTP API document](https://github.com/pingcap/tidb/blob/master/docs/tidb_http_api.md) for details.
* `VERSION`: The semantic version number of the corresponding node. To be compatible with the MySQL version number, the TiDB version is displayed in the format of `${mysql-version}-${tidb-version}`.
* `GIT_HASH`: The Git Commit Hash when compiling the node version, which is used to identify whether two nodes are of the absolutely consistent version.
* `START_TIME`: The starting time of the corresponding node.
* `UPTIME`: The uptime of the corresponding node.
* `VERSION`: The semantic version number of the corresponding instance. To be compatible with the MySQL version number, the TiDB version is displayed in the format of `${mysql-version}-${tidb-version}`.
* `GIT_HASH`: The Git Commit Hash when compiling the instance version, which is used to identify whether two instances are of the absolutely consistent version.
* `START_TIME`: The starting time of the corresponding instance.
* `UPTIME`: The uptime of the corresponding instance.

{{< copyable "sql" >}}

```sql
select * from cluster_info;
```

```
```sql
+------+-----------------+-----------------+----------------------------------------+------------------------------------------+---------------------------+--------------+
| TYPE | INSTANCE | STATUS_ADDRESS | VERSION | GIT_HASH | START_TIME | UPTIME |
+------+-----------------+-----------------+----------------------------------------+------------------------------------------+---------------------------+--------------+
Expand Down
65 changes: 65 additions & 0 deletions reference/system-databases/cluster-load.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
---
title: CLUSTER_LOAD
summary: Learn the `CLUSTER_LOAD` cluster load table.
category: reference
---

# CLUSTER_LOAD

The `CLUSTER_LOAD` cluster load table provides the current load information of the server where each instance of the TiDB cluster is located.

{{< copyable "sql" >}}

```sql
desc cluster_load;
```

```sql
+-------------+--------------+------+------+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+--------------+------+------+---------+-------+
| TYPE | varchar(64) | YES | | NULL | |
| INSTANCE | varchar(64) | YES | | NULL | |
| DEVICE_TYPE | varchar(64) | YES | | NULL | |
| DEVICE_NAME | varchar(64) | YES | | NULL | |
| NAME | varchar(256) | YES | | NULL | |
| VALUE | varchar(128) | YES | | NULL | |
+-------------+--------------+------+------+---------+-------+
```

Field description:

* `TYPE`: Corresponds to the `TYPE` field in the [`information_schema.cluster_info`](/reference/system-databases/cluster-info.md) table. The optional values are `tidb`, `pd`, and `tikv`.
* `INSTANCE`: Corresponds to the `INSTANCE` field in the [`information_schema.cluster_info`](/reference/system-databases/cluster-info.md) cluster information table.
* `DEVICE_TYPE`: Hardware type. Currently, you can query the `cpu`, `memory`, `disk`, and `net` types.
* `DEVICE_NAME`: Hardware name. The value of `DEVICE_NAME` varies with `DEVICE_TYPE`.
* `cpu`: The hardware name is cpu.
* `disk`: The disk name.
* `net`: The network card name.
* `memory`: The hardware name is memory.
* `NAME`: Different load types. For example, cpu has three load types: `load1`, `load5`, and `load15`, which respectively mean the average load of cpu within 1 minute, 5 minutes, and 15 minutes.
* `VALUE`: The value of the hardware load. For example, `1min`, `5min`, and `15min` respectively mean the average load of the hardware within 1 minute, 5 minutes, and 15 minutes.

The following example shows how to query the current load information of cpu using the `CLUSTER_LOAD` table:

{{< copyable "sql" >}}

```sql
select * from cluster_load where device_type='cpu' and device_name='cpu';
```

```sql
+------+-----------------+-------------+-------------+--------+---------------+
| TYPE | INSTANCE | DEVICE_TYPE | DEVICE_NAME | NAME | VALUE |
+------+-----------------+-------------+-------------+--------+---------------+
| tidb | 127.0.0.1:10080 | cpu | cpu | load1 | 1.94 |
| tidb | 127.0.0.1:10080 | cpu | cpu | load5 | 2.16 |
| tidb | 127.0.0.1:10080 | cpu | cpu | load15 | 2.24 |
| pd | 127.0.0.1:2379 | cpu | cpu | load1 | 1.94 |
| pd | 127.0.0.1:2379 | cpu | cpu | load5 | 2.16 |
| pd | 127.0.0.1:2379 | cpu | cpu | load15 | 2.24 |
| tikv | 127.0.0.1:20160 | cpu | cpu | load1 | 1.94287109375 |
| tikv | 127.0.0.1:20160 | cpu | cpu | load5 | 2.15576171875 |
| tikv | 127.0.0.1:20160 | cpu | cpu | load15 | 2.2421875 |
+------+-----------------+-------------+-------------+--------+---------------+
```
73 changes: 73 additions & 0 deletions reference/system-databases/cluster-log.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
---
title: CLUSTER_LOG
summary: Learn the `CLUSTER_LOG` cluster log table.
category: reference
---

# CLUSTER_LOG

You can query cluster logs on the `CLUSTER_LOG` cluster log table. By pushing down query conditions to each instance, the impact of the query on cluster performance is less than that of the `grep` command.

To get the logs of the TiDB cluster before v4.0, you need to log in to each instance to summarize logs. This cluster log table in 4.0 provides the global and time-ordered log search result, which makes it easier to track full-link events. For example, by searching logs according to the `region id`, you can query all logs in the life cycle of this Region. Similarly, by searching the full link log through the slow log's `txn id`, you can query the flow and the number of keys scanned by this transaction at each instance.

{{< copyable "sql" >}}

```sql
desc cluster_log;
```

```sql
+----------+---------------------------+------+------+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+----------+---------------------------+------+------+---------+-------+
| TIME | varchar(32) | YES | | NULL | |
| TYPE | varchar(64) | YES | | NULL | |
| INSTANCE | varchar(64) | YES | | NULL | |
| LEVEL | varchar(8) | YES | | NULL | |
| MESSAGE | var_string(1024) unsigned | YES | | NULL | |
+----------+---------------------------+------+------+---------+-------+
5 rows in set (0.00 sec)
```

Field description:

* `TIME`: The time to print the log.
* `TYPE`: The instance type. The optional values are `tidb`, `pd`, and `tikv`.
* `INSTANCE`: The service address of the instance.
* `LEVEL`: The log level.
* `MESSAGE`: The log content.

> **Note:**
>
> + All fields of the cluster log table are pushed down to the corresponding instance for execution. So to reduce the overhead of using the cluster log table, specify as many conditions as possible. For example, the `select * from cluter_log where instance='tikv-1'` statement only executes the log search on the `tikv-1` instance.
>
> + The `message` field supports the `like` and `regexp` regular expressions, and the corresponding pattern is encoded as `regexp`. Specifying multiple `message` conditions is equivalent to the `pipeline` form of the `grep` command. For example, executing the `select * from cluster_log where message like 'coprocessor%' and message regexp '.*slow.*'` statement is equivalent to executing `grep 'coprocessor' xxx.log | grep -E '.*slow.*'` on all cluster instances.

The following example shows how to query the execution process of a DDL statement using the `CLUSTER_LOG` table:

{{< copyable "sql" >}}

```sql
select * from `CLUSTER_LOG` where message like '%ddl%' and message like '%job%58%' and type='tidb' and time > '2020-03-27 15:39:00';
```

```sql
+-------------------------+------+------------------+-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| TIME | TYPE | INSTANCE | LEVEL | MESSAGE |
+-------------------------+------+------------------+-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 2020/03/27 15:39:36.140 | tidb | 172.16.5.40:4008 | INFO | [ddl_worker.go:253] ["[ddl] add DDL jobs"] ["batch count"=1] [jobs="ID:58, Type:create table, State:none, SchemaState:none, SchemaID:1, TableID:57, RowCount:0, ArgLen:1, start time: 2020-03-27 15:39:36.129 +0800 CST, Err:<nil>, ErrCount:0, SnapshotVersion:0; "] |
| 2020/03/27 15:39:36.140 | tidb | 172.16.5.40:4008 | INFO | [ddl.go:457] ["[ddl] start DDL job"] [job="ID:58, Type:create table, State:none, SchemaState:none, SchemaID:1, TableID:57, RowCount:0, ArgLen:1, start time: 2020-03-27 15:39:36.129 +0800 CST, Err:<nil>, ErrCount:0, SnapshotVersion:0"] [query="create table t3 (a int, b int,c int)"] |
| 2020/03/27 15:39:36.879 | tidb | 172.16.5.40:4009 | INFO | [ddl_worker.go:554] ["[ddl] run DDL job"] [worker="worker 1, tp general"] [job="ID:58, Type:create table, State:none, SchemaState:none, SchemaID:1, TableID:57, RowCount:0, ArgLen:0, start time: 2020-03-27 15:39:36.129 +0800 CST, Err:<nil>, ErrCount:0, SnapshotVersion:0"] |
| 2020/03/27 15:39:36.936 | tidb | 172.16.5.40:4009 | INFO | [ddl_worker.go:739] ["[ddl] wait latest schema version changed"] [worker="worker 1, tp general"] [ver=35] ["take time"=52.165811ms] [job="ID:58, Type:create table, State:done, SchemaState:public, SchemaID:1, TableID:57, RowCount:0, ArgLen:1, start time: 2020-03-27 15:39:36.129 +0800 CST, Err:<nil>, ErrCount:0, SnapshotVersion:0"] |
| 2020/03/27 15:39:36.938 | tidb | 172.16.5.40:4009 | INFO | [ddl_worker.go:359] ["[ddl] finish DDL job"] [worker="worker 1, tp general"] [job="ID:58, Type:create table, State:synced, SchemaState:public, SchemaID:1, TableID:57, RowCount:0, ArgLen:0, start time: 2020-03-27 15:39:36.129 +0800 CST, Err:<nil>, ErrCount:0, SnapshotVersion:0"] |
| 2020/03/27 15:39:36.140 | tidb | 172.16.5.40:4009 | INFO | [ddl_worker.go:253] ["[ddl] add DDL jobs"] ["batch count"=1] [jobs="ID:58, Type:create table, State:none, SchemaState:none, SchemaID:1, TableID:57, RowCount:0, ArgLen:1, start time: 2020-03-27 15:39:36.129 +0800 CST, Err:<nil>, ErrCount:0, SnapshotVersion:0; "] |
| 2020/03/27 15:39:36.140 | tidb | 172.16.5.40:4009 | INFO | [ddl.go:457] ["[ddl] start DDL job"] [job="ID:58, Type:create table, State:none, SchemaState:none, SchemaID:1, TableID:57, RowCount:0, ArgLen:1, start time: 2020-03-27 15:39:36.129 +0800 CST, Err:<nil>, ErrCount:0, SnapshotVersion:0"] [query="create table t3 (a int, b int,c int)"] |
| 2020/03/27 15:39:37.141 | tidb | 172.16.5.40:4008 | INFO | [ddl.go:489] ["[ddl] DDL job is finished"] [jobID=58] |
+-------------------------+------+------------------+-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
```

The above query results show the following process:

1. The request with a DDL JOB ID of `58` is sent to the `172.16.5.40: 4008` TiDB instance.
2. The `172.16.5.40: 4009` TiDB instance processes this DDL request, which indicates that the `172.16.5.40: 4009` instance is the DDL owner at that time.
3. The request with a DDL JOB ID of `58` has been processed.
Loading