Skip to content

Commit

Permalink
cloud: add SQL and TiFlash related docs (#9001) (#9479)
Browse files Browse the repository at this point in the history
  • Loading branch information
ti-chi-bot committed Jul 21, 2022
1 parent bf23bec commit 81f07d1
Show file tree
Hide file tree
Showing 67 changed files with 1,466 additions and 110 deletions.
295 changes: 289 additions & 6 deletions TOC-tidb-cloud.md

Large diffs are not rendered by default.

14 changes: 12 additions & 2 deletions agg-distinct-optimization.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,19 @@ mysql> explain SELECT DISTINCT a from t;

Usually, aggregate functions with the `DISTINCT` option is executed in the TiDB layer in a single-threaded execution model.

The [`tidb_opt_distinct_agg_push_down`](/system-variables.md#tidb_opt_distinct_agg_push_down) system variable or the [`distinct-agg-push-down`](/tidb-configuration-file.md#distinct-agg-push-down) configuration item in TiDB controls whether to rewrite the distinct aggregate queries and push them to the TiKV/TiFlash Coprocessor.
<CustomContent platform="tidb">

Take the following queries as an example of this optimization. `tidb_opt_distinct_agg_push_down` is disabled by default, which means the aggregate functions are executed in the TiDB layer. After enabling this optimization by setting its value to `1`, the `distinct a` part of `count(distinct a)` is pushed to TiKV/TiFlash Coprocessor: there is a HashAgg_5 to remove the duplicated values on column a in the TiKV Coprocessor. It might reduce the computation overhead of `HashAgg_8` in the TiDB layer.
The [`tidb_opt_distinct_agg_push_down`](/system-variables.md#tidb_opt_distinct_agg_push_down) system variable or the [`distinct-agg-push-down`](/tidb-configuration-file.md#distinct-agg-push-down) configuration item in TiDB controls whether to rewrite the distinct aggregate queries and push them to the TiKV or TiFlash Coprocessor.

</CustomContent>

<CustomContent platform="tidb-cloud">

The [`tidb_opt_distinct_agg_push_down`](/system-variables.md#tidb_opt_distinct_agg_push_down) system variable in TiDB controls whether to rewrite the distinct aggregate queries and push them to the TiKV or TiFlash Coprocessor.

</CustomContent>

Take the following queries as an example of this optimization. `tidb_opt_distinct_agg_push_down` is disabled by default, which means the aggregate functions are executed in the TiDB layer. After enabling this optimization by setting its value to `1`, the `distinct a` part of `count(distinct a)` is pushed to TiKV or TiFlash Coprocessor: there is a HashAgg_5 to remove the duplicated values on column a in the TiKV Coprocessor. It might reduce the computation overhead of `HashAgg_8` in the TiDB layer.

```sql
mysql> desc select count(distinct a) from test.t;
Expand Down
12 changes: 12 additions & 0 deletions auto-increment.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,22 @@ summary: Learn the `AUTO_INCREMENT` column attribute of TiDB.

This document introduces the `AUTO_INCREMENT` column attribute, including its concept, implementation principles, auto-increment related features, and restrictions.

<CustomContent platform="tidb">

> **Note:**
>
> The `AUTO_INCREMENT` attribute might cause hotspot in production environments. See [Troubleshoot HotSpot Issues](/troubleshoot-hot-spot-issues.md) for details. It is recommended to use [`AUTO_RANDOM`](/auto-random.md) instead.
</CustomContent>

<CustomContent platform="tidb-cloud">

> **Note:**
>
> The `AUTO_INCREMENT` attribute might cause hotspot in production environments. See [Troubleshoot HotSpot Issues](https://docs.pingcap.com/tidb/stable/troubleshoot-hot-spot-issues#handle-auto-increment-primary-key-hotspot-tables-using-auto_random) for details. It is recommended to use [`AUTO_RANDOM`](/auto-random.md) instead.
</CustomContent>

## Concept

`AUTO_INCREMENT` is a column attribute that is used to automatically fill in default column values. When the `INSERT` statement does not specify values for the `AUTO_INCREMENT` column, the system automatically assigns values to this column.
Expand Down
10 changes: 8 additions & 2 deletions auto-random.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,17 @@ summary: Learn the AUTO_RANDOM attribute.

> **Note:**
>
> `AUTO_RANDOM` was marked as stable in v4.0.3.
> `AUTO_RANDOM` has been generally available since v4.0.3.
## User scenario

When you write data intensively into TiDB and TiDB has the table with a primary key of the auto-increment integer type, hotspot issue might occur. To solve the hotspot issue, you can use the `AUTO_RANDOM` attribute. Refer to [Highly Concurrent Write Best Practices](/best-practices/high-concurrency-best-practices.md#complex-hotspot-problems) for details.
When you write data intensively into TiDB and TiDB has a table with the primary key of the auto-increment integer type, hotspot issue might occur. To solve the hotspot issue, you can use the `AUTO_RANDOM` attribute.

<CustomContent platform="tidb">

For more information, see [Highly Concurrent Write Best Practices](/best-practices/high-concurrency-best-practices.md#complex-hotspot-problems).

</CustomContent>

Take the following created table as an example:

Expand Down
10 changes: 0 additions & 10 deletions basic-sql-operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,18 +5,8 @@ summary: Learn about the basic SQL statements for the TiDB database.

# Explore SQL with TiDB

<CustomContent platform="tidb">

TiDB is compatible with MySQL, you can use MySQL statements directly in most of the cases. For unsupported features, see [Compatibility with MySQL](/mysql-compatibility.md#unsupported-features).

</CustomContent>

<CustomContent platform="tidb-cloud">

TiDB is compatible with MySQL, you can use MySQL statements directly in most of the cases. For unsupported features, see [Compatibility with MySQL](https://docs.pingcap.com/tidb/stable/mysql-compatibility#unsupported-features).

</CustomContent>

<CustomContent platform="tidb">

To experiment with SQL and test out TiDB compatibility with MySQL queries, you can [run TiDB directly in your web browser without installing it](https://tour.tidb.io/). You can also first deploy a TiDB cluster and then run SQL statements in it.
Expand Down
14 changes: 12 additions & 2 deletions certificate-authentication.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,17 @@ The rest of the document introduces in detail how to perform these operations.

## Create security keys and certificates

It is recommended that you use [OpenSSL](https://www.openssl.org/) to create keys and certificates. The certificate generation process is similar to the process described in [Enable TLS Between TiDB Clients and Servers](/enable-tls-between-clients-and-servers.md). The following paragraphs demonstrate on how to configure more attribute fields that need to be verified in the certificate.
<CustomContent platform="tidb">

It is recommended that you use [OpenSSL](https://www.openssl.org/) to create keys and certificates. The certificate generation process is similar to the process described in [Enable TLS Between TiDB Clients and Servers](/enable-tls-between-clients-and-servers.md). The following paragraphs demonstrate how to configure more attribute fields that need to be verified in the certificate.

</CustomContent>

<CustomContent platform="tidb-cloud">

It is recommended that you use [OpenSSL](https://www.openssl.org/) to create keys and certificates. The certificate generation process is similar to the process described in [Enable TLS Between TiDB Clients and Servers](https://docs.pingcap.com/tidb/stable/enable-tls-between-clients-and-servers). The following paragraphs demonstrate how to configure more attribute fields that need to be verified in the certificate.

</CustomContent>

### Generate CA key and certificate

Expand Down Expand Up @@ -277,7 +287,7 @@ The user certificate information can be specified by `require subject`, `require
openssl x509 -noout -subject -in ca-cert.pem | sed 's/.\{8\}//' | sed 's/, /\//g' | sed 's/ = /=/g' | sed 's/^/\//'
```

+ `require san`: Specifies the `Subject Alternative Name` information of the CA certificate that issues the user certificate. The information to be specified is consistent with the [`alt_names` of the `openssl.cnf` configuration file](/generate-self-signed-certificates.md) used to generate the client certificate.
+ `require san`: Specifies the `Subject Alternative Name` information of the CA certificate that issues the user certificate. The information to be specified is consistent with the [`alt_names` of the `openssl.cnf` configuration file](https://docs.pingcap.com/tidb/stable/generate-self-signed-certificates) used to generate the client certificate.

+ Execute the following command to get the information of the `require san` item in the generated certificate:

Expand Down
18 changes: 17 additions & 1 deletion character-set-and-collation.md
Original file line number Diff line number Diff line change
Expand Up @@ -381,8 +381,12 @@ To disable this error reporting, use `set @@tidb_skip_utf8_check=1;` to skip the

## Collation support framework

<CustomContent platform="tidb">

The syntax support and semantic support for the collation are influenced by the [`new_collations_enabled_on_first_bootstrap`](/tidb-configuration-file.md#new_collations_enabled_on_first_bootstrap) configuration item. The syntax support and semantic support are different. The former indicates that TiDB can parse and set collations. The latter indicates that TiDB can correctly use collations when comparing strings.

</CustomContent>

Before v4.0, TiDB provides only the [old framework for collations](#old-framework-for-collations). In this framework, TiDB supports syntactically parsing most of the MySQL collations but semantically takes all collations as binary collations.

Since v4.0, TiDB supports a [new framework for collations](#new-framework-for-collations). In this framework, TiDB semantically parses different collations and strictly follows the collations when comparing strings.
Expand All @@ -406,7 +410,11 @@ Query OK, 1 row affected # In TiDB, it is successfully executed. In MySQL, becau

### New framework for collations

In TiDB 4.0, a complete framework for collations is introduced. This new framework supports semantically parsing collations and introduces the `new_collations_enabled_on_first_bootstrap` configuration item to decide whether to enable the new framework when a cluster is first initialized. To enable the new framework, set `new_collations_enabled_on_first_bootstrap` to `true`. For details, see [`new_collations_enabled_on_first_bootstrap`](/tidb-configuration-file.md#new_collations_enabled_on_first_bootstrap). If you initialize the cluster after the configuration item is enabled, you can check whether the new collation is enabled through the `new_collation_enabled` variable in the `mysql`.`tidb` table:
Since TiDB v4.0, a complete framework for collations is introduced.

<CustomContent platform="tidb">

This new framework supports semantically parsing collations and introduces the `new_collations_enabled_on_first_bootstrap` configuration item to decide whether to enable the new framework when a cluster is first initialized. To enable the new framework, set `new_collations_enabled_on_first_bootstrap` to `true`. For details, see [`new_collations_enabled_on_first_bootstrap`](/tidb-configuration-file.md#new_collations_enabled_on_first_bootstrap). If you initialize the cluster after the configuration item is enabled, you can check whether the new collation is enabled through the `new_collation_enabled` variable in the `mysql`.`tidb` table:

{{< copyable "sql" >}}

Expand All @@ -423,6 +431,14 @@ SELECT VARIABLE_VALUE FROM mysql.tidb WHERE VARIABLE_NAME='new_collation_enabled
1 row in set (0.00 sec)
```

</CustomContent>

<CustomContent platform="tidb-cloud">

This new framework supports semantically parsing collations. TiDB enables the new framework by default when a cluster is first initialized.

</CustomContent>

Under the new framework, TiDB supports the `utf8_general_ci`, `utf8mb4_general_ci`, `utf8_unicode_ci`, `utf8mb4_unicode_ci`, `gbk_chinese_ci`, and `gbk_bin` collations, which is compatible with MySQL.

When one of `utf8_general_ci`, `utf8mb4_general_ci`, `utf8_unicode_ci`, `utf8mb4_unicode_ci`, and `gbk_chinese_ci` is used, the string comparison is case-insensitive and accent-insensitive. At the same time, TiDB also corrects the collation's `PADDING` behavior:
Expand Down
10 changes: 10 additions & 0 deletions character-set-gbk.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,18 @@ This section provides the compatibility information between MySQL and TiDB.

The default collation of the GBK character set in MySQL is `gbk_chinese_ci`. Unlike MySQL, the default collation of the GBK character set in TiDB is `gbk_bin`. Additionally, because TiDB converts GBK to UTF8MB4 and then uses a binary collation, the `gbk_bin` collation in TiDB is not the same as the `gbk_bin` collation in MySQL.

<CustomContent platform="tidb">

To make TiDB compatible with the collations of MySQL GBK character set, when you first initialize the TiDB cluster, you need to set the TiDB option [`new_collations_enabled_on_first_bootstrap`](/tidb-configuration-file.md#new_collations_enabled_on_first_bootstrap) to `true` to enable the [new framework for collations](/character-set-and-collation.md#new-framework-for-collations).

</CustomContent>

<CustomContent platform="tidb-cloud">

To make TiDB compatible with the collations of MySQL GBK character set, when you first initialize the TiDB cluster, TiDB Cloud enables the [new framework for collations](/character-set-and-collation.md#new-framework-for-collations) by default.

</CustomContent>

After enabling the new framework for collations, if you check the collations corresponding to the GBK character set, you can see that the TiDB GBK default collation is changed to `gbk_chinese_ci`.

```sql
Expand Down
2 changes: 1 addition & 1 deletion clustered-indexes.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,7 @@ Currently, there are several different types of limitations for the clustered in
- Situations that are not supported yet but in the support plan:
- Adding, dropping, and altering clustered indexes using `ALTER TABLE` statements are not supported.
- Limitations for specific versions:
- In v5.0, using the clustered index feature together with TiDB Binlog is not supported. After TiDB Binlog is enabled, TiDB only allows creating a single integer column as the clustered index of a primary key. TiDB Binlog does not replicate data changes (such as insertion, deletion, and update) on existing tables with clustered indexes to the downstream. If you need to replicate tables with clustered indexes to the downstream, upgrade your cluster to v5.1 or use [TiCDC](/ticdc/ticdc-overview.md) for replication instead.
- In v5.0, using the clustered index feature together with TiDB Binlog is not supported. After TiDB Binlog is enabled, TiDB only allows creating a single integer column as the clustered index of a primary key. TiDB Binlog does not replicate data changes (such as insertion, deletion, and update) on existing tables with clustered indexes to the downstream. If you need to replicate tables with clustered indexes to the downstream, upgrade your cluster to v5.1 or use [TiCDC](https://docs.pingcap.com/tidb/stable/ticdc-overview) for replication instead.

After TiDB Binlog is enabled, if the clustered index you create is not a single integer primary key, TiDB returns the following error:

Expand Down
10 changes: 10 additions & 0 deletions coprocessor-cache.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,18 @@ Starting from v4.0, the TiDB instance supports caching the results of the calcul

## Configuration

<CustomContent platform="tidb">

You can configure Coprocessor Cache via the `tikv-client.copr-cache` configuration items in the TiDB configuration file. For details about how to enable and configure Coprocessor Cache, see [TiDB Configuration File](/tidb-configuration-file.md#tikv-clientcopr-cache-new-in-v400).

</CustomContent>

<CustomContent platform="tidb-cloud">

The Coprocessor Cache feature is enabled by default. The maximum size of the data that can be cached is 1000 MB.

</CustomContent>

## Feature description

+ When a SQL statement is executed on a single TiDB instance for the first time, the execution result is not cached.
Expand Down
10 changes: 10 additions & 0 deletions enable-tls-between-components.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,18 @@ Currently, it is not supported to only enable encrypted transmission of some spe

You can use tools like `openssl`, `easy-rsa` and `cfssl` to generate self-signed certificates.

<CustomContent platform="tidb">

If you choose `openssl`, you can refer to [generating self-signed certificates](/generate-self-signed-certificates.md).

</CustomContent>

<CustomContent platform="tidb-cloud">

If you choose `openssl`, you can refer to [generating self-signed certificates](https://docs.pingcap.com/tidb/stable/generate-self-signed-certificates).

</CustomContent>

2. Configure certificates.

To enable mutual authentication among TiDB components, configure the certificates of TiDB, TiKV, and PD as follows.
Expand Down
10 changes: 10 additions & 0 deletions explain-views.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,18 @@ summary: Learn about the execution plan information returned by the `EXPLAIN` st

`EXPLAIN` displays the tables and indexes that a [view](/views.md) references, not the name of the view itself. This is because views are only virtual tables and do not store any data themselves. The definition of the view and the rest of the statement are merged together during SQL optimization.

<CustomContent platform="tidb">

From the [bikeshare example database](/import-example-data.md), you can see that the following two queries are executed in a similar manner:

</CustomContent>

<CustomContent platform="tidb-cloud">

From the [bikeshare example database](/tidb-cloud/import-sample-data.md), you can see that the following two queries are executed in a similar manner:

</CustomContent>

{{< copyable "sql" >}}

```sql
Expand Down
12 changes: 11 additions & 1 deletion explain-walkthrough.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,17 @@ summary: Learn how to use EXPLAIN by walking through an example statement

Because SQL is a declarative language, you cannot automatically tell whether a query is executed efficiently. You must first use the [`EXPLAIN`](/sql-statements/sql-statement-explain.md) statement to learn the current execution plan.

The following statement from the [bikeshare example database](/import-example-data.md) counts how many trips were taken on the July 1, 2017:
<CustomContent platform="tidb">

The following statement from the [bikeshare example database](/import-example-data.md) counts how many trips were taken on July 1, 2017:

</CustomContent>

<CustomContent platform="tidb-cloud">

The following statement from the [bikeshare example database](/tidb-cloud/import-sample-data.md) counts how many trips were taken on July 1, 2017:

</CustomContent>

{{< copyable "sql" >}}

Expand Down
22 changes: 22 additions & 0 deletions information-schema/information-schema-deadlocks.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,18 @@ The meaning of each column field in the `DEADLOCKS` table is as follows:
* `KEY_INFO`: The detailed information of `KEY`. See the [KEY_INFO](#key_info) section.
* `TRX_HOLDING_LOCK`: The ID of the transaction that currently holds the lock on the key and causes blocking. This ID is also the `start_ts` of the transaction.

<CustomContent platform="tidb">

To adjust the maximum number of deadlock events that can be recorded in the `DEADLOCKS` table, adjust the [`pessimistic-txn.deadlock-history-capacity`](/tidb-configuration-file.md#deadlock-history-capacity) configuration in the TiDB configuration file. By default, the information of the recent 10 deadlock events is recorded in the table.

</CustomContent>

<CustomContent platform="tidb-cloud">

The information of the recent 10 deadlock events is recorded in the `DEADLOCKS` table.

</CustomContent>

> **Warning:**
>
> * Only users with the [PROCESS](https://dev.mysql.com/doc/refman/8.0/en/privileges-provided.html#priv_process) privilege can query this table.
Expand Down Expand Up @@ -78,10 +88,22 @@ In the above fields, if the information of a field is not applicable or currentl
## Retryable deadlock errors

<CustomContent platform="tidb-cloud">

> **Note:**
>
> This section is not applicable to TiDB Cloud.
</CustomContent>

<CustomContent platform="tidb">

> **Note:**
>
> The `DEADLOCKS` table does not collect the information of retryable deadlock errors by default. If you want the table to collect the retryable deadlock error information, you can adjust the value of [`pessimistic-txn.deadlock-history-collect-retryable`](/tidb-configuration-file.md#deadlock-history-collect-retryable) in the TiDB configuration file.
</CustomContent>

When transaction A is blocked by a lock already held by transaction B, and transaction B is directly or indirectly blocked by the lock held by the current transaction A, a deadlock error will occur. In this deadlock, there might be two cases:

+ Case 1: Transaction B might be (directly or indirectly) blocked by a lock generated by a statement that has been executed after transaction A starts and before transaction A gets blocked.
Expand Down
Loading

0 comments on commit 81f07d1

Please sign in to comment.