sql: introduce invisible index

TOC.md

@@ -206,6 +206,7 @@
       + [`ADD INDEX`](/sql-statements/sql-statement-add-index.md)
       + [`ADMIN`](/sql-statements/sql-statement-admin.md)
       + [`ALTER DATABASE`](/sql-statements/sql-statement-alter-database.md)
+      + [`ALTER INDEX`](/sql-statements/sql-statement-alter-index.md)
       + [`ALTER INSTANCE`](/sql-statements/sql-statement-alter-instance.md)
       + [`ALTER TABLE`](/sql-statements/sql-statement-alter-table.md)
       + [`ALTER USER`](/sql-statements/sql-statement-alter-user.md)

sql-statements/sql-statement-add-index.md

@@ -113,6 +113,7 @@ mysql> EXPLAIN SELECT * FROM t1 WHERE c1 = 3;
 * [CREATE INDEX](/sql-statements/sql-statement-create-index.md)
 * [DROP INDEX](/sql-statements/sql-statement-drop-index.md)
 * [RENAME INDEX](/sql-statements/sql-statement-rename-index.md)
+* [ALTER INDEX](/sql-statements/sql-statement-alter-index.md)
 * [ADD COLUMN](/sql-statements/sql-statement-add-column.md)
 * [CREATE TABLE](/sql-statements/sql-statement-create-table.md)
 * [EXPLAIN](/sql-statements/sql-statement-explain.md)

sql-statements/sql-statement-alter-database.md

@@ -9,7 +9,7 @@ aliases: ['/docs/dev/sql-statements/sql-statement-alter-database/','/docs/dev/re
 
 `ALTER DATABASE` is used to specify or modify the default character set and collation of the current database. `ALTER SCHEMA` has the same effect as `ALTER DATABASE`.
 
-## 语法图
+## Synopsis
 
 **AlterDatabaseStmt:**
 

sql-statements/sql-statement-alter-index.md

@@ -0,0 +1,204 @@
+---
+title: ALTER INDEX
+summary: An overview of the usage of ALTER INDEX for the TiDB database.
+category: reference
+---
+
+# ALTER INDEX
+
+The `ALTER INDEX` statement is used to modify the visibility of the index to `Visible` or `Invisible`. When the visibility of an index is set to `Invisible`, this index cannot be used by the optimizer.
+
+## Synopsis
+
+**AlterTableStmt:**
+
+![AlterTableStmt](/media/sqlgram/AlterTableStmt.png)
+
+**AlterTableSpec:**
+
+![AlterTableSpec](/media/sqlgram/AlterTableSpec.png)
+
+**IndexInvisible:**
+
+![IndexInvisible](/media/sqlgram/IndexInvisible.png)
+
+## Syntax
+
+{{< copyable "sql" >}}
+
+```sql
+ALTER TABLE [table_name] ALTER INDEX [index_name] {VISIBLE | INVISIBLE}
+```
+
+## Example
+
+You can modify the visibility of an index using the `ALTER TABLE ... ALTER INDEX ...` statement.
+
+{{< copyable "sql" >}}
+
+```sql
+CREATE TABLE t1 (c1 INT, UNIQUE(c1));
+ALTER TABLE t1 ALTER INDEX c1 INVISIBLE;
+```
+
+```sql
+Query OK, 0 rows affected (0.02 sec)
+```
+
+{{< copyable "sql" >}}
+
+```sql
+SHOW CREATE TABLE t1;
+```
+
+```sql
++-------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+| Table | Create Table
+                                    |
++-------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+| t1    | CREATE TABLE `t1` (
+  `c1` int(11) DEFAULT NULL,
+  UNIQUE KEY `c1` (`c1`) /*!80000 INVISIBLE */
+) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin |
++-------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+1 row in set (0.00 sec)
+```
+
+## Invisible index
+
+Invisible indexes are a new feature introduced in MySQL 8.0 that sets an index to invisible so that the optimizer no longer uses this index. Using this feature, you can easily check whether to use or not to use the query plan of an index, and avoid resource-consuming operations such as `DROP INDEX` or `ADD INDEX`.
+
+{{< copyable "sql" >}}
+
+```sql
+CREATE TABLE t1 (c1 INT, c2 INT, UNIQUE(c2));
+CREATE UNIQUE INDEX c1 ON t1 (c1) INVISIBLE;
+```
+
+You can check an invisible index using the `CREATE TABLE` statement. The invisible index is identified with `/*!80000 INVISIBLE */`:
+
+{{< copyable "sql" >}}
+
+```sql
+SHOW CREATE TABLE t1;
+```
+
+```sql
++-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+| Table | Create Table
+                                                              |
++-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+| t1    | CREATE TABLE `t1` (
+  `c1` int(11) DEFAULT NULL,
+  `c2` int(11) DEFAULT NULL,
+  UNIQUE KEY `c2` (`c2`),
+  UNIQUE KEY `c1` (`c1`) /*!80000 INVISIBLE */
+) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin |
++-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+1 row in set (0.00 sec)
+```
+
+The optimizer cannot use the **invisible index** of `c1`.
+
+{{< copyable "sql" >}}
+
+```sql
+EXPLAIN SELECT c1 FROM t1 ORDER BY c1;
+```
+
+```sql
++-------------------------+----------+-----------+---------------+--------------------------------+
+| id                      | estRows  | task      | access object | operator info                  |
++-------------------------+----------+-----------+---------------+--------------------------------+
+| Sort_4                  | 10000.00 | root      |               | test.t1.c1:asc                 |
+| └─TableReader_8         | 10000.00 | root      |               | data:TableFullScan_7           |
+|   └─TableFullScan_7     | 10000.00 | cop[tikv] | table:t1      | keep order:false, stats:pseudo |
++-------------------------+----------+-----------+---------------+--------------------------------+
+3 rows in set (0.00 sec)
+```
+
+By comparison, `c2` is a **visible index** and can be used by the optimizer.
+
+{{< copyable "sql" >}}
+
+```sql
+EXPLAIN SELECT c2 FROM t1 ORDER BY c2;
+```
+
+```sql
++------------------------+----------+-----------+------------------------+-------------------------------+
+| id                     | estRows  | task      | access object          | operator info                 |
++------------------------+----------+-----------+------------------------+-------------------------------+
+| IndexReader_13         | 10000.00 | root      |                        | index:IndexFullScan_12        |
+| └─IndexFullScan_12     | 10000.00 | cop[tikv] | table:t1, index:c2(c2) | keep order:true, stats:pseudo |
++------------------------+----------+-----------+------------------------+-------------------------------+
+2 rows in set (0.00 sec)
+```
+
+Even if you use the `USE INDEX` SQL hint to forcibly use indexes, the optimizer still cannot use invisible indexes; otherwise, an error is returned.
+
+{{< copyable "sql" >}}
+
+```sql
+SELECT * FROM t1 USE INDEX(c1);
+```
+
+```sql
+ERROR 1176 (42000): Key 'c1' doesn't exist in table 't1'
+```
+
+> **Note:**
+>
+> "Invisible" here means invisible only to the optimizer. You can still modify or delete invisible indexes.
+
+{{< copyable "sql" >}}
+
+```sql
+ALTER TABLE t1 DROP INDEX c1;
+```
+
+```sql
+Query OK, 0 rows affected (0.02 sec)
+```
+
+### Restriction
+
+There is a restriction on invisible indexes in MySQL: the **primary key** cannot be set to `Invisible`. TiDB is compatible with this restriction. If you set the primary key to `Invisible` in TiDB, an error is returned.
+
+{{< copyable "sql" >}}
+
+```sql
+CREATE TABLE t2(c1 INT, PRIMARY KEY(c1) INVISIBLE);
+```
+
+```sql
+ERROR 3522 (HY000): A primary key index cannot be invisible
+```
+
+The **primary key** above includes both the explicit primary key (specified using the `PRIMARY KEY` keyword) and the implicit primary key.
+
+If the explicit primary key does not exist in the table, the first `UNIQUE` index (every column of the index must be `NOT NULL`) automatically becomes the implicit index.
+
+This means that you cannot set this implicit primary key to `Invisible`.
+
+> **Note:**
+>
+> TiDB does not actually create an **implicit primary key**. TiDB is compatible with this MySQL restriction only in behaviors.
+
+{{< copyable "sql" >}}
+
+```sql
+CREATE TABLE t2(c1 INT NOT NULL, UNIQUE(c1) INVISIBLE);
+```
+
+```
+ERROR 3522 (HY000): A primary key index cannot be invisible
+```
+
+## See also
+
+* [CREATE TABLE](/sql-statements/sql-statement-create-table.md)
+* [CREATE INDEX](/sql-statements/sql-statement-create-index.md)
+* [ADD INDEX](/sql-statements/sql-statement-add-index.md)
+* [DROP INDEX](/sql-statements/sql-statement-drop-index.md)
+* [RENAME INDEX](/sql-statements/sql-statement-rename-index.md)

sql-statements/sql-statement-alter-table.md

@@ -70,6 +70,7 @@ mysql> EXPLAIN SELECT * FROM t1 WHERE c1 = 3;
 * [ADD INDEX](/sql-statements/sql-statement-add-index.md)
 * [DROP INDEX](/sql-statements/sql-statement-drop-index.md)
 * [RENAME INDEX](/sql-statements/sql-statement-rename-index.md)
+* [ALTER INDEX](/sql-statements/sql-statement-alter-index.md)
 * [CREATE TABLE](/sql-statements/sql-statement-create-table.md)
 * [DROP TABLE](/sql-statements/sql-statement-drop-table.md)
 * [SHOW CREATE TABLE](/sql-statements/sql-statement-show-create-table.md)

sql-statements/sql-statement-create-index.md

@@ -142,6 +142,17 @@ Expression indexes have the same syntax and limitations as in MySQL. They are im
 
 Currently, the optimizer can use the indexed expressions when the expressions are only in the `FIELD` clause, `WHERE` clause, and `ORDER BY` clause. The `GROUP BY` clause will be supported in future updates.
 
+## Invisible index
+
+Invisible indexes are a new feature introduced in MySQL 8.0 that sets an index to invisible so that the optimizer no longer uses this index.
+
+```sql
+CREATE TABLE t1 (c1 INT, c2 INT, UNIQUE(c2));
+CREATE UNIQUE INDEX c1 ON t1 (c1) INVISIBLE;
+```
+
+For details, see [Invisible index](/sql-statements/sql-statement-alter-index.md#invisible-index)
+
 ## Associated session variables
 
 The global variables associated with the `CREATE INDEX` statement are `tidb_ddl_reorg_worker_cnt`, `tidb_ddl_reorg_batch_size` and `tidb_ddl_reorg_priority`. Refer to [TiDB-specific system variables](/tidb-specific-system-variables.md#tidb_ddl_reorg_worker_cnt) for details.
@@ -157,6 +168,7 @@ The global variables associated with the `CREATE INDEX` statement are `tidb_ddl_
 * [ADD INDEX](/sql-statements/sql-statement-add-index.md)
 * [DROP INDEX](/sql-statements/sql-statement-drop-index.md)
 * [RENAME INDEX](/sql-statements/sql-statement-rename-index.md)
+* [ALTER INDEX](/sql-statements/sql-statement-alter-index.md)
 * [ADD COLUMN](/sql-statements/sql-statement-add-column.md)
 * [CREATE TABLE](/sql-statements/sql-statement-create-table.md)
 * [EXPLAIN](/sql-statements/sql-statement-explain.md)

sql-statements/sql-statement-create-table.md

@@ -185,6 +185,7 @@ index_option:
     KEY_BLOCK_SIZE [=] value
   | index_type
   | COMMENT 'string'
+  | {VISIBLE | INVISIBLE}
 ```
 
 The `KEY_BLOCK_SIZE` in `index_option` is currently only supported in syntax.

sql-statements/sql-statement-drop-index.md

@@ -81,3 +81,4 @@ Query OK, 0 rows affected (0.30 sec)
 * [CREATE INDEX](/sql-statements/sql-statement-create-index.md)
 * [ADD INDEX](/sql-statements/sql-statement-add-index.md)
 * [RENAME INDEX](/sql-statements/sql-statement-rename-index.md)
+* [ALTER INDEX](/sql-statements/sql-statement-alter-index.md)

sql-statements/sql-statement-rename-index.md

@@ -61,3 +61,4 @@ This statement is understood to be fully compatible with MySQL. Any compatibilit
 * [CREATE INDEX](/sql-statements/sql-statement-create-index.md)
 * [DROP INDEX](/sql-statements/sql-statement-drop-index.md)
 * [SHOW INDEX](/sql-statements/sql-statement-show-index.md)
+* [ALTER INDEX](/sql-statements/sql-statement-alter-index.md)

sql-statements/sql-statement-show-indexes.md

@@ -38,30 +38,36 @@ mysql> CREATE TABLE t1 (id int not null primary key AUTO_INCREMENT, col1 INT, IN
 Query OK, 0 rows affected (0.12 sec)
 
 mysql> SHOW INDEXES FROM t1;
-+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
-| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
-+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
-| t1    |          0 | PRIMARY  |            1 | id          | A         |           0 |     NULL | NULL   |      | BTREE      |         |               |
-| t1    |          1 | col1     |            1 | col1        | A         |           0 |     NULL | NULL   | YES  | BTREE      |         |               |
-+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
++-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+
+| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment | Visible | Expression |
++-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+
+| t1    |          0 | PRIMARY  |            1 | id          | A         |           0 |     NULL | NULL   |      | BTREE      |         |               | YES
+    | NULL       |
+| t1    |          1 | col1     |            1 | col1        | A         |           0 |     NULL | NULL   | YES  | BTREE      |         |               | YES
+    | NULL       |
++-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+
 2 rows in set (0.00 sec)
 
 mysql> SHOW INDEX FROM t1;
-+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
-| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
-+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
-| t1    |          0 | PRIMARY  |            1 | id          | A         |           0 |     NULL | NULL   |      | BTREE      |         |               |
-| t1    |          1 | col1     |            1 | col1        | A         |           0 |     NULL | NULL   | YES  | BTREE      |         |               |
-+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
++-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+
+| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment | Visible | Expression |
++-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+
+| t1    |          0 | PRIMARY  |            1 | id          | A         |           0 |     NULL | NULL   |      | BTREE      |         |               | YES
+    | NULL       |
+| t1    |          1 | col1     |            1 | col1        | A         |           0 |     NULL | NULL   | YES  | BTREE      |         |               | YES
+    | NULL       |
++-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+
 2 rows in set (0.00 sec)
 
 mysql> SHOW KEYS FROM t1;
-+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
-| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
-+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
-| t1    |          0 | PRIMARY  |            1 | id          | A         |           0 |     NULL | NULL   |      | BTREE      |         |               |
-| t1    |          1 | col1     |            1 | col1        | A         |           0 |     NULL | NULL   | YES  | BTREE      |         |               |
-+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
++-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+
+| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment | Visible | Expression |
++-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+
+| t1    |          0 | PRIMARY  |            1 | id          | A         |           0 |     NULL | NULL   |      | BTREE      |         |               | YES
+    | NULL       |
+| t1    |          1 | col1     |            1 | col1        | A         |           0 |     NULL | NULL   | YES  | BTREE      |         |               | YES
+    | NULL       |
++-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+
 2 rows in set (0.00 sec)
 ```