diff --git a/docs/sql-manual/sql-statements/data-modification/DML/INSERT-OVERWRITE.md b/docs/sql-manual/sql-statements/data-modification/DML/INSERT-OVERWRITE.md index 33936b29c527c..f7dcd067113ab 100644 --- a/docs/sql-manual/sql-statements/data-modification/DML/INSERT-OVERWRITE.md +++ b/docs/sql-manual/sql-statements/data-modification/DML/INSERT-OVERWRITE.md @@ -26,9 +26,6 @@ specific language governing permissions and limitations under the License. --> - - - ## Description The function of this statement is to overwrite a table or some partitions of a table @@ -61,7 +58,7 @@ INSERT OVERWRITE table table_name > > query: a common query, the result of the query will overwrite the target. > -> hint: some indicator used to indicate the execution behavior of `INSERT`. You can choose one of this values: `/*+ STREAMING */`, `/*+ SHUFFLE */` or `/*+ NOSHUFFLE */. +> hint: some indicator used to indicate the execution behavior of `INSERT`. You can choose one of this values: `/*+ STREAMING */`, `/*+ SHUFFLE */` or `/*+ NOSHUFFLE */`. > > 1. STREAMING: At present, it has no practical effect and is only reserved for compatibility with previous versions. (In the previous version, adding this hint would return a label, but now it defaults to returning a label) > 2. SHUFFLE: When the target table is a partition table, enabling this hint will do repartiiton. @@ -72,7 +69,7 @@ Notice: 1. In the current version, the session variable `enable_insert_strict` is set to `true` by default. If some data that does not conform to the format of the target table is filtered out during the execution of the `INSERT OVERWRITE` statement, such as when overwriting a partition and not all partition conditions are satisfied, overwriting the target table will fail. 2. The `INSERT OVERWRITE` statement first creates a new table, inserts the data to be overwritten into the new table, and then atomically replaces the old table with the new table and modifies its name. Therefore, during the process of overwriting the table, the data in the old table can still be accessed normally until the overwriting is completed. -#### For Auto Partition Table +### For Auto Partition Table If the target table of the INSERT OVERWRITE is an autopartitioned table, the behaviour is controlled by the [Session Variable](../../session/variable/SET-VARIABLE.md) `enable_auto_create_when_overwrite` controls the behaviour as follows: @@ -80,7 +77,25 @@ If the target table of the INSERT OVERWRITE is an autopartitioned table, the beh 2. If an overwrite PARTITION is specified, the AUTO PARTITION table behaves as a normal partitioned table during this process, and data that does not satisfy the conditions of an existing partition is filtered instead of creating a new partition. 3. If you specify PARTITION as `partition(*)` (auto detect partition and overwrite), when `enable_auto_create_when_overwrite` is `true`, for the data that have corresponding partitions in the table, overwrite their corresponding partitions, and leave the other existing partitions unchanged. At the same time, for data without corresponding partitions, create partitions according to the table's auto-partitioning rules, and accommodate the data without corresponding partitions. If `enable_auto_create_when_overwrite` is `false`, data for which no partition is found will accumulate error rows until it fails. -`enable_auto_create_when_overwrite` was introduced since 2.1.6. In versions without `enable_auto_create_when_overwrite`, the behaviour is as if the variable had a value of `false`. +In versions without `enable_auto_create_when_overwrite`, the behaviour is as if the variable had a value of `false`. + +The quick check conclusion is as follows: + +1. For auto-partition tables that have `enable_auto_create_when_overwrite` enabled: + +| | Partitions to Overwrite | Clear Other Partitions | Auto Create for Unpartitioned Data | +|-|-|-|-| +| Unlabeled (Full Table) | All | √ | √ | +| Designated Partition | Explicit partition | × | × | +| `partition(*)` | The partition that the data belongs to | × | √ | + +2. For trivial tables, auto-partition tables disabled `enable_auto_create_when_overwrite`: + +| | Partitions to Overwrite | Clear Other Partitions | Auto Create for Unpartitioned Data | +|-|-|-|-| +| Unlabeled (Full Table) | All | √ | × | +| Designated Partition | Explicit partition | × | × | +| `partition(*)` | The partition that the data belongs to | × | × | Examples are shown below: @@ -155,7 +170,7 @@ PROPERTIES ( ); ``` -#### Overwrite Table +### Overwrite Table 1. Overwrite the `test` table using the form of `VALUES`. @@ -195,8 +210,7 @@ PROPERTIES ( - Users can use the `SHOW LOAD;` command to check the status of the job imported by this `label`. It should be noted that the label is unique. - -#### Overwrite Table Partition +### Overwrite Table Partition When using INSERT OVERWRITE to rewrite partitions, we actually encapsulate the following three steps into a single transaction and execute it. If it fails halfway through, the operations that have been performed will be rolled back: @@ -243,8 +257,7 @@ The following is examples: INSERT OVERWRITE table test PARTITION(p1,p2) WITH LABEL `label4` (c1, c2) SELECT * from test2; ``` - -#### Overwrite Auto Detect Partition +### Overwrite Auto Detect Partition > This feature is available since version 2.1.3. @@ -297,4 +310,4 @@ As you can see, all data in partitions `p10` and `pMAX`, where data 3 and 1234 a ## Keywords - INSERT OVERWRITE, OVERWRITE, AUTO DETECT +INSERT OVERWRITE, OVERWRITE, AUTO DETECT \ No newline at end of file diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-statements/data-modification/DML/INSERT-OVERWRITE.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-statements/data-modification/DML/INSERT-OVERWRITE.md index c79d3175d339a..6b4e4e1c9ecf5 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-statements/data-modification/DML/INSERT-OVERWRITE.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-statements/data-modification/DML/INSERT-OVERWRITE.md @@ -25,7 +25,6 @@ specific language governing permissions and limitations under the License. --> - ## 描述 该语句的功能是重写表或表的某些分区 @@ -69,14 +68,33 @@ INSERT OVERWRITE table table_name 1. 在当前版本中,会话变量 `enable_insert_strict` 默认为 `true`,如果执行 `INSERT OVERWRITE` 语句时,对于有不符合目标表格式的数据被过滤掉的话会重写目标表失败(比如重写分区时,不满足所有分区条件的数据会被过滤)。 2. INSERT OVERWRITE 语句会首先创建一个新表,将需要重写的数据插入到新表中,最后原子性的用新表替换旧表并修改名称。因此,在重写表的过程中,旧表中的数据在重写完毕之前仍然可以正常访问。 -#### For Auto Partition Table +### For Auto Partition Table 如果 INSERT OVERWRITE 的目标表是自动分区表,那么行为受到 [Session Variable](../../session/variable/SET-VARIABLE.md) `enable_auto_create_when_overwrite` 的控制,具体行为如下: + 1. 若未指定 PARTITION(覆写整表),当 `enable_auto_create_when_overwrite` 为 `true`,在覆写整表已有数据的同时,对于没有对应分区的数据,按照该表的自动分区规则创建分区,并容纳这些原本没有对应分区的数据。如果 `enable_auto_create_when_overwrite` 为 `false`,未找到分区的数据将累计错误行直到失败。 2. 如果指定了覆写的 PARTITION,那么在此过程中,AUTO PARTITION 表表现得如同普通分区表一样,不满足现有分区条件的数据将被过滤,而非创建新的分区。 3. 若指定 PARTITION 为 `partition(*)` (自动检测分区并覆写),当 `enable_auto_create_when_overwrite` 为 `true`,对于那些在表中有对应分区的数据,覆写它们对应的分区,其他已有分区不变。同时,对于没有对应分区的数据,按照该表的自动分区规则创建分区,并容纳这些原本没有对应分区的数据。如果 `enable_auto_create_when_overwrite` 为 `false`,未找到分区的数据将累计错误行直到失败。 -`enable_auto_create_when_overwrite` 自 3.0.3 引入,在没有 `enable_auto_create_when_overwrite` 的版本,行为如同该变量值为 `false`。 +在没有 `enable_auto_create_when_overwrite` 的版本,行为如同该变量值为 `false`。 + +速查结论如下: + +1. 对于开启 `enable_auto_create_when_overwrite` 的自动分区表: + +| | 覆写的分区 | 清空其他分区 | 无分区的数据自动创建 | +|-|-|-|-| +| 无标识(全表) | 所有 | √ | √ | +| 指定分区 | 写明的分区 | × | × | +| `partition(*)` | 数据所属的分区 | × | √ | + +2. 对于普通表、关闭 `enable_auto_create_when_overwrite` 的自动分区表: + +| | 覆写的分区 | 清空其他分区 | 无分区的数据自动创建 | +|-|-|-|-| +| 无标识(全表) | 所有 | √ | × | +| 指定分区 | 写明的分区 | × | × | +| `partition(*)` | 数据所属的分区 | × | × | 示例如下: @@ -151,7 +169,7 @@ PROPERTIES ( ); ``` -#### Overwrite Table +### Overwrite Table 1. VALUES 的形式重写`test`表 @@ -192,10 +210,10 @@ PROPERTIES ( - 使用 label 会将此任务封装成一个**异步任务**,执行语句之后,相关操作都会异步执行,用户可以通过`SHOW LOAD;`命令查看此`label`导入作业的状态。需要注意的是 label 具有唯一性。 - -#### Overwrite Table Partition +### Overwrite Table Partition 使用 INSERT OVERWRITE 重写分区时,实际我们是将如下三步操作封装为一个事务并执行,如果中途失败,已进行的操作将会回滚: + 1. 假设指定重写分区 p1,首先创建一个与重写的目标分区结构相同的空临时分区 `pTMP` 2. 向 `pTMP` 中写入数据 3. 使用 `pTMP` 原子替换 `p1` 分区 @@ -239,7 +257,7 @@ PROPERTIES ( INSERT OVERWRITE table test PARTITION(p1,p2) WITH LABEL `label4` (c1, c2) SELECT * from test2; ``` -#### Overwrite Auto Detect Partition +### Overwrite Auto Detect Partition 当 INSERT OVERWRITE 命令指定的 PARTITION 子句为 `PARTITION(*)` 时,此次覆写将会自动检测分区数据所在的分区。例如: @@ -290,5 +308,4 @@ mysql> select * from test order by k0; ## 关键词 - INSERT OVERWRITE, OVERWRITE, AUTO DETECT - +INSERT OVERWRITE, OVERWRITE, AUTO DETECT diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/sql-manual/sql-statements/data-modification/DML/INSERT-OVERWRITE.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/sql-manual/sql-statements/data-modification/DML/INSERT-OVERWRITE.md index fd4983897ba60..d81a25139ce9c 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/sql-manual/sql-statements/data-modification/DML/INSERT-OVERWRITE.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/sql-manual/sql-statements/data-modification/DML/INSERT-OVERWRITE.md @@ -25,8 +25,6 @@ specific language governing permissions and limitations under the License. --> - - ## 描述 该语句的功能是重写表或表的某些分区 @@ -48,7 +46,6 @@ INSERT OVERWRITE table table_name > >> 1. 分区名。必须是 `table_name` 中存在的分区,多个分区名称用逗号分隔。 >> 2. 星号 (*)。开启[自动检测分区](#overwrite-auto-detect-partition)功能。写入操作将会自动检测数据所涉及的分区,并覆写这些分区。该功能自 Apache Doris 2.1.3 版本开始支持。 ->> > > label: 为 Insert 任务指定一个 label > @@ -71,15 +68,34 @@ INSERT OVERWRITE table table_name 1. 在当前版本中,会话变量 `enable_insert_strict` 默认为 `true`,如果执行 `INSERT OVERWRITE` 语句时,对于有不符合目标表格式的数据被过滤掉的话会重写目标表失败(比如重写分区时,不满足所有分区条件的数据会被过滤)。 2. INSERT OVERWRITE 语句会首先创建一个新表,将需要重写的数据插入到新表中,最后原子性的用新表替换旧表并修改名称。因此,在重写表的过程中,旧表中的数据在重写完毕之前仍然可以正常访问。 -#### For Auto Partition Table +### For Auto Partition Table 如果 INSERT OVERWRITE 的目标表是自动分区表,那么行为受到 [Session Variable](../../session/variable/SET-VARIABLE.md) `enable_auto_create_when_overwrite` 的控制,具体行为如下: + 1. 若未指定 PARTITION(覆写整表),当 `enable_auto_create_when_overwrite` 为 `true`,在覆写整表已有数据的同时,对于没有对应分区的数据,按照该表的自动分区规则创建分区,并容纳这些原本没有对应分区的数据。如果 `enable_auto_create_when_overwrite` 为 `false`,未找到分区的数据将累计错误行直到失败。 2. 如果指定了覆写的 PARTITION,那么在此过程中,AUTO PARTITION 表表现得如同普通分区表一样,不满足现有分区条件的数据将被过滤,而非创建新的分区。 3. 若指定 PARTITION 为 `partition(*)` (自动检测分区并覆写),当 `enable_auto_create_when_overwrite` 为 `true`,对于那些在表中有对应分区的数据,覆写它们对应的分区,其他已有分区不变。同时,对于没有对应分区的数据,按照该表的自动分区规则创建分区,并容纳这些原本没有对应分区的数据。如果 `enable_auto_create_when_overwrite` 为 `false`,未找到分区的数据将累计错误行直到失败。 `enable_auto_create_when_overwrite` 自 2.1.8 引入,在没有 `enable_auto_create_when_overwrite` 的版本,行为如同该变量值为 `false`。 +速查结论如下: + +1. 对于开启 `enable_auto_create_when_overwrite` 的自动分区表: + +| | 覆写的分区 | 清空其他分区 | 无分区的数据自动创建 | +|-|-|-|-| +| 无标识(全表) | 所有 | √ | √ | +| 指定分区 | 写明的分区 | × | × | +| `partition(*)` | 数据所属的分区 | × | √ | + +2. 对于普通表、关闭 `enable_auto_create_when_overwrite` 的自动分区表: + +| | 覆写的分区 | 清空其他分区 | 无分区的数据自动创建 | +|-|-|-|-| +| 无标识(全表) | 所有 | √ | × | +| 指定分区 | 写明的分区 | × | × | +| `partition(*)` | 数据所属的分区 | × | × | + 示例如下: ```sql @@ -153,7 +169,7 @@ PROPERTIES ( ); ``` -#### Overwrite Table +### Overwrite Table 1. VALUES 的形式重写`test`表 @@ -195,9 +211,10 @@ PROPERTIES ( - 用户可以通过`SHOW LOAD;`命令查看此`label`导入作业的状态。需要注意的是 label 具有唯一性。 -#### Overwrite Table Partition +### Overwrite Table Partition 使用 INSERT OVERWRITE 重写分区时,实际我们是将如下三步操作封装为一个事务并执行,如果中途失败,已进行的操作将会回滚: + 1. 假设指定重写分区 p1,首先创建一个与重写的目标分区结构相同的空临时分区 `pTMP` 2. 向 `pTMP` 中写入数据 3. 使用 `pTMP` 原子替换 `p1` 分区 @@ -241,7 +258,7 @@ PROPERTIES ( INSERT OVERWRITE table test PARTITION(p1,p2) WITH LABEL `label4` (c1, c2) SELECT * from test2; ``` -#### Overwrite Auto Detect Partition +### Overwrite Auto Detect Partition 当 INSERT OVERWRITE 命令指定的 PARTITION 子句为 `PARTITION(*)` 时,此次覆写将会自动检测分区数据所在的分区。例如: @@ -292,5 +309,4 @@ mysql> select * from test order by k0; ## 关键词 - INSERT OVERWRITE, OVERWRITE, AUTO DETECT - +INSERT OVERWRITE, OVERWRITE, AUTO DETECT diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/sql-manual/sql-statements/data-modification/DML/INSERT-OVERWRITE.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/sql-manual/sql-statements/data-modification/DML/INSERT-OVERWRITE.md index c79d3175d339a..0b6bce5e7a324 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/sql-manual/sql-statements/data-modification/DML/INSERT-OVERWRITE.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/sql-manual/sql-statements/data-modification/DML/INSERT-OVERWRITE.md @@ -25,7 +25,6 @@ specific language governing permissions and limitations under the License. --> - ## 描述 该语句的功能是重写表或表的某些分区 @@ -69,15 +68,34 @@ INSERT OVERWRITE table table_name 1. 在当前版本中,会话变量 `enable_insert_strict` 默认为 `true`,如果执行 `INSERT OVERWRITE` 语句时,对于有不符合目标表格式的数据被过滤掉的话会重写目标表失败(比如重写分区时,不满足所有分区条件的数据会被过滤)。 2. INSERT OVERWRITE 语句会首先创建一个新表,将需要重写的数据插入到新表中,最后原子性的用新表替换旧表并修改名称。因此,在重写表的过程中,旧表中的数据在重写完毕之前仍然可以正常访问。 -#### For Auto Partition Table +### For Auto Partition Table 如果 INSERT OVERWRITE 的目标表是自动分区表,那么行为受到 [Session Variable](../../session/variable/SET-VARIABLE.md) `enable_auto_create_when_overwrite` 的控制,具体行为如下: + 1. 若未指定 PARTITION(覆写整表),当 `enable_auto_create_when_overwrite` 为 `true`,在覆写整表已有数据的同时,对于没有对应分区的数据,按照该表的自动分区规则创建分区,并容纳这些原本没有对应分区的数据。如果 `enable_auto_create_when_overwrite` 为 `false`,未找到分区的数据将累计错误行直到失败。 2. 如果指定了覆写的 PARTITION,那么在此过程中,AUTO PARTITION 表表现得如同普通分区表一样,不满足现有分区条件的数据将被过滤,而非创建新的分区。 3. 若指定 PARTITION 为 `partition(*)` (自动检测分区并覆写),当 `enable_auto_create_when_overwrite` 为 `true`,对于那些在表中有对应分区的数据,覆写它们对应的分区,其他已有分区不变。同时,对于没有对应分区的数据,按照该表的自动分区规则创建分区,并容纳这些原本没有对应分区的数据。如果 `enable_auto_create_when_overwrite` 为 `false`,未找到分区的数据将累计错误行直到失败。 `enable_auto_create_when_overwrite` 自 3.0.3 引入,在没有 `enable_auto_create_when_overwrite` 的版本,行为如同该变量值为 `false`。 +速查结论如下: + +1. 对于开启 `enable_auto_create_when_overwrite` 的自动分区表: + +| | 覆写的分区 | 清空其他分区 | 无分区的数据自动创建 | +|-|-|-|-| +| 无标识(全表) | 所有 | √ | √ | +| 指定分区 | 写明的分区 | × | × | +| `partition(*)` | 数据所属的分区 | × | √ | + +2. 对于普通表、关闭 `enable_auto_create_when_overwrite` 的自动分区表: + +| | 覆写的分区 | 清空其他分区 | 无分区的数据自动创建 | +|-|-|-|-| +| 无标识(全表) | 所有 | √ | × | +| 指定分区 | 写明的分区 | × | × | +| `partition(*)` | 数据所属的分区 | × | × | + 示例如下: ```sql @@ -151,7 +169,7 @@ PROPERTIES ( ); ``` -#### Overwrite Table +### Overwrite Table 1. VALUES 的形式重写`test`表 @@ -192,10 +210,10 @@ PROPERTIES ( - 使用 label 会将此任务封装成一个**异步任务**,执行语句之后,相关操作都会异步执行,用户可以通过`SHOW LOAD;`命令查看此`label`导入作业的状态。需要注意的是 label 具有唯一性。 - -#### Overwrite Table Partition +### Overwrite Table Partition 使用 INSERT OVERWRITE 重写分区时,实际我们是将如下三步操作封装为一个事务并执行,如果中途失败,已进行的操作将会回滚: + 1. 假设指定重写分区 p1,首先创建一个与重写的目标分区结构相同的空临时分区 `pTMP` 2. 向 `pTMP` 中写入数据 3. 使用 `pTMP` 原子替换 `p1` 分区 @@ -239,7 +257,7 @@ PROPERTIES ( INSERT OVERWRITE table test PARTITION(p1,p2) WITH LABEL `label4` (c1, c2) SELECT * from test2; ``` -#### Overwrite Auto Detect Partition +### Overwrite Auto Detect Partition 当 INSERT OVERWRITE 命令指定的 PARTITION 子句为 `PARTITION(*)` 时,此次覆写将会自动检测分区数据所在的分区。例如: @@ -290,5 +308,4 @@ mysql> select * from test order by k0; ## 关键词 - INSERT OVERWRITE, OVERWRITE, AUTO DETECT - +INSERT OVERWRITE, OVERWRITE, AUTO DETECT diff --git a/versioned_docs/version-2.1/sql-manual/sql-statements/data-modification/DML/INSERT-OVERWRITE.md b/versioned_docs/version-2.1/sql-manual/sql-statements/data-modification/DML/INSERT-OVERWRITE.md index f458600d52403..5efc14dffaea0 100644 --- a/versioned_docs/version-2.1/sql-manual/sql-statements/data-modification/DML/INSERT-OVERWRITE.md +++ b/versioned_docs/version-2.1/sql-manual/sql-statements/data-modification/DML/INSERT-OVERWRITE.md @@ -25,9 +25,6 @@ specific language governing permissions and limitations under the License. --> - - - ## Description The function of this statement is to overwrite a table or some partitions of a table @@ -60,7 +57,7 @@ INSERT OVERWRITE table table_name > > query: a common query, the result of the query will overwrite the target. > -> hint: some indicator used to indicate the execution behavior of `INSERT`. You can choose one of this values: `/*+ STREAMING */`, `/*+ SHUFFLE */` or `/*+ NOSHUFFLE */. +> hint: some indicator used to indicate the execution behavior of `INSERT`. You can choose one of this values: `/*+ STREAMING */`, `/*+ SHUFFLE */` or `/*+ NOSHUFFLE */`. > > 1. STREAMING: At present, it has no practical effect and is only reserved for compatibility with previous versions. (In the previous version, adding this hint would return a label, but now it defaults to returning a label) > 2. SHUFFLE: When the target table is a partition table, enabling this hint will do repartiiton. @@ -71,15 +68,34 @@ Notice: 1. In the current version, the session variable `enable_insert_strict` is set to `true` by default. If some data that does not conform to the format of the target table is filtered out during the execution of the `INSERT OVERWRITE` statement, such as when overwriting a partition and not all partition conditions are satisfied, overwriting the target table will fail. 2. The `INSERT OVERWRITE` statement first creates a new table, inserts the data to be overwritten into the new table, and then atomically replaces the old table with the new table and modifies its name. Therefore, during the process of overwriting the table, the data in the old table can still be accessed normally until the overwriting is completed. -#### For Auto Partition Table +### For Auto Partition Table If the target table of the INSERT OVERWRITE is an autopartitioned table, the behaviour is controlled by the [Session Variable](../../session/variable/SET-VARIABLE.md) `enable_auto_create_when_overwrite` controls the behaviour as follows: + 1. If PARTITION is not specified (overwrite the whole table), when `enable_auto_create_when_overwrite` is `true`, the table is overwritten and partitions are created according to the table's auto-partitioning rules for data that does not have a corresponding partition, and those datas is admit. If `enable_auto_create_when_overwrite` is `false`, data for which no partition is found will accumulate error rows until it fails. 2. If an overwrite PARTITION is specified, the AUTO PARTITION table behaves as a normal partitioned table during this process, and data that does not satisfy the conditions of an existing partition is filtered instead of creating a new partition. 3. If you specify PARTITION as `partition(*)` (auto detect partition and overwrite), when `enable_auto_create_when_overwrite` is `true`, for the data that have corresponding partitions in the table, overwrite their corresponding partitions, and leave the other existing partitions unchanged. At the same time, for data without corresponding partitions, create partitions according to the table's auto-partitioning rules, and accommodate the data without corresponding partitions. If `enable_auto_create_when_overwrite` is `false`, data for which no partition is found will accumulate error rows until it fails. `enable_auto_create_when_overwrite` was introduced since 2.1.8. In versions without `enable_auto_create_when_overwrite`, the behaviour is as if the variable had a value of `false`. +The quick check conclusion is as follows: + +1. For auto-partition tables that have `enable_auto_create_when_overwrite` enabled: + +| | Partitions to Overwrite | Clear Other Partitions | Auto Create for Unpartitioned Data | +|-|-|-|-| +| Unlabeled (Full Table) | All | √ | √ | +| Designated Partition | Explicit partition | × | × | +| `partition(*)` | The partition that the data belongs to | × | √ | + +2. For trivial tables, auto-partition tables disabled `enable_auto_create_when_overwrite`: + +| | Partitions to Overwrite | Clear Other Partitions | Auto Create for Unpartitioned Data | +|-|-|-|-| +| Unlabeled (Full Table) | All | √ | × | +| Designated Partition | Explicit partition | × | × | +| `partition(*)` | The partition that the data belongs to | × | × | + Examples are shown below: ```sql @@ -153,7 +169,7 @@ PROPERTIES ( ); ``` -#### Overwrite Table +### Overwrite Table 1. Overwrite the `test` table using the form of `VALUES`. @@ -193,8 +209,7 @@ PROPERTIES ( - Users can use the `SHOW LOAD;` command to check the status of the job imported by this `label`. It should be noted that the label is unique. - -#### Overwrite Table Partition +### Overwrite Table Partition When using INSERT OVERWRITE to rewrite partitions, we actually encapsulate the following three steps into a single transaction and execute it. If it fails halfway through, the operations that have been performed will be rolled back: 1. Assuming that partition `p1` is specified to be rewritten, first create an empty temporary partition `pTMP` with the same structure as the target partition to be rewritten. @@ -240,8 +255,7 @@ The following is examples: INSERT OVERWRITE table test PARTITION(p1,p2) WITH LABEL `label4` (c1, c2) SELECT * from test2; ``` - -#### Overwrite Auto Detect Partition +### Overwrite Auto Detect Partition > This feature is available since version 2.1.3. @@ -294,4 +308,4 @@ As you can see, all data in partitions `p10` and `pMAX`, where data 3 and 1234 a ## Keywords - INSERT OVERWRITE, OVERWRITE, AUTO DETECT +INSERT OVERWRITE, OVERWRITE, AUTO DETECT diff --git a/versioned_docs/version-3.0/sql-manual/sql-statements/data-modification/DML/INSERT-OVERWRITE.md b/versioned_docs/version-3.0/sql-manual/sql-statements/data-modification/DML/INSERT-OVERWRITE.md index 33936b29c527c..dfbb4eec638ab 100644 --- a/versioned_docs/version-3.0/sql-manual/sql-statements/data-modification/DML/INSERT-OVERWRITE.md +++ b/versioned_docs/version-3.0/sql-manual/sql-statements/data-modification/DML/INSERT-OVERWRITE.md @@ -26,9 +26,6 @@ specific language governing permissions and limitations under the License. --> - - - ## Description The function of this statement is to overwrite a table or some partitions of a table @@ -72,7 +69,7 @@ Notice: 1. In the current version, the session variable `enable_insert_strict` is set to `true` by default. If some data that does not conform to the format of the target table is filtered out during the execution of the `INSERT OVERWRITE` statement, such as when overwriting a partition and not all partition conditions are satisfied, overwriting the target table will fail. 2. The `INSERT OVERWRITE` statement first creates a new table, inserts the data to be overwritten into the new table, and then atomically replaces the old table with the new table and modifies its name. Therefore, during the process of overwriting the table, the data in the old table can still be accessed normally until the overwriting is completed. -#### For Auto Partition Table +### For Auto Partition Table If the target table of the INSERT OVERWRITE is an autopartitioned table, the behaviour is controlled by the [Session Variable](../../session/variable/SET-VARIABLE.md) `enable_auto_create_when_overwrite` controls the behaviour as follows: @@ -80,7 +77,25 @@ If the target table of the INSERT OVERWRITE is an autopartitioned table, the beh 2. If an overwrite PARTITION is specified, the AUTO PARTITION table behaves as a normal partitioned table during this process, and data that does not satisfy the conditions of an existing partition is filtered instead of creating a new partition. 3. If you specify PARTITION as `partition(*)` (auto detect partition and overwrite), when `enable_auto_create_when_overwrite` is `true`, for the data that have corresponding partitions in the table, overwrite their corresponding partitions, and leave the other existing partitions unchanged. At the same time, for data without corresponding partitions, create partitions according to the table's auto-partitioning rules, and accommodate the data without corresponding partitions. If `enable_auto_create_when_overwrite` is `false`, data for which no partition is found will accumulate error rows until it fails. -`enable_auto_create_when_overwrite` was introduced since 2.1.6. In versions without `enable_auto_create_when_overwrite`, the behaviour is as if the variable had a value of `false`. +`enable_auto_create_when_overwrite` was introduced since 3.0.3. In versions without `enable_auto_create_when_overwrite`, the behaviour is as if the variable had a value of `false`. + +The quick check conclusion is as follows: + +1. For auto-partition tables that have `enable_auto_create_when_overwrite` enabled: + +| | Partitions to Overwrite | Clear Other Partitions | Auto Create for Unpartitioned Data | +|-|-|-|-| +| Unlabeled (Full Table) | All | √ | √ | +| Designated Partition | Explicit partition | × | × | +| `partition(*)` | The partition that the data belongs to | × | √ | + +2. For trivial tables, auto-partition tables disabled `enable_auto_create_when_overwrite`: + +| | Partitions to Overwrite | Clear Other Partitions | Auto Create for Unpartitioned Data | +|-|-|-|-| +| Unlabeled (Full Table) | All | √ | × | +| Designated Partition | Explicit partition | × | × | +| `partition(*)` | The partition that the data belongs to | × | × | Examples are shown below: @@ -155,7 +170,7 @@ PROPERTIES ( ); ``` -#### Overwrite Table +### Overwrite Table 1. Overwrite the `test` table using the form of `VALUES`. @@ -195,8 +210,7 @@ PROPERTIES ( - Users can use the `SHOW LOAD;` command to check the status of the job imported by this `label`. It should be noted that the label is unique. - -#### Overwrite Table Partition +### Overwrite Table Partition When using INSERT OVERWRITE to rewrite partitions, we actually encapsulate the following three steps into a single transaction and execute it. If it fails halfway through, the operations that have been performed will be rolled back: @@ -243,8 +257,7 @@ The following is examples: INSERT OVERWRITE table test PARTITION(p1,p2) WITH LABEL `label4` (c1, c2) SELECT * from test2; ``` - -#### Overwrite Auto Detect Partition +### Overwrite Auto Detect Partition > This feature is available since version 2.1.3. @@ -297,4 +310,4 @@ As you can see, all data in partitions `p10` and `pMAX`, where data 3 and 1234 a ## Keywords - INSERT OVERWRITE, OVERWRITE, AUTO DETECT +INSERT OVERWRITE, OVERWRITE, AUTO DETECT