Release Note 1.2.1

[morningman]

For 1.2.0 Release Note, See: Release Note 1.2.0

[中文 ChangeLog 见二楼]

Improvement

1. Support Iceberg V2

   Support Iceberg V2 (only Position Delete is supported, Equality Delete will be supported in subsequent versions).

   Tables in Iceberg V2 format can be accessed through the Multi-Catalog feature.

2. Support OR condition to IN ([enhancement](rewrite) add OrToIn rule && fix expr clone problems #12872) ([enhancement](session var) varariable to control whether to rewrite OR to IN or not #15437)

   Search rewrite_or_to_in_predicate_threshold in Documentation: https://doris.apache.org/en/docs/dev/advanced/variables

3. Add function substring_index ([improvment](function) add new function substring_index #15373)

   Documentation: https://doris.apache.org/en/docs/dev/sql-manual/sql-functions/string-functions/substring_index

4. Optimize the import and query performance of JSONB type ([Improvement](JSONB) improve performance JSONB initial json parsing using simdjson #15219)

5. Stream load supports quoted csv data ([feature](load) stream load trim double quotes for csv #15241)

   Search trim_double_quotes in Document: https://doris.apache.org/en/docs/dev/sql-manual/sql-reference/Data-Manipulation-Statements/Load/STREAM-LOAD

6. Broker supports Tencent Cloud CHDFS ([Improvement](multi-catalog) support hive external tables which store data on tencent chdfs #15297) and Baidu Cloud BOS, AFS ([Broker](bos) support baidu bos object storage file system for broker #15448)

   Data on CHDFS, BOS, and AFS can be accessed through Broker.

Bug Fix

1. In some cases, after upgrading from version 1.1 to version 1.2, the user permission information will be lost ([fix](auth) fix bug that user info may lost when upgrading to 1.2.0 #15144)

2. Fix the problem that the partition value is wrong when using datev2/datetimev2 type for partitioning ([Bug](datev2) Fix wrong result when use datev2 as partition key #15094)

3. Bug fixes for a large number of released features

   For a complete list see：https://github.com/apache/doris/pulls?q=is%3Apr+label%3Adev%2F1.2.1-merged+is%3Aclosed

Upgrade Notice

Known Issues

• Do not use JDK11 as the runtime JDK of BE, it will cause BE Crash.

Behavior Changed

1. The default value of the BE configuration item high_priority_flush_thread_num_per_store is changed from 1 to 6 ([improvement](config) update high_priority_flush_thread_num_per_store default value to 6 #14775)

   To improve the write efficiency of Routine Load.

2. The default value of the FE configuration item enable_new_load_scan_node is changed to true. ([config](load) enable new load scan node by default #14808)

   Import tasks will be performed using the new File Scan Node. No impact on users.

3. Delete the FE configuration item enable_multi_catalog.

   The Multi-Catalog function is enabled by default.

4. Decimal v3 precision display changes.

   When the user uses the Decimal V3 type, the result will be returned according to the actual defined precision. And support the precision derivation of function transformation.

5. The vectorized execution engine is forced to be enabled by default ([branch-1.2](vec) enable vec enable by default #15213)

   The session variable enable_vectorized_engine will no longer take effect. Enabled by default.

   To make it valid again, set the FE configuration item disable_enable_vectorized_engine to false, and restart FE to make enable_vectorized_engine valid again.

Big Thanks

Thanks to ALL who contributed to this release! (alphabetically)

@adonis0147
@AshinGau
@BePPPower
@BiteTheDDDDt
@ByteYue
@caiconghui
@cambyzju
@chenlinzhong
@dataroaring
@Doris-Extras
@dutyu
@eldenmoon
@englefly
@freemandealer
@Gabriel39
@HappenLee
@Henry2SS
@hf200012
@jacktengg
@Jibing-Li
@Kikyou1997
@liaoxin01
@luozenglin
@morningman
@morrySnow
@mrhhsg
@nextdreamblue
@qidaye
@spaces-X
@starocean999
@wangshuo128
@weizuo93
@wsjz
@xiaokang
@xinyiZzz
@xutaoustc
@yangzhg
@yiguolei
@yixiutt
@Yulei-Yang
@yuxuan-luo
@zenoyang
@zhangstar333
@zhannngchen
@zhengshengjun

[morningman]

For 1.2.0 Release Note, See: Release Note 1.2.0

Improvement

1. 支持 Iceberg V2

   支持 Iceberg V2 (仅支持 Position Delete， Equality Delete 会在后续版本支持)。可以通过 Multi-Catalog 功能访问 Iceberg V2 格式的表。

2. 支持 OR 条件转 IN ([enhancement](rewrite) add OrToIn rule && fix expr clone problems #12872) ([enhancement](session var) varariable to control whether to rewrite OR to IN or not #15437)

   文档：TODO

3. 新增函数 substring_index ([improvment](function) add new function substring_index #15373)

   文档：https://doris.apache.org/zh-CN/docs/dev/sql-manual/sql-functions/string-functions/substring_index

4. 优化 JSONB 类型的导入和查询性能 ([Improvement](JSONB) improve performance JSONB initial json parsing using simdjson #15219)

5. Stream load 支持带引号的 csv 数据 ([feature](load) stream load trim double quotes for csv #15241)

   文档搜索 trim_double_quotes：https://doris.apache.org/zh-CN/docs/dev/sql-manual/sql-reference/Data-Manipulation-Statements/Load/STREAM-LOAD

6. Broker 支持腾讯云 CHDFS ([Improvement](multi-catalog) support hive external tables which store data on tencent chdfs #15297) 和 百度云 BOS、AFS ([Broker](bos) support baidu bos object storage file system for broker #15448)

   可以通过 Broker 访问 CHDFS、BOS、AFS 上的数据。

Bug Fix

1. 修复部分情况下，从 1.1 版本升级到1.2版本后，用户权限信息丢失的问题 ([fix](auth) fix bug that user info may lost when upgrading to 1.2.0 #15144)

2. 修复使用 date/datetimev2 类型进行分区时，分区值错误的问题 ([Bug](datev2) Fix wrong result when use datev2 as partition key #15094)

3. 修复大量已发布功能的 Bug

   完整列表可参阅：https://github.com/apache/doris/pulls?q=is%3Apr+label%3Adev%2F1.2.1-merged+is%3Aclosed

Upgrade Notice

Known Issues

• 请勿使用 JDK11 作为 BE 的运行时 JDK，会导致 BE Crash。

Behavior Changed

1. BE 配置项 high_priority_flush_thread_num_per_store 默认值由 1 改成 6 ([improvement](config) update high_priority_flush_thread_num_per_store default value to 6 #14775)，以提升 Routine Load 的写入效率。

2. FE 配置项 enable_new_load_scan_node 默认值改为 true。([config](load) enable new load scan node by default #14808)

   将使用新的 File Scan Node 执行导入任务。对用户无影响。

3. 删除FE 配置项 enable_multi_catalog。

   Multi-Catalog 功能默认开启。

4. Decimal v3 精度显示变化。

   当用户使用 Decimal V3 类型时，将按照实际定义精度返回结果。并且支持函数变换的精度推导。

5. 默认强制开启向量化执行引擎 ([branch-1.2](vec) enable vec enable by default #15213)

   会话变量 enable_vectorized_engine 将不再生效。默认开启。如需重新生效，需将 FE 配置项 disable_enable_vectorized_engine 设为 false，并重启 FE，以重新生效 enable_vectorized_engine。