v4.2.1_CE_BP2

Version information

Information	Description
Release date	December 7, 2023
Version	V4.2.1_CE_BP2
Commit number	ccdde7d3
OBServer RPM version	oceanbase-ce-4.2.1.2-102000042023120514

Enhanced features

• The MySQL mode of OceanBase Database now supports paging queries while maintaining the same sorting method for each query.
• A new command TRANSFER PARTITION is provided to manually adjust the distribution of partitions. With this command, you can migrate specific partitions to specific log streams to aggregate or scatter partitions.
• OBKV now supports data compression for remote procedure calls (RPCs), reducing network bandwidth usage and user costs.
• This version now allows you to set a threshold for switching the storage method of LOB data from INROW (stored in the primary table by using row storage) to OUTROW (stored in a separate LOB auxiliary table). This enhancement aims to improve the performance of LOB data queries.

Product behavioral changes

• The backup_data_file_size parameter, which is related to backup and restore, is changed from a cluster-level parameter to a tenant-level parameter.

• The tableapi_transport_compress_func parameter, which is related to TableAPI, is renamed as kv_transport_compress_func.

• A new parameter kv_transport_compress_threshold is introduced to specify the minimum threshold for compressing OBKV query result sets.

• The value range for adaptive adjustment of the server_cpu_quota_min and server_cpu_quota_max parameters is increased to meet requirements of larger specifications.

  cpu_count	(0C, 8C)	[8C, 16C)	[16C, 32C)	[32C, +∞)
  server_cpu_quota_min	1C	2C	3C	4C
  server_cpu_quota_max	1C	2C	3C	4C

Bug fixes

• Fixed the issue where the disk bandwidth was used up during a major compaction after a large amount of data was deleted. This issue occurred because, for each deleted data row, a macroblock was opened to verify whether the data row exists in the macroblock.
• Fixed the issue where a core dump occurred on an OBServer node due to a concurrency error in an earlier version of the OpenSSL library during concurrent calls to encryption functions.
• Fixed the issue where, during the execution of a transfer task, errors were encountered and LOB data writes were repeatedly retried due to the failure to refresh the location cache.
• Fixed the issue where, during the concurrent creation of indexes on tables with many partitions, the concurrent execution of INNER SQL statements internally could result in a livelock, causing the task to be unable to progress.
• Fixed the issue where the period of time consumed in executing UPDATE operations on tables containing LOB data was long.
• Fixed the issue where excessive disk space was occupied due to the retention of data snapshots during bypass imports of tables containing LOB data.
• Fixed the issue where modifying the auto-increment column attribute in a table schema and performing a Truncate operation would result in Error 4016 due to the incorrect updates in internal tables.
• Fixed the issue where data of multiple versions may exist when expired data was deleted based on MaxVersions in the HBase mode of OBKV.
• Fixed the issue of memory leaks within the SysJobMgr module that caused Error 4013 due to memory exhaustion in the sys500 tenant during the execution of table-level restore tasks.
• Fixed the issue where the units remained in the WAIT GC state for a long period of time and failed to release after deleting a tenant.
• Fixed the issue where, even after setting IOPS for resource isolation, the number of I/O operations would still exceed the limits in some scenarios.
• Fixed the issue where a long period of time was consumed in a major compaction that involved millions of tablets.
• Optimized the log synchronization and physical restore performance.
• Optimized the performance of spatial index queries by reducing the number of table access operations and minimizing the performance overhead in spatial relationship calculations.

Considerations

• Before V4.2.1_CE_BP2, there was a problem with the limitation of PX threads, potentially resulting in high CPU load on OBServer nodes. However, this issue has been resolved in V4.2.1_CE_BP2, and an upgrade is recommended.

• In the current version, there are some instability issues when using the INSERT INTO SELECT statement for bypass import, particularly when triggering a data transfer. Therefore, it is not recommended to use the INSERT INTO SELECT statement for bypass import.

版本信息

项目	描述
发布日期	2023-12-07
版本号	V4.2.1_CE_BP2
Commit 号	ccdde7d3
OBServer RPM 版本号	oceanbase-ce-4.2.1.2-102000042023120514

特性增强

• MySQL Mode 支持分页查询保序。
• 新增 TRANSFER PARTITION 命令用于手动调整分区分布的能力。通过该命令，用户可以选择将特定的分区迁移到特定的日志流上，从而实现不同分区的聚合或打散分布。
• OBKV 支持 RPC 数据压缩，减小网络带宽占用，降低用户成本。
• 支持设置 LOB 数据存储方式由 INROW （和主表行存储在一起）转换为 OUTROW（将 LOB 数据存储在 LOB 辅助表中）的阈值，优化了大对象场景下的性能。

产品行为变更

• 备份恢复相关配置项 backup_data_file_size 由集群级调整为租户级。

• tableAPI 相关配置项 tableapi_transport_compress_func 更名为 kv_transport_compress_func。

• 新增配置项 kv_transport_compress_threshold 用于指定需要进行压缩的 OBKV 查询结果集大小的最小阈值。

• 配置项 server_cpu_quota_min 和 server_cpu_quota_max 增大自适应变更的范围，以满足更大规格的需求。

  cpu_count	(0c, 8c)	[8c, 16c)	[16c, 32c)	[32c, +∞)
  server_cpu_quota_min	1c	2c	3c	4c
  server_cpu_quota_max	1c	2c	3c	4c

缺陷修复

• 修复在大量删除数据后做合并操作时，由于每行删除的数据都会打开对应宏块，判断是否存在于宏块中，导致磁盘带宽打爆的问题。
• 修复并发调用加密函数时，由于低版本 OpenSSL 库存在并发问题，可能导致 OBServer Core 的问题。
• 修复 Transfer 任务执行期间，在写入LOB 数据场景下，由于 Location Cache 没有刷新，导致写入一直重试并报错的问题。
• 修复对分区数较多的表大量并发建索引时，内部发起的 INNER SQL 并发执行可能形成活锁，导致无法推进任务的问题。
• 修复对包含 LOB 类型数据的表执行 Update 操作耗时高的问题。
• 修复旁路导入包含 LOB 数据时，保留了数据快照，导致磁盘空间占用过多的问题。
• 修复表结构进行自增列属性变更且执行 Truncate 操作后，由于内部表没有正确更新，可能导致表操作报错 4016 的问题。
• 修复 OBKV HBase 模式基于 MaxVersions 进行过期删除时，可能存在多版本数据的问题。
• 修复执行表级恢复任务时，SysJobMgr 模块内存泄漏导致 500 租户内存耗尽报错 4013 的问题。
• 修复删除租户后，Unit 长时间不释放，状态卡在 WAIT GC 的问题。
• 修复设置 IOPS 资源隔离后，部分场景下 IO 仍会超出限制的问题。
• 优化百万 Tablet 场景下，合并耗时高的问题。
• 优化日志同步以及物理恢复性能。
• 优化空间索引查询性能，减少回表次数并降低空间关系计算的性能开销。

注意事项

• V4.2.1_CE_BP2 之前版本，存在 PX 线程限制失效问题，可能会导致 OBServer 的 CPU 负载过高，该问题在 V4.2.1_CE_BP2 已修复，并推荐进行升级。
• 当前版本使用 INSERT INTO SELECT 语句旁路导入数据时，如果触发 Transfer，会存在一些不稳定的问题，因此不推荐通过此方法进行旁路导入。