4.0.8 Release Notes

[morningman]

4.0.8 Release Notes

4.0.7 Release Notes: #65399

Apache Doris 4.0.8 is a maintenance release in the 4.0 series. It focuses on compute-storage decoupled deployments, load and transaction stability, File Cache behavior, security hardening of internal endpoints, and Lakehouse compatibility. All 4.0.x users are advised to upgrade.

Highlights of this release include:

• Security hardening: authentication and a configuration gate for the BE _stream_load_forward endpoint, an internal auth token for FE meta-service HTTP calls, auth checks on manager REST APIs, and masking of Kafka Routine Load credentials.
• Cloud enhancements: rendezvous hashing for colocate tablet placement, snapshot reads of the table version to avoid transaction conflicts, a bthread-aware tablet header lock, and a new mode that writes only index content into File Cache.
• Upgrade behavior: clusters upgrading from 3.x now enable the Nereids distribute planner instead of silently keeping the legacy one.
• Many correctness and stability fixes covering Group Commit, transactions, Routine Load, Paimon and Iceberg reads, File Cache, memory tracking, and several BE crash and deadlock paths.

Before upgrading, read the Behavior Changes section below.

Behavior Changes

• Upgrading to 4.0 now enables the Nereids distribute planner. A cluster whose variable version is below 400 previously restored a persisted enable_nereids_distribute_planner=false from its image and kept using the legacy distribute planner after the upgrade; the variable-version-400 migration now enables it ([opt](planner) Enable Nereids distribute planner after upgrade #66604).
• Compaction is no longer paused on high memory by default. Pausing made high-Compaction-Score tablets harder to merge, which reinforced the metadata and memory backlog it was meant to relieve ([emhancement](compaction) disable compaction pause on high memory by default #61288).
• The BE _stream_load_forward endpoint is now registered only when the new BE configuration enable_group_commit_streamload_be_forward is enabled (default false), and it now requires authentication. Deployments that rely on Group Commit Stream Load forwarding must enable the configuration on both FE and BE ([fix](security) Add auth and config gate for _stream_load_forward endpoint #64935).
• autobucket_min_buckets now defaults to 3 instead of 1, so the auto bucket strategy no longer computes a bucket count too small for reasonable parallelism and data distribution ([fix](bucket) Raise autobucket_min_buckets default from 1 to 3 #63729).
• The Recycler no longer enables enable_mark_delete_rowset_before_recycle or enable_abort_txn_and_job_for_delete_rowset_before_recycle by default, because the feature currently carries a performance cost ([chore](recycler) Disable enable_mark_delete_rowset_before_recycle and enable_abort_txn_and_job_for_delete_rowset_before_recycle for recycler #66150).
• S3 SlowDown retry handling is configurable again and now differs by role: rate-limited retries remain enabled for BE clients and are disabled for the Recycler, refining the change made in 4.0.7 ([chore](s3) Adjust S3 SlowDown retry configurable #65508).
• The unused session variable plan_nereids_dump has been removed. It only applied to minidump replay and is now an internal flag set by MinidumpUtils, so PLAY '<dumpfile>' continues to work ([chore](session-variable) remove the unused session variable plan_nereids_dump #66371).
• In Cloud mode, tablet data size is consistently reported as remote size, so SHOW TABLETS and information_schema.partitions no longer diverge from information_schema.backend_tablets ([fix] (cloud) Fix local/remote tablet size semantics in schema views #60887).
• In Cloud mode, SHOW PROC '/colocation_group/{GroupId}' now reports ReplicaAllocation as NULL, and SHOW PROC '/cluster_health/tablet_health' no longer reports UNRECOVERABLE because of local-mode health checks ([fix](cloud) Align colocate proc output and tablet health in cloud mode #60944).

New Features & Improvements

Compute-Storage Decoupling

• Support writing only index content into File Cache through the new BE configuration enable_file_cache_write_index_file_only (default false). Rowset writes and Compaction outputs then protect Inverted Index files and Segment index and footer ranges, instead of polluting a limited local cache with large data pages ([feature](cloud) Support file cache write index only #64995).
• Use rendezvous hashing for colocate tablet placement ([improvement](cloud) Use rendezvous hashing for colocate tablet placement #64638).
• Add an option to skip File Cache writes during TopN lazy materialization ([improvement](topn) Add option to skip file cache writes in topn lazy materialization #65021).
• Use snapshot reads for the table-version hint in commit_txn, commit_partition, commit_index and drop_partition, removing the read-conflict range that made concurrent commits on the same table fail with KV_TXN_CONFLICT ([fix](cloud) use snapshot read for table version to avoid txn conflict #64647).
• Increase the get-version timeout to 30 seconds ([enhance](cloud) Increase get version timeout to 30 seconds #65625).
• Support multiple configuration files in start.sh ([opt](cloud) Support multiple config files in start.sh #63924).
• Add an HTTP API to synchronize and export Cloud FE metadata, allowing metadata backup across FE and FDB ([feat](http) Add sync and export cloud meta API #60739).

Security & Access Control

• Add fe_meta_auth_token for FE meta-service internal HTTP authentication ([improvement](fe) add fe_meta_auth_token for FE meta-service internal HTTP auth #65551).
• Add auth checks for the manager node and query-error REST APIs ([fix](auth) add auth check for manager node and query qerror REST APIs #65042).
• Mask sensitive Kafka properties of Routine Load jobs ([fix](fe) Mask Kafka routine load sensitive properties #64786).
• Reset skipAuth on all INSERT OVERWRITE exit paths ([fix](insert) reset skipAuth on all INSERT OVERWRITE exit paths #66383).

Observability & Operations

• Show the first error message of a Routine Load job ([feat](job) show first error message for routine load #65553).
• Record the task finish time when a scheduled job execution fails ([fix](job scheduler) Record task finish time on execution failure #66232).
• Add a virtual compute group switch metric ([improvement](fe) Add virtual compute group switch metric #63036).
• Add queue time for delete bitmap tasks ([improvement](be) Add queue time for delete bitmap tasks #65248).
• Show warm-up timestamps with the date ([fix](fe) Show warm-up timestamps with date #64606).
• Report the root cause of a failed Iceberg metadata commit instead of the misleading Self-suppression not permitted message (branch-4.0: [fix](docker)(case) Fix the remaining 4 external regression failures: kerberos HDFS SASL, Paimon fixture, file cache flakiness #66417).
• Stop wrapping data and expression errors in a misleading "storage reader" message ([fix](scan) Avoid misleading "storage reader" wrapper for data/expression errors #64755).
• Upgrade third-party dependencies ([fix](fe) address dependency check findings #62858, [chore](deps) remove unused commons-lang (2.x) from fe #64196, [chore](dep)Upgrade dependencies #64208).

Performance

• Optimize the information_schema.tables status scan for large OLAP tables: BE now passes the required columns to FE, so lightweight queries no longer pay for tablet-derived status fields ([opt](schema) Optimize tables schema scan status #64933).
• Push down expandable S3 glob prefixes ([perf](s3) push down expandable S3 glob prefixes #64684).
• Speed up the File Cache LRU restore path during BE startup ([fix](file-cache): speed up LRU restore startup path #65174).
• Cache the credentials provider in ConfigurationAWSCredentialsProvider for Glue catalogs ([fix](glue) cache credentials provider in ConfigurationAWSCredentialsProvider #65165).
• Reuse a single Ranger Doris access controller, so a metadata checkpoint no longer leaks two PolicyRefresher threads ([fix](fe) Reuse Ranger Doris access controller #65570).

Important Bug Fixes

Query Result Correctness

• Verify injectivity before SimplifyAggGroupBy rewrites a GROUP BY expression ([fix](simplify agg) SimplifyAggGroupBy should verify injectivity #64335).
• Reject a mismatched compare plan in PullUpJoinFromUnionAll ([fix](rewrite rule) Reject mismatched compare plan in PullUpJoinFromUnionAll #65472).
• Guard UniqueFunction in the filter and TopN pushdown rules ([fix](nereids) Guard UniqueFunction in multiple filter/topn pushdown rules #62742).
• Require the window function partition key chosen by the Partition TopN optimization to be a subset of the co-located keys ([fix](nereids) partition topn opt requires chosen window func partition key to be a subset of co-located ones #64764).
• Fix missing precision and scale when reading legacy decimal data ([fix](be) Fix missing decimal precision and scale for legacy decimal data #65419).
• Correct from_base64 and to_base64 output buffer sizing and invalid input handling ([fix](function) size from_base64/to_base64 output buffer correctly on branch-4.1 #65141).

Load & Transactions

• Fix a lost row when a prepared statement reuses its plan and Group Commit shares one load_id ([fix](group commit) fix lost row when prepared stmt reused plan shares one load_id #64362).
• Avoid aborting a newer transaction that carries the same label ([fix](transaction) avoid aborting newer same-label transactions #64939).
• Fail fast when a Backend restarts during an INSERT ([fix](transaction) Fail fast when a backend restarts during insert #65525).
• Handle generated columns in delete partial update ([fix](fe) Handle generated columns in delete partial update #64884).
• Fix Arrow Stream Load with uppercase column names ([fix](load) Fix Arrow stream load with uppercase columns #65127).
• Support Compute Groups in FE Stream Load routing and planning ([fix](load) support compute_group in FE stream load routing and planning #65571).
• Skip decommissioning Backends when assigning load work ([fix](fe) Skip decommissioning BE for load #65049).
• Let only load source scanners update load counters ([fix](load) only load source scanners update load counters #63781).
• Isolate schemas for CDC snapshot splits in streaming jobs ([fix](streaming-job) Isolate schemas for CDC snapshot splits #65645).

Compute-Storage Decoupling

• Exclude decommissioning Backends from Cloud replica routing, Routine Load assignment, and Kafka proxy selection ([fix](cloud) Exclude decommissioning BE from BE selection #65221).
• Prevent urgent load from preempting the Schema Change lock ([fix](cloud) Prevent urgent load from preempting schema change lock #66082).
• Use a bthread-aware shared mutex for the tablet header lock. The write lock can be held across a suspending warm-up call, and a migrating bthread would otherwise unlock a pthread_rwlock_t from a different worker thread ([fix](cloud) Use bthread-aware shared mutex for tablet header lock #64574).
• Temporarily set version_cache_ttl to 0 when retrying a query that failed with -230 ([fix](fe) set cloud version_cache_ttl to 0 temporarily if retry a query with -230 #63721).
• Inspect and abort failed conflicting transactions while CloudUpgradeMgr waits ([fix](cloud) CloudUpgradeMgr inspect and abort failed conflict txns while waiting #60830).
• Preserve resource IDs when recycling empty Rowsets ([fix](cloud) Preserve resource IDs when recycling empty rowsets #65862).
• Skip packed slice deletion while recycling Rowsets ([opt](recycler) Optimize rowset recycling for recycler #63295).
• Set requestTimeoutMs on the Recycler S3 client to avoid curl-28 errors on slow DeleteObjects calls ([fix](cloud) set recycler S3 client requestTimeoutMs to avoid curl-28 on slow DeleteObjects (mirror #49315) #64758).
• Make the S3 rate limiter configuration take effect dynamically in Cloud mode ([fix](cloud) make S3 rate limiter config take effect dynamically in cloud mode #64554).
• Cancel rebuilt virtual compute group warm-up jobs when the group is dropped ([fix](fe) Cancel rebuilt VCG warm up jobs on drop #65426).
• Clear the warm-up error message after a successful retry ([fix](fe) Clear warm-up error message after successful retry #64813).
• Fix the colocate replica allocation edit log ([fix](fe) Fix colocate replica allocation edit log #65457).
• Resolve conflicting default replica properties, which could leave a stale default.replication_num in table metadata and make SHOW CREATE TABLE report the wrong default allocation ([fix](replica) Resolve conflicting default replica properties #65836).
• Fix a SchemaVariablesScanner crash caused by an incomplete FE response ([fix](cloud) Fix SchemaVariablesScanner crash due to incomplete FE response #65994).

File Cache & Storage

• Fix File Cache queue evict size metrics ([fix](be) Fix file cache queue evict size metrics #64897).
• Bound the LRU recorder shadow queue size ([fix](filecache) Limit LRU recorder shadow queue size #64798).
• Handle asynchronous task submission failures in the S3 file writer ([fix](s3) handle S3 file writer async task submission failures #64779).

Lakehouse & External Data Sources

• Preserve Paimon partition metadata across splits. Partition metadata was generated only when runtime partition pruning was enabled, so count, native and JNI splits reached BE without stable partition values, and the reused prune block was corrupted by inserted replacement columns (branch-4.0: [fix](paimon) preserve partition metadata across splits #65583).
• Use the split file format for Iceberg scans ([bug](iceberg) use split file format for iceberg scan #65760).
• Preserve external table column name case for Iceberg and Paimon ([fix](fe) Preserve external table column name case #65094).
• Support an IOManager for Paimon JNI reads, and add Paimon JNI scanner observability ([improvement](paimon) Support IOManager for JNI reads #65332, [improvement](paimon) Add Paimon JNI scanner observability #65354).
• Validate task executor scan handles ([fix](be) Validate task executor scan handles #65054).
• Fix an Avro JNI reader crash on a null pointer ([bug](avro) fix avro jni reader coredump with nullptr #64699).

Memory & Stability

• Avoid a local Runtime Filter merge deadlock ([fix](be) Avoid local runtime filter merge deadlock #64866).
• Catch block serialization exceptions instead of letting BE core dump ([fix](be) catch block serialization exceptions to avoid coredump #64852).
• Make secondary brpc package aliases non-owning, avoiding a double free during brpc::Server shutdown ([fix](brpc) Make secondary package aliases non-owning #65777).
• Disable the SSL BIO buffer in brpc ([fix](brpc) disable SSL BIO buffer  #64962).
• Force a refresh of the Workload Group configuration when its memory limit changes ([bugfix](memory) should force refresh workload group's config when memlimit changed #65542).
• Update memory statistics more quickly when the user changes the cgroup limit directly ([bugfix](memory) should update memory more quickly when user changes memory in cgroup directly in serveless mode #65695).

Protocol & Observability

• Fix FE memory issues in Arrow Flight SQL: close the temporary VectorSchemaRoot in createPreparedStatement ([fix](arrow-flight-sql) Close temporary VectorSchemaRoot in createPreparedStatement to fix FE direct memory leak #65311), fix the remaining off-heap leaks ([bugfix](arrowflight) fix some off heap memory leak on FE #66437), and keep the coordinator alive across GetFlightInfo and DoGet for external table scans ([fix](arrow-flight) Keep coordinator alive across GetFlightInfo/DoGet for external table scan #64799).
• Serve information_schema.routine_load_jobs from the master FE, so FIRST_ERROR_MSG and ERROR_LOG_URLS are no longer empty when the query lands on a non-master FE (branch-4.0: [fix] fix routine_load_jobs first_error_msg on non-master FE and flaky ann_index_basic #65942).
• Guard execution profile access with the profile manager read lock ([fix](profile) guard execution profile access #65353).
• Fix the queued task count reported by the time-sharing executor ([fix](be) Fix time-sharing executor queued task count #63568).
• Fix a -parameters error thrown by the node action REST API, caused by a PathVariable annotation without a declared value ([Fix](restapi) fix node action restapi throw '-parameters' error because of PathVariable annotation without value declaration #59708).

These notes focus on user-visible and operations-visible changes. Test-only commits, CI adjustments, external Docker environment work, release version updates, and purely internal changes without external symptoms are omitted. Refer to the complete 4.0.7 to 4.0.8-rc02 comparison for the full commit history.

Acknowledgments

Thanks to all contributors whose pull requests are included in this release:

@0AyanamiRei @924060929 @BiteTheDDDDt @bobhan1 @CalvinKirs @deardeng @freemandealer @Gabriel39 @gavinchou @GJ100 @heguanhui @hello-stephen @hubgeter @jacktengg @JNSimba @Jungzhang @liaoxin01 @luwei16 @morningman @Mryange @mymeiyi @shuke987 @sollhui @stevenpall @suxiaogang223 @w41ter @wyxxxcat @xylaaaaa @yiguolei @yujun777 @zgxme @zhangstar333