[test](fe) Establish year-end FEUT CPU baseline#65937
Draft
hello-stephen wants to merge 4477 commits into
Draft
Conversation
… for partial update (apache#58782) ### What problem does this PR solve? related issue: apache#58780 Under the load to 5000-columns wide table, flush memtable consume lots of memory: <img width="526" height="430" alt="image" src="https://github.com/user-attachments/assets/816bcc8b-d9d0-4105-a96b-3fbcc931b0b1" /> Set source content column by column when flush memtable for partial update to solve this problem, the memory usage of `_append_block_with_partial_content` is hardly visible in the profile after optimization.
### What problem does this PR solve? log print flight sql ticket error Co-authored-by: wanglong16 <wanglong16@xiaomi.com>
### What problem does this PR solve? #### Background `build_current_tablet_schema` may consume a significant amount of memory when load to a wide table with a large number of buckets: <img width="716" height="718" alt="image" src="https://github.com/user-attachments/assets/0b08c807-39df-4ab5-b7af-5896e1265009" /> #### Solution Reuse tablet schema in load path to save load memory. #### Test Result Machine configuration: 16c 64G 3BE Table Schema: 5000 column, 256 bucket Each Be reduces 1G memory.
… to make mv regression test case stable (apache#58827) ### What problem does this PR solve? In the regression testing for synchronous materialized views, many cases specify` set enable_stats=false.` This can result in row counts being reported as unknown, causing the Cost-Based Optimizer (CBO) to inconsistently choose query plans that include materialized views. In real‑world environments, enable_statsdefaults to true. Therefore, to stabilize the pipeline and fix the flakiness of synchronous materialized view test cases, we are removing `set enable_stats=false `from the tests.
### What problem does this PR solve? 1. Fixes Pinyin filter behavior inconsistency with Elasticsearch implementation. 2. Increase performance
…ot nodes (apache#58767) PullUpSubqueryAliasToCTE rule is to pull up all subquery alias with leading info into LogicalCTE. But it only do so for UnboundResultSink root node, we need also do it for UnboundTableSink and LogicalFileSink
…che#58728) ### What problem does this PR solve? When a page contains some NULL values, we set both the min value and its associated flag to NULL for filtering purposes in IN and comparison operations. In such cases, we can directly compare the min/max values to determine whether the zone map is effective, without needing to handle the scenario where the min value is NULL. However, if all values in the page are NULL, the entire page can be directly filtered out.
…he#58835) ### What problem does this PR solve? fail reason is recently some PR use two slot id, cause random's slot id + 2
…he#58833) Make scanner wait worker time include the time consumption of the following code: <img width="1360" height="694" alt="QQ_1765247113654" src="https://github.com/user-attachments/assets/54e2fadf-a51c-4bcf-804c-4b1ec52d7eb1" /> This pull request refactors how scanner resource usage counters are updated in `ScannerScheduler::_scanner_scan`. The update logic for scan CPU time and real-time counters is now managed using a deferred execution pattern, ensuring counters are updated immediately after scanning, regardless of how the function exits. This change improves reliability and correctness, especially in error scenarios. Resource usage counter updates: * Replaced the manual update of scan CPU time and real-time counters at the end of the function with a `Defer` object that ensures these counters are updated as soon as possible after scanning, even if the function exits early. * Removed the previous explicit counter update block at the end of the function, as this is now handled by the deferred logic.
…he#58756) Fixes an undercount of compaction IO metrics. Vertical compaction only populated bytes_read_from_local/remote and cached_bytes_total for the key column group, so the final stats missed IO from value column groups. We now collect IO stats per group and accumulate them in vertical_merge_rowsets, while keeping row counters sourced from the key group.
doc: apache/doris-website#3146 ```text Doris> SELECT TO_SECONDS('0000-01-01'); +--------------------------+ | TO_SECONDS('0000-01-01') | +--------------------------+ | 86400 | +--------------------------+ 1 row in set (0.01 sec) ```
…bles (apache#58755) ### What problem does this PR solve? - **Issue Number**: part of apache#58199 - **Related PR**: N/A Problem Summary: This PR implements the `publish_changes` action for Iceberg tables. This action serves as the "Publish" step in the Write-Audit-Publish (WAP) pattern. The procedure locates a snapshot tagged with a specific `wap.id` property and cherry-picks it into the current table state. This allows users to atomically make "staged" data visible after validation. Syntax: ```sql EXECUTE TABLE catalog.db.table_name publish_changes("wap_id" = "batch_123"); ```` Output: Returns `previous_snapshot_id` (STRING) and `current_snapshot_id` (STRING) indicating the state transition. Use cases: 1. Implement Write-Audit-Publish (WAP) workflows. 2. Atomically publish validated data to the main branch. 3. Manage staged snapshots based on custom WAP IDs. Co-authored-by: Chenjunwei <chenjunwei@ChenjunweideMacBook-Pro.local>
### What problem does this PR solve? fix flight sql when contains &, such as: select 'a&b' as c error message: error org.apache.arrow.adbc.core.AdbcException: Malformed ticket, size: 5 Co-authored-by: wanglong16 <wanglong16@xiaomi.com>
…tributes conflict check (apache#58054) In a Routine Load job, if the `load_to_single_tablet `property is set for a table with a `hash distribution` scheme, the load job should be canceled. Similarly, in a Broker Load job with the same configuration, the job should also fail.
…8655) Fix aggregate error: 1. GROUP BY need forbit window expression. ``` MySQL root@127.0.0.1:db1> select a from t group by sum(a) over(); (1105, 'errCode = 2, detailMessage = GROUP BY expression must not contain window functions: sum(a) OVER()') ``` 2. opt GROUP BY hint when its slot reference to aggregate function or window expression. Without this PR, when binding group by expression, it will exclude the aggregate function slots, then if the group by contains aggregate function slots, it will throw unknown column, the hint is not friendly. So when bind group expression, it include the aggregate function slots too. ``` select sum(id) as k from t1 group by k; => exception: Unknown column 'k' in 'table list' in AGGREGATE clause select SUM(id) OVER() as k FROM t1 group by k; => exception: Unknown column 'k' in 'table list' in AGGREGATE clause select sum(id) as x, max(id) as y from t1 group by grouping sets((x), (y)) => exception: aggregate function is not allowed in LOGICAL_PROJECT ``` With this PR: ``` select sum(id) as k from t1 group by k; => exception: GROUP BY expression must not contain aggregate functions: sum(id) select SUM(id) OVER() as k FROM t1 group by k; => exception: GROUP BY expression must not contain window functions: sum(id) OVER() select sum(id) as x, max(id) as y from t1 group by grouping sets((x), (y)) => exception: GROUP BY expression must not contain aggregate functions: sum(id) ``` 4. HAVING need forbit window expression. ``` MySQL root@127.0.0.1:db1> select sum(a) from t group by a having sum(a) over() > 10; (1105, 'errCode = 2, detailMessage = LOGICAL_HAVING can not contains WindowExpression expression: sum(a) OVER()') MySQL root@127.0.0.1:db1> select sum(a) over() as k from t group by a having k > 10; (1105, 'errCode = 2, detailMessage = LOGICAL_HAVING can not contains WindowExpression expression: sum(a) OVER()') ``` Notice the rule CheckAnalysis cann't detect the above error because the aggregate function of having node have been replace with slot reference. 5. run rule CheckAnalysis twice, run twice because fill missing slots will replace expression with slot reference for having / sort / .., then check expression unexpected types may not work. We also merge rule checkAfterBind into checkAnalysis. 6. WINDOW need run after HAVING The root cause is because nereids allow LogicalAggregate contains window expression. When nereids analyze sql, it put window expression into LogicalAggregate too. But this is logical error, Window run after Aggregate, so aggregate should not contains windows expression. To fix this problem, we first try to extract window expression into a top projection, and make sure LogicalAggregate nerver contain window expression, but it make more problems because when analyze, it use pattern match for analyze rewritting. So when add a project for window expressions, then analyze rule BindExpression, FillMissingSlot's pattern need to adapt for the new add PROJECT, well, quite a lot work. So we allow LogicalAggregate contain window expression, though it have logical error, but make pattern match more simple. Then after NormalAggregate, the aggregate node will extract the window expression into a top project node, then aggregate will not contains window expression. At the same time, when apply NormalAggregate, if we have ``` having | -- aggregate(agg function, window expression, other expression) ``` Without this PR, NormalAggregate may generate plan as follow: ``` having |--- project(window expression, other expression) |-- aggregate(agg function) ``` then later the project will rewrite to logical window. we need to pull the window expression above the having node, then we will have: ``` project(window expression) | -- having |-- project(other expression) |-- agg(agg expression) ``` so fix the problem. What's more, we found trino can handle this problem more easy, when analyze from AST, trino don't build logical tree immediately, it will first collect aggreate / window and other information, because it don't use tree and pattern matching, it can analyze the information for combination of having, window, aggregate, order by more easy, then after base analyze finish, it then build the logical tree, and trino can make sure aggregate will nerver contains the window expression.
…dd debug log (apache#58857) ### What problem does this PR solve? ## Overview This pull request (authored by HappenLee) focuses on **performance optimization** (via sorting algorithm replacement) and **observability enhancement** (via logging expansion) for Apache Doris, along with a critical fix to ensure accurate digest calculation in predicate expressions. The changes span core data structure handling, segment iteration, and vectorized expression logic. ## Key Changes ### 1. Performance: Replace `std::sort` with `pdqsort` for Faster Set Sorting - **File**: `be/src/exprs/hybrid_set.h` - Modifications : - Added `#include <pdqsort.h>` to enable the pdqsort algorithm (a fast, adaptive quicksort variant optimized for real-world data). - Replaced ``` std::sort(elems.begin(), elems.end()) ``` with ``` pdqsort(elems.begin(), elems.end()) ``` in three set classes: - `HybridSet`: For generic element type sets. - `StringSet`: For string reference (`StringRef`) sets. - `StringValueSet`: For string value-based sets. - **Purpose**: pdqsort outperforms `std::sort` in most practical scenarios (e.g., partially sorted data, duplicate values), reducing the time to sort elements during digest calculation for set-based operations. ### 2. Observability: Add Debug Logs for Condition Cache Operations - **File**: `be/src/olap/rowset/segment_v2/segment_iterator.cpp` - Modifications : - Cache Hit Logging (Line 132-138): Added ``` VLOG_DEBUG ``` output when a condition cache hit occurs, including: - Query ID (from `_opts.runtime_state->query_id()`). - Segment ID (`_segment->id()`). - Cache digest (`_opts.condition_cache_digest`). - Rowset ID (`_opts.rowset_id.to_string()`). - **Cache Insert Logging** (Line 2379-2383): Added `VLOG_DEBUG` output when inserting data into the condition cache, including the same fields as the hit log. - **Purpose**: Improve debuggability for cache-related issues (e.g., false misses, incorrect cache entries) by linking cache events to specific queries and data segments. ### 3. Correctness: Fix Digest Calculation for `VDirectInPredicate` - **File**: `be/src/vec/exprs/vdirect_in_predicate.h` - Modifications : - Updated the ``` get_digest(uint64_t seed) ``` method to: 1. First incorporate the digest of the predicate’s child expression (`_children[0]->get_digest(seed)`). 2. Only propagate the filter’s digest (`_filter->get_digest(seed)`) if the child digest is non-zero; otherwise, return the original seed. - Replaced the previous implementation (which directly returned `_filter->get_digest(seed)` without including the child expression). - **Purpose**: Ensure the digest uniquely identifies the full predicate logic (including both the child expression and the filter), preventing hash collisions that could lead to incorrect cache lookups or data processing. ## Impact - **Performance**: Faster sorting for set-based digest calculations may reduce latency in query operations involving `IN` predicates or set comparisons. - **Debuggability**: Detailed cache logs enable quicker diagnosis of cache performance issues. - **Correctness**: Fixes a potential source of incorrect digest values, improving the reliability of cache-dependent features (e.g., condition cache, query result caching).
…ache#58666) ```sql SELECT avg((ss_quantity * ss_list_price)) average_sales from store_sales; ``` before: partial_avg(CAST((CAST(ss_quantity[apache#23] AS decimalv3(10,0)) * ss_list_price[apache#24]) AS decimalv3(19,4)))[apache#25] doris 28.42 sec doris disable check 25.17 sec after: partial_avg((CAST(ss_quantity[apache#23] AS decimalv3(10,0)) * ss_list_price[apache#24]))[apache#25] doris 19.14 sec doris disable check 17.12 sec
```sql
select
count(*)
from
(
select
brand_id,
class_id,
category_id
from
(
SELECT
iss.i_brand_id brand_id,
iss.i_class_id class_id,
iss.i_category_id category_id
FROM
store_sales,
item iss,
date_dim d1
WHERE
(ss_item_sk = iss.i_item_sk)
AND (ss_sold_date_sk = d1.d_date_sk)
AND (
d1.d_year BETWEEN 1999
AND (1999 + 2)
)
)tmp
group by
brand_id,
class_id,
category_id
)tmp2;
```
before:
<img width="904" height="262" alt="QQ_1764934227381"
src="https://github.com/user-attachments/assets/771a51d7-049d-49a0-a4af-eab318047c2d"
/>
after:
<img width="808" height="250" alt="QQ_1764934235361"
src="https://github.com/user-attachments/assets/56ea2e41-04d4-4cd7-a3a9-3c1f8eab596c"
/>
This pull request adds support for new fixed-width hash key types,
specifically `UInt96` and `UInt104`, across the aggregation, join, set,
partition, and dictionary hash map utilities in the codebase. The
changes ensure that these new types are fully integrated into the
relevant data structures, hash functions, and test coverage, improving
flexibility and performance for scenarios that require these key sizes.
**Support for new hash key types (`UInt96` and `UInt104`):**
* Added new struct definitions for `UInt96` and `UInt104` in
`uint128.h`, including equality operators.
* Updated the `HashKeyType` enum and `get_hash_key_type_with_fixed`
function to include `fixed96` and `fixed104` options.
[[1]](diffhunk://#diff-4f1fb8a89cd0e13a719c3427b1ae7581b42cb7325755a3ceac4c44bdc64bd144R41-R42)
[[2]](diffhunk://#diff-4f1fb8a89cd0e13a719c3427b1ae7581b42cb7325755a3ceac4c44bdc64bd144R67-R70)
* Implemented `HashCRC32` specializations for `UInt96` and `UInt104` to
enable CRC32 hashing for these types.
**Integration into aggregation, set, join, partition, and dictionary
utilities:**
* Extended the variant types and initialization logic in aggregation
(`agg_utils.h`), distinct aggregation (`distinct_agg_utils.h`), set
(`set_utils.h`), join (`join_utils.h`), partition sort
(`partition_sort_utils.h`), and dictionary hash map
(`complex_dict_hash_map.h`) utilities to support the new key types.
[[1]](diffhunk://#diff-50d8f62236d4e1f81d52e945edee5377b7b22d52e04128eea2c8b7f679b37254R85-R86)
[[2]](diffhunk://#diff-50d8f62236d4e1f81d52e945edee5377b7b22d52e04128eea2c8b7f679b37254R147-R154)
[[3]](diffhunk://#diff-62ad0a1cb1b62de5393935298725cfd2e9766215bdd7653d84cd1fd5e7f59fe3R109-R110)
[[4]](diffhunk://#diff-62ad0a1cb1b62de5393935298725cfd2e9766215bdd7653d84cd1fd5e7f59fe3R166-R173)
[[5]](diffhunk://#diff-8b095a1e764b3856129d9fd06fb9122a7e9eb16bc5c293d8dcaa4ff841a587edR71-R72)
[[6]](diffhunk://#diff-8b095a1e764b3856129d9fd06fb9122a7e9eb16bc5c293d8dcaa4ff841a587edR115-R122)
[[7]](diffhunk://#diff-66cf4052118abf5abbef2e0d9193df3c35a46f70db35853c5884d56d4118a963R70)
[[8]](diffhunk://#diff-66cf4052118abf5abbef2e0d9193df3c35a46f70db35853c5884d56d4118a963R112-R119)
[[9]](diffhunk://#diff-c557434b23ebbb39ef2851b7926d61af5be4bf8f56b83a92b98f9a574f805a90R144-R145)
[[10]](diffhunk://#diff-c557434b23ebbb39ef2851b7926d61af5be4bf8f56b83a92b98f9a574f805a90R209-R216)
[[11]](diffhunk://#diff-60243aa7720001b0983bd282c74f77c8a8542a9a6fed08d80061c4f25847b650R51)
[[12]](diffhunk://#diff-60243aa7720001b0983bd282c74f77c8a8542a9a6fed08d80061c4f25847b650R91-R97)
* Updated template instantiations and type extraction logic to handle
the new key types in join probe implementation.
**Test coverage:**
* Added test cases to verify initialization and type handling for the
new key types in set and distinct aggregation utilities.
[[1]](diffhunk://#diff-9e0e850ab93037077da8e96f7d72b1d45c40835221ccca205cc20ef571115603R167-R176)
[[2]](diffhunk://#diff-96eb9173d84e4c838fcef6dcef716e5e4519ea678f842b4a512c66bbd2f275b1R100-R107)
…me. (apache#58679) ### What problem does this PR solve? Related PR: apache#58124
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
previous used machine `macOS-13` is deprecated. change to `macOS-15` now
… and could not find revocable tasks (apache#59330) ### What problem does this PR solve? There are some problems in the previous code: 1. revocable memory size is ineffective for non-spill operators and remains 0 in all cases. In this scenario, if we disable spill and enable reserve, it appears that memlimit will stop working entirely. 2. too complicated in process reserve failed. In this PR, I simplified the logic: 1. If the query is not enable spill, and if reserve memory failed, just disable reserve memory and it will enable memory check immediately and let the query run. 2. If reserve memory failed and not find any revocable tasks, then just cancel the query. 3. If reserve memory failed, should add to paused query immediately because it maybe the last block for join build. If not paused, it may try to allocate a lot of memory. 4. If there are some query in cancel stage, then not check paused query for a bit while because the cancelling query may release some memory. Related PR: #xxx Problem Summary: ### Release note None ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
…onRegrData (apache#59224) ### What problem does this PR solve? Issue Number: close apache#38977 Problem Summary: This PR migrates regr_sxx/syy/sxy onto the shared Moment(AggregateFunctionRegrData) introduced in apache#55940. The original implementation and tests were done in apache#39187 by @wyxxxcat. This PR builds on top of that work, refactoring it to reuse the same state and merge logic. --------- Co-authored-by: wyxxxcat <1520358997@qq.com>
apache#59204) During the execution of `init_file_cache_factory`, the following call path is triggered: ```txt init_file_cache_factory -> FileCacheFactory::create_file_cache -> cache->initialize() -> initialize_unlocked -> _storage->init(this) -> FSFileCacheStorage::init() ``` (At this point, a thread named `_cache_background_load_thread` is created, and the remaining operations run within this thread) `-> upgrade_cache_dir_if_necessary -> read_file_cache_version -> FileSystem::open_file -> open_file_impl -> LocalFileReader::LocalFileReader -> BeConfDataDirReader::get_data_dir_by_file_path` After `FSFileCacheStorage::init` completes (spawning the `_cache_background_load_thread`), `ExecEnv::_init` continues to execute `doris::io::BeConfDataDirReader::init_be_conf_data_dir`. This function performs push operations on `be_config_data_dir_list`. Simultaneously, `BeConfDataDirReader::get_data_dir_by_file_path` (running in the background thread) iterates over this same `be_config_data_dir_list`. This leads to a race condition: if `doris::io::BeConfDataDirReader::init_be_conf_data_dir` is inserting data while the vector is being read, two issues arise: 1. Modifying `be_config_data_dir_list` while iterating over it via a range-based for loop results in **Undefined Behavior (UB)**. 2. If `be_config_data_dir_list` triggers a reallocation (expansion) during the insertion, concurrent read operations on its elements will access dangling references, triggering a **heap-use-after-free** error. Since `init_be_conf_data_dir` depends on `cache_paths` derived from `init_file_cache_factory`, we must carefully manage the synchronization sequence to prevent these errors.
…sequence mapping (apache#54936) In some business scenarios, business operations need to update different columns in the same wide table through two or more data streams. For example, one data stream will write in real time to update some fields of this table; another data stream will perform imports on demand to update other columns of this table. During the update process, both data stream jobs need to ensure the order of replacement; and during queries, data from all columns should be queryable. ```shell CREATE TABLE `upsert_test` ( `a` bigint(20) NULL COMMENT "", `b` int(11) NULL COMMENT "", `c` int(11) NULL COMMENT "", `d` int(11) NULL COMMENT "", `e` int(11) NULL COMMENT "", `s1` int(11) NULL COMMENT "", `s2` int(11) NULL COMMENT "" ) ENGINE=OLAP UNIQUE KEY(`a`, `b`) COMMENT "OLAP" DISTRIBUTED BY HASH(`a`, `b`) BUCKETS 1 PROPERTIES ( "replication_num" = "1", "sequence_mapping.s1" = "c,d", "sequence_mapping.s2" = "e" ); MySQL [test]> insert into upsert_test(a, b, c, d, s1) values (1,1,2,2,2); Query OK, 1 row affected (0.15 sec) {'label':'insert_fadb7c3fe6041c6-a51094f635017b12', 'status':'VISIBLE', 'txnId':'2'} MySQL [test]> insert into upsert_test(a, b, c, d, s1) values (1,1,1,1,1); Query OK, 1 row affected (0.07 sec) {'label':'insert_1f3d1d5eb28447fe-889427c9da075c11', 'status':'VISIBLE', 'txnId':'3'} MySQL [test]> select * from upsert_test; +------+------+------+------+------+------+------+ | a | b | c | d | e | s1 | s2 | +------+------+------+------+------+------+------+ | 1 | 1 | 2 | 2 | NULL | 2 | NULL | +------+------+------+------+------+------+------+ MySQL [test]> insert into upsert_test(a, b, e, s2) values (1,1,2,2); Query OK, 1 row affected (0.05 sec) {'label':'insert_3b9614ce9a4e4dc3-97bbbb9b881c24aa', 'status':'VISIBLE', 'txnId':'4'} MySQL [test]> select * from upsert_test; +------+------+------+------+------+------+------+ | a | b | c | d | e | s1 | s2 | +------+------+------+------+------+------+------+ | 1 | 1 | 2 | 2 | 2 | 2 | 2 | +------+------+------+------+------+------+------+ 1 row in set (0.01 sec) ``` documentation: apache/doris-website#2954
…ax score after filtering (apache#59268)
…9464) This pull request refactors error handling in the `BitmapValue` class to improve robustness and provide clearer error messages. Instead of using `CHECK` macros, it now throws exceptions with detailed information when inconsistencies are detected in the bitmap value's set count. **Error handling improvements:** * Replaced `CHECK` and `CHECK_EQ` macros with explicit exception throwing (`Exception(ErrorCode::INTERNAL_ERROR, ...)`) when the set count exceeds the threshold or does not match the expected size, including detailed error messages with the problematic values.
…#59478) ### What problem does this PR solve? Issue Number: close #xxx Related PR: #xxx Problem Summary: ### Release note None ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
### What problem does this PR solve?
The function `update_hash_with_value` and `update_hashes_with_value` are
used by some `EngineChecksumTask::_compute_checksum()`.
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
- remove colocate group - change tables' schema
…he#59395) related PR apache#58964 Problem Summary: This pull request refactors how child physical plans are accessed in the `ChildrenPropertiesRegulator` class, simplifying the code and improving test clarity. The main change is the removal of the `getChildPhysicalPlan` helper method, replacing its usage with direct access to the plan from the `children` list. The tests are also updated to build child mocks more locally, improving test isolation and readability. Refactoring and Simplification: * Removed the `getChildPhysicalPlan` method from `ChildrenPropertiesRegulator`, and replaced its usage in `visitPhysicalFilter` and `visitPhysicalProject` with direct access to the child plan via `children.get(0).getPlan()`. This simplifies the code by eliminating unnecessary indirection.
### What problem does this PR solve? Issue Number: close apache#57367
…pache#59445) - Modified ExplainCommand.java to remove checkBlockRules() call - EXPLAIN statements should not be blocked by SQL block rules - Updated test_sql_block_rule.groovy to expect EXPLAIN not blocked - Added comprehensive external table tests for Hive, Paimon, Iceberg - All tests verify EXPLAIN bypasses block rules for partition_num, tablet_num, cardinality, and regex rules
…w group. (apache#59053) ### What problem does this PR solve? Problem Summary: This pull request achieves better filtering by fetching the latest join runtime filter when creating the Parquet row group reader. Previously, the join runtime filter was fetched at the Parquet file level.
### What problem does this PR solve? Issue Number: close #xxx Problem Summary: Run the FE unit-test pipeline against the last master revision from 2025-12-31 to separate code growth from current TeamCity parallelism and agent sizing. This is a disposable test PR and must not be merged. ### Release note None ### Check List (For Author) - Test: No need to test (documentation-only marker used to trigger FEUT) - Behavior changed: No - Does this need documentation: No
Contributor
Author
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
run feut |
### What problem does this PR solve? Issue Number: close #xxx Problem Summary: Make the disposable 2025-12-31 FEUT baseline visible to the existing FE path filter so the current TeamCity FEUT pipeline runs instead of reporting a documentation-only skip. ### Release note None ### Check List (For Author) - Test: No need to test (comment-only trigger marker) - Behavior changed: No - Does this need documentation: No
### What problem does this PR solve? Issue Number: close #xxx Problem Summary: Preserve the exact README layout from the 2025-12-31 baseline so the disposable PR has only the FE test marker as its effective diff. ### Release note None ### Check List (For Author) - Test: No need to test (whitespace restoration only) - Behavior changed: No - Does this need documentation: No
Contributor
Author
|
run feut |
Contributor
Author
|
run feut |
Contributor
Author
FE UT Coverage ReportIncrement line coverage `` 🎉 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
This is a disposable FEUT CPU baseline PR. It checks out the last Apache Doris
masterrevision from 2025-12-31 (Beijing time) and adds a comment marker under the FE test tree so the old code is exercised by the current community TeamCity FEUT pipeline.Do not merge this PR. Close it after the FEUT comparison is complete.
The temporary
hbobase is intentional: the production FEUT runner merges PRs targetingmasterinto the latestmaster, which would replace this historical baseline.hbois accepted by the TeamCity PR provider but is outside that merge-to-latest branch list, so TeamCity executes the historical head directly. The target branch is only a test harness and is not a proposed merge destination.Baseline
e9da494a6a51219484604cde0472d9d926ccbb78[minor](log) Add logs for WorkThreadPool (#59503)62baea354e6What to compare
FE_UT_PARALLELvalueValidation
git diff --check