branch-4.1 [fix](fd) Propagate constants out of one row relation to enable partition pruning - #67400
Draft
englefly wants to merge 3 commits into
Draft
branch-4.1 [fix](fd) Propagate constants out of one row relation to enable partition pruning#67400englefly wants to merge 3 commits into
englefly wants to merge 3 commits into
Conversation
…e partition pruning Problem Summary: A CTE that defines a single constant row (e.g. `WITH params AS (SELECT CAST(...) AS begin_time, ...)`) is inlined into its consumers as a one-row relation, but the constant projects were only registered as uniform slots without values. Constant propagation and predicate inference outside the relation therefore could not substitute the constant values into predicates over the CTE columns, so predicates like `dt BETWEEN DATE_SUB(params.begin_time, INTERVAL params.period_days DAY) AND DATE_SUB(params.begin_time, INTERVAL 1 DAY)` were not folded, the `dt` predicates were left as nested-loop-join conjuncts with runtime filters, and partition pruning failed: the scan read all 526/537 partitions instead of the single needed partition. Fix: `LogicalOneRowRelation.computeUniform` now registers the values of constant projects as uniform constants, so constant propagation can substitute them into predicates over the CTE columns, fold functions such as DATE_SUB over them, and push the resulting predicates into the scan for partition pruning. None - Test: FE unit test ConstantCteTest (asserts both scans prune to the expected single partition); regression test test_constant_cte_partition_prune. - Behavior changed: No - Does this need documentation: No
…ghJoin for one-row-relation constant propagation ### What problem does this PR solve? Problem Summary: LogicalOneRowRelation.computeUniform now registers the values of a one-row relation's constant projects as uniform literals (constant CTE propagation). For the query `select coalesce(element_at(s, 'city'), 'abc') from (select * from tbl)a join (select 100 id, 'f1' name)b on a.id=b.id`, the join condition `a.id = b.id` now folds to `a.id = 100`, the predicate is pushed into a filter above the scan, and the join becomes a cross join. The left project therefore no longer needs to output the join key `id` and keeps only the pushed-down nested column access `element_at(s, 'city')` (1 project instead of 2). The old test asserted the previous plan shape and failed; update the assertion to the new expected plan, which is semantically equivalent and enables pushing `id = 100` into the scan. ### Release note None ### Check List (For Author) - Test: PruneNestedColumnTest (50/50) and ConstantCteTest pass - Behavior changed: No - Does this need documentation: No
…relation constant propagation ### What problem does this PR solve? Problem Summary: - pull_up_predicate_literal: after LogicalOneRowRelation.computeUniform registers the values of a one-row relation's constant projects as uniform literals, the join condition `tmp.col1 = ds.col1 AND tmp.col2 = ds.col2` (tmp being a one-row relation with the literals 'abc'/'def') is folded into `ds.col1 = 'abc' AND ds.col2 = 'def'` and pushed into a filter below the scan; the join itself keeps no condition and becomes a nested loop join. Regenerate the .out file for the two view-based queries; the remaining queries in the suite use project-over-scan constant subqueries and are unaffected. - test_constant_cte_partition_prune: the new test explains the expected single-partition pruning of a constant CTE, but on an empty table PRUNE_EMPTY_PARTITION rewrites the scans into empty relations, so the `partitions=1/6 (p20260728)` info never appears. Seed each partition with one row so the scans stay and the asserted pruning shows up. ### Release note None ### Check List (For Author) - Test: regression tests pull_up_predicate_literal and test_constant_cte_partition_prune pass - Behavior changed: No - Does this need documentation: No
Contributor
Author
|
run buildall |
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cherry-pick of #66905 to branch-4.1
fix Propagate constants out of one row relation to enable partition pruning