[fix](fd)drop Function dependencies from join outer side - #65982
Open
englefly wants to merge 1 commit into
Open
Conversation
englefly
requested review from
924060929,
morrySnow and
starocean999
as code owners
July 24, 2026 03:07
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
run buildall |
Contributor
Author
|
/review |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Request changes: one P2 optimizer performance regression remains.
Critical checkpoint conclusions:
- Goal and correctness: the patch prevents unsafe direct functional dependencies from surviving a null-extended outer-join side, and no unsafe edge is reconstructed through unique, uniform, equality, transitive, or pruning paths. However, it also drops still-valid dependencies whose determinant contains an originally non-null child slot; see the inline finding.
- Scope and parallel paths: all 14 current join types, ASOF directionality, semi/anti/NAAJ and mark shapes were checked. The logical and physical hash-join switches agree; nested-loop recomputation blocks FD propagation. No additional path-specific issue was found.
- Tests: the changed FdTest assertions inspect only the logical plan and use NOT NULL determinants, so they codify the safe dependency loss rather than reproduce the nullable-determinant collision. The physical reset/recomputation path is not covered.
- Performance: the accepted issue prevents EliminateGroupByKey and EliminateOrderByKey from applying valid reductions. No other CPU, memory, or hot-path concern was found.
- Concurrency, lifecycle, configuration, compatibility, persistence/transactions, data writes, FE-BE propagation, and observability: not involved in this change.
- User focus: no additional focus was provided; the complete PR was reviewed.
Validation was static only. No build or test was run because the governing review prompt prohibits builds in this environment.
| case NULL_AWARE_LEFT_ANTI_JOIN: | ||
| case LEFT_OUTER_JOIN: | ||
| case ASOF_LEFT_OUTER_JOIN: | ||
| builder.addFuncDepsDG(left().getLogicalProperties().getTrait()); |
Contributor
There was a problem hiding this comment.
[P2] Preserve FDs with an originally non-null determinant
This drops the nullable child's whole FD graph, including edges that remain valid after null extension. In the new test, t2.id is NOT NULL and unique, so t2.id -> t2.id2: matched rows always have a non-null id, while every unmatched row contributes (id,id2)=(NULL,NULL). Those cases cannot collide, yet this arm discards the edge and makes EliminateGroupByKey retain GROUP BY t2.id,t2.id2 (and likewise prevents redundant order-key elimination).
The actual unsafe shape needs a nullable determinant, e.g. a matched (a=NULL, coalesce(a,1)=1) plus an unmatched (NULL,NULL). Please filter nullable-side edges based on whether their determinant can be all-null in an original child row, keep the current NOT NULL-key dependency, and test that nullable collision; the mirrored physical recomputation path should be covered as well.
Contributor
TPC-H: Total hot run time: 23165 ms |
Contributor
TPC-DS: Total hot run time: 117085 ms |
Contributor
ClickBench: Total hot run time: 18.65 s |
morrySnow
approved these changes
Jul 27, 2026
Contributor
|
PR approved by at least one committer and no changes requested. |
github-actions
Bot
added
the
approved
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)