-
Notifications
You must be signed in to change notification settings - Fork 5.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix cte filter pushdown wrong results by splitting SpecialFormExpressions #22700
Conversation
fbeb8cc
to
07e1101
Compare
07e1101
to
515cb5a
Compare
} | ||
return new FilterNode(node.getSourceLocation(), idAllocator.getNextId(), node, predicate); | ||
resultPredicate = SimplifyRowExpressions.rewrite(resultPredicate, metadata, session.toConnectorSession()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Simplifying the or conditions is essential and I observed a case in prod where it helped push these filters further down in predicatePushdown rule
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Curious what does this simplify do? Is it to rebalance multiple OR condition?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah looks like it breaks down the multiple ors into efficient cnf and dnf forms. In the predicate pushdown optimizer, the current visitJoin method appears to only decompose projects when they are pre broken by this optimizer.
I've noticed instances where 'or' conditions were not pushed down beneath joins.
Just a nit, suggest release note entry revision.
|
The release notes should mention that it's a wrong results fix and a bit more user facing. e.g. maybe something like
|
Can be separate but we should add argument checks when creating special form expressions that they have the right number of arguments. |
Description
Fixes #22147. This is needed due to a bug in SpecialFormExpressions #22698 if > 2 conditions are used.
Also simplified the extra filters so that predicatePushdown optimizer could push down the conditions further
Motivation and Context
Impact
Test Plan
Contributor checklist
Release Notes
Please follow release notes guidelines and fill in the release notes below.