-
-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: Only push predicates depending on the subset columns past unique()
#14668
Changes from 4 commits
ab8229e
553ba5e
8e1f386
6e2978c
5fb343d
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -404,24 +404,14 @@ impl<'a> PredicatePushDown<'a> { | |
input, | ||
options | ||
} => { | ||
|
||
if matches!(options.keep_strategy, UniqueKeepStrategy::Any | UniqueKeepStrategy::None) { | ||
// currently the distinct operation only keeps the first occurrences. | ||
// this may have influence on the pushed down predicates. If the pushed down predicates | ||
// contain a binary expression (thus depending on values in multiple columns) | ||
// the final result may differ if it is pushed down. | ||
|
||
let mut root_count = 0; | ||
|
||
// if this condition is called more than once, its a binary or ternary operation. | ||
let condition = |_| { | ||
if root_count == 0 { | ||
root_count += 1; | ||
false | ||
} else { | ||
true | ||
} | ||
if let Some(ref subset) = options.subset { | ||
// Predicates on the subset can pass. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. since |
||
let mut names_set = PlHashSet::<Arc<str>>::with_capacity(subset.len()); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can we use |
||
for name in subset.iter() { | ||
names_set.insert(Arc::<str>::from(&**name)); | ||
}; | ||
|
||
let condition = |name: Arc<str>| !names_set.contains(&name); | ||
let local_predicates = | ||
transfer_to_local_by_name(expr_arena, &mut acc_predicates, condition); | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -115,8 +115,9 @@ pub(super) fn predicate_is_sort_boundary(node: Node, expr_arena: &Arena<AExpr>) | |
has_aexpr(node, expr_arena, matches) | ||
} | ||
|
||
/// Transfer a predicate from `acc_predicates` that will be pushed down | ||
/// to a local_predicates vec based on a condition. | ||
/// Evaluates a condition on the column name inputs of every predicate, where if | ||
/// the condition evaluates to true on any column name the predicate is | ||
/// transferred to local. | ||
pub(super) fn transfer_to_local_by_name<F>( | ||
expr_arena: &Arena<AExpr>, | ||
acc_predicates: &mut PlHashMap<Arc<str>, Node>, | ||
|
@@ -132,7 +133,7 @@ where | |
for name in root_names { | ||
if condition(name) { | ||
remove_keys.push(key.clone()); | ||
continue; | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. would insert the same key multiple times |
||
break; | ||
} | ||
} | ||
} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From what I understand this wanted to block predicates that had more than 1 column input, but because
mut root_count
is not reset per predicate it would end up blocking all predicates after the first call.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, yeap. That had to reset.