-
Notifications
You must be signed in to change notification settings - Fork 1.8k
Closed
Labels
bugSomething isn't workingSomething isn't working
Description
Describe the bug
A clear and concise description of what the bug is.
The filter pushdown optimizer will not push down predicates through extension plans even if the extension plan allows it.
To Reproduce
Steps to reproduce the behavior:
#[derive(Debug)]
struct NoopPlan {
input: Vec<LogicalPlan>,
schema: DFSchemaRef,
}
impl UserDefinedLogicalNode for NoopPlan {
fn as_any(&self) -> &dyn Any {
self
}
fn inputs(&self) -> Vec<&LogicalPlan> {
self.input.iter().collect()
}
fn schema(&self) -> &DFSchemaRef {
&self.schema
}
fn expressions(&self) -> Vec<Expr> {
self.input
.iter()
.flat_map(|child| child.expressions())
.collect()
}
fn prevent_predicate_push_down_columns(&self) -> HashSet<String> {
HashSet::from_iter(vec!["c".to_string()])
}
fn fmt_for_explain(&self, f: &mut Formatter) -> std::fmt::Result {
write!(f, "NoopPlan")
}
fn from_template(
&self,
_exprs: &[Expr],
inputs: &[LogicalPlan],
) -> Arc<dyn UserDefinedLogicalNode> {
Arc::new(Self {
input: inputs.iter().cloned().collect(),
schema: self.schema.clone(),
})
}
}
#[test]
fn user_defined_plan() -> Result<()> {
let table_scan = test_table_scan()?;
let custom_plan = LogicalPlan::Extension(Extension {
node: Arc::new(NoopPlan {
input: vec![table_scan.clone()],
schema: table_scan.schema().clone(),
}),
});
let plan = LogicalPlanBuilder::from(custom_plan)
.filter(col("a").eq(lit(1i64)))?
.build()?;
// Push filter below NoopPlan
let expected = "\
NoopPlan\
\n Filter: test.a = Int64(1)\
\n TableScan: test";
assert_optimized_plan_eq(&plan, expected)?;
}
Expected behavior
A clear and concise description of what you expected to happen.
If a predicate does not contain any columns in UserDefinedLogicalNode::prevent_predicate_push_down_columns it should get pushed down
Additional context
Add any other context about the problem here.
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working