-
Notifications
You must be signed in to change notification settings - Fork 1.8k
Open
Description
4. Update pruning to handle CastColumnExpr
-
Goal: Preserve statistics-based pruning when predicates include struct-aware casts.
-
Scope & Location:
datafusion/physical-optimizer/pruningand its tests. -
Key Tasks:
- Modify
rewrite_expr_to_prunable(and related helpers) to unwrap or appropriately interpretCastColumnExpr. - Ensure predicate evaluation during pruning handles missing fields/nullable differences correctly.
- Add regression tests covering struct predicates before/after pruning to confirm behavior is unchanged or improved.
- Modify
-
Dependencies: Specs Add a dedicated physical expression for struct-aware casting that mirrors the existing
cast_columnhelper semantics. #17760, Teach physical-expression utilities aboutCastColumnExpr#17761, Ensure plans containingCastColumnExprround-trip through protobuf for distributed execution. #17762 should be merged; pruning relies on expression and serialization support already in place. -
Out of Scope: Schema adapter rewrites or broader optimizer refactors.
Metadata
Metadata
Assignees
Labels
No labels