Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add LogicalPlan::recompute_schema for handling rewrite passes #10410

Closed
wants to merge 5 commits into from

Conversation

alamb
Copy link
Contributor

@alamb alamb commented May 7, 2024

Which issue does this PR close?

Part of #9637

Rationale for this change

LogicalPlan::recompute_schema is needed for several subtasks of #9637 (for example #10356 and #10405)

I felt making its own PR with tests might make those PRs easier to review

What changes are included in this PR?

  1. Add LogicalPlan::recompute_schema
  2. Add tests for LogicalPlan::recompute_schema

Are these changes tested?

Yes, new tests (though full coverage will come in #10356 and #10405)

Are there any user-facing changes?

/// # Notes
///
/// * Does not recursively recompute schema for input (child) plans.
pub fn recompute_schema(self) -> Result<Self> {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All this logic is the same as new_with_expr, even some questionable code like for Filter and Union

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a question which may be stupid, what's the difference between this and getting the inputs and exprs of the LogicalPlan and pass them as params to new_with_expr? 🤔

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The key difference is that to use new_with_expr it requires providing a copy (Vec<Expr> and Vec<LogicalPlan>) where as this function can be used after modifying the Exprs (or children) via methods such as map_children and map_expressions

Avoiding those copies is a key part of improve planning performance in PRs like #10356 (the change is basically to stop copying)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pushed bd7b62f to try and clarify

@@ -3138,4 +3350,86 @@ digraph {
let actual = format!("{}", plan.display_indent());
assert_eq!(expected.to_string(), actual)
}

#[test]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here are some basic tests that show what it does and how to use it.

@alamb
Copy link
Contributor Author

alamb commented May 10, 2024

Included as part of #10410, so closing this PR

@alamb
Copy link
Contributor Author

alamb commented May 10, 2024

PR #10443 to add comments based on @yyy1000 's feedback on this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
logical-expr Logical plan and expressions
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants