-
Notifications
You must be signed in to change notification settings - Fork 3.9k
ARROW-6089: [Rust] [DataFusion] Implement physical plan for "selection" operator #5320
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
48875e4
to
831e695
Compare
@paddyhoran @sunchao @nevi-me This PR is now ready for review. PTAL when you can. My stretch goal is to implement the aggregate operator as well in time for the 0.15 release if I can finish it this weekend. This would mean that people can start experimenting with parallel queries and then for the next release (1.0?) we can drop the code that directly executes the logical plan and change it to create the physical plan and execute that instead. |
704ec89
to
fbd2278
Compare
@paddyhoran @sunchao @nevi-me This is my final PR that I'm hoping can make it into 0.15. I could break this out into smaller PRs by having separate PRs for the new expression types (literal, cast, binary) if that helps. |
It's probably too late to try and get this reviewed for 0.15.0 so I have started breaking this down into smaller PRs that are easier to review. |
@andygrove we might get it in. I had started reviewing but am struggling to find time now. Breaking it into smaller PR's definitely helps. |
OK thanks @paddyhoran I created the following PRs for the expressions: These PRs have more extensive tests too than this current PR so it was good to pull them out. I'll need to rebase of course once those are merged. |
27c2280
to
b95bcde
Compare
@andygrove can you rebase now please? |
b95bcde
to
6cad327
Compare
@paddyhoran done! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM pending CI
@paddyhoran Thanks so much for reviewing this late flood of PRs. I really appreciate it! |
Great job pushing the project forward. The progress you are making is really exciting! |
This PR implements the physical execution plan for the selection operator (the WHERE clause in a SQL query).
In order to have working tests, I also had to implement some subset of expressions (column reference, literal value, comparison expressions, and CAST). However, the goal of this PR is not to add complete support for all expressions but to implement the Selection operator. I will create separate JIRA/PRs for adding support for other expressions and data types in the physical query plan.