Optimizer that removes redundant DELIM_GET and DELIM_JOIN operators #1296
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…r duplicate eliminated
Mytherin
reviewed
Jan 14, 2021
src/optimizer/deliminator.cpp
Outdated
| for (auto &child : op->children) { | ||
| FindCandidates(&child, candidates); | ||
| } | ||
| if ( // Projection/Aggregate |
There was a problem hiding this comment.
I would prefer to break up this huge if statement into several blocks using early outs, e.g.
// search for a projection or aggregate
if (op->type != LogicalOperatorType::LOGICAL_PROJECTION &&
op->type != LogicalOperatorType::LOGICAL_AGGREGATE_AND_GROUP_BY) {
return;
}
// followed by a join
if (op->children[0]->type != LogicalOperatorType::LOGICAL_COMPARISON_JOIN) {
return;
}
...This keeps the same logic but is more readable than a ton of nested conditions (in my opinion).
There was a problem hiding this comment.
Agreed. I split it up and it is much more readable now
|
Thanks, looks great now! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR implements an optimizer which I have affectionately called the Deliminator, an optimizer that removes redundant operators related to correlated subqueries.
When a query containing a correlated subquery is issued, a dependent join is created. Consider the following query:
DuckDB has implemented a way of generically flattening these queries, following Unnesting Arbitrary Queries by Thomas Neumann and Alfons Kemper, to avoid quadratic complexity. This is done by creating a duplicate-eliminated join, or delim join, and pushing it down the query plan to decorrelate it.
This process introduces delim scans, which are usually joined (or in a cross product) with another part of the query plan. Under specific circumstances, joining with a delim scan does not introduce new information, and this can be removed. If all delim scans belonging to a delim join can be removed, the delim join can be transformed into a comparison join instead.
This is more efficient for two reasons:
For the example query this means that the original query plan
Can be simplified to:
The original projection can also be removed in this specific case, but in other cases the projection may project
+(i, 3), in which case it is still necessary.This will improve the performance of the FTS extension, and some TPC-H queries.
I am happy to receive any feedback on this PR!