Change Null-safe equal operator from cross join to hash join #2156
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Closed
morningman
requested changes
Nov 7, 2019
be/src/exec/hash_table.cpp
Outdated
| @@ -42,12 +42,18 @@ const char* HashTable::_s_llvm_class_name = "class.doris::HashTable"; | |||
|
|
|||
| HashTable::HashTable(const vector<ExprContext*>& build_expr_ctxs, | |||
| const vector<ExprContext*>& probe_expr_ctxs, | |||
| int num_build_tuples, bool stores_nulls, int32_t initial_seed, | |||
| int num_build_tuples, bool null_preserved, | |||
| const std::vector<bool> finds_nulls, | |||
There was a problem hiding this comment.
Suggested change
| const std::vector<bool> finds_nulls, | |
| const std::vector<bool>& finds_nulls, |
be/src/exec/hash_table.h
Outdated
| @@ -98,7 +98,9 @@ class HashTable { | |||
| HashTable( | |||
| const std::vector<ExprContext*>& build_exprs, | |||
| const std::vector<ExprContext*>& probe_exprs, | |||
| int num_build_tuples, bool stores_nulls, int32_t initial_seed, | |||
| int num_build_tuples, bool stores_nulls, | |||
| const std::vector<bool> finds_nulls, | |||
There was a problem hiding this comment.
Suggested change
| const std::vector<bool> finds_nulls, | |
| const std::vector<bool>& finds_nulls, |
be/src/exec/olap_scan_node.cpp
Outdated
| @@ -194,8 +194,10 @@ Status OlapScanNode::open(RuntimeState* state) { | |||
| RETURN_IF_CANCELLED(state); | |||
| RETURN_IF_ERROR(ExecNode::open(state)); | |||
|
|
|||
| LOG(INFO) << "olap scan node conjunct in open " << Expr::debug_string(_conjunct_ctxs); | |||
There was a problem hiding this comment.
I will remove it.
imay
reviewed
Nov 7, 2019
| @@ -53,8 +54,8 @@ | |||
|
|
|||
| private final TableRef innerRef; | |||
| private final JoinOperator joinOp; | |||
| // conjuncts of the form "<lhs> = <rhs>", recorded as Pair(<lhs>, <rhs>) | |||
| private List<Pair<Expr, Expr>> eqJoinConjuncts; | |||
| // predicates of the form "<lhs> = <rhs>", recorded as Pair(<lhs>, <rhs>) | |||
There was a problem hiding this comment.
why you change conjuncts to predicates?
Better to keep it conjuncts, becuase it means and and same with otherJoinConjuncts
be/src/exec/hash_table.h
Outdated
| // the row in hash table is preserved such as RIGHT_OUTER_JOIN | ||
| const bool _null_preserved; | ||
| // true: the null value should be keep in | ||
| // false: the null value should be filter |
There was a problem hiding this comment.
what keep in means? and what filter means?
imay
reviewed
Nov 7, 2019
| @@ -384,13 +384,13 @@ private PlanFragment createHashJoinFragment(HashJoinNode node, PlanFragment righ | |||
| // TODO: create equivalence classes based on equality predicates | |||
|
|
|||
| // first, extract join exprs | |||
| List<Pair<Expr, Expr>> eqJoinConjuncts = node.getEqJoinConjuncts(); | |||
| List<BinaryPredicate> eqJoinPredicates = node.getEqJoinConjuncts(); | |||
ISSUE-2136 This commit change the join method from cross join to hash join when the equal operator is Null-safe '<=>'. It will improve the speed of query which has the Null-safe equal operator. The finds_nulls field is used to save if there is Null-safe operator. The finds_nulls[i] is true means that the i-th equal operator is Null-safe. The equal function in hash table will return true, if both val and loc are NULL when finds_nulls[i] is true.
EmmyMiao87
force-pushed
the
eq_for_null
branch
from
4d477ea
to
b88d542
Compare
morningman
pushed a commit
to morningman/doris
that referenced
this pull request
Nov 8, 2019
…2156) * Change Null-safe equal operator from cross join to hash join ISSUE-2136 This commit change the join method from cross join to hash join when the equal operator is Null-safe '<=>'. It will improve the speed of query which has the Null-safe equal operator. The finds_nulls field is used to save if there is Null-safe operator. The finds_nulls[i] is true means that the i-th equal operator is Null-safe. The equal function in hash table will return true, if both val and loc are NULL when finds_nulls[i] is true.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
ISSUE-2136
This commit change the join method from cross join to hash join when the equal operator is Null-safe '<=>'.
It will improve the speed of query which has the Null-safe equal operator.
The finds_nulls field is used to save if there is Null-safe operator.
The finds_nulls[i] is true means that the i-th equal operator is Null-safe.
The equal function in hash table will return true, if both val and loc are NULL when finds_nulls[i] is true.