Skip to content

Commit

Permalink
[SPARK-35604][SQL] Fix condition check for FULL OUTER sort merge join
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?

The condition check for FULL OUTER sort merge join (https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala#L1368 ) has unnecessary trip when `leftIndex == leftMatches.size` or `rightIndex == rightMatches.size`. Though this does not affect correctness (`scanNextInBuffered()` returns false anyway). But we can avoid it in the first place.

### Why are the changes needed?

Better readability for developers and avoid unnecessary execution.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Existing unit tests, such as `OuterJoinSuite.scala`.

Closes #32736 from c21/join-bug.

Authored-by: Cheng Su <chengsu@fb.com>
Signed-off-by: Gengliang Wang <ltnwgl@gmail.com>
  • Loading branch information
c21 authored and gengliangwang committed Jun 2, 2021
1 parent 48252ba commit 54e9999
Showing 1 changed file with 1 addition and 1 deletion.
Expand Up @@ -1365,7 +1365,7 @@ private class SortMergeFullOuterJoinScanner(

def advanceNext(): Boolean = {
// If we already buffered some matching rows, use them directly
if (leftIndex <= leftMatches.size || rightIndex <= rightMatches.size) {
if (leftIndex < leftMatches.size || rightIndex < rightMatches.size) {
if (scanNextInBuffered()) {
return true
}
Expand Down

0 comments on commit 54e9999

Please sign in to comment.