New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-31719][SQL] Refactor JoinSelection #28540
Conversation
ok to test |
add to whitelist |
Test build #122670 has finished for PR 28540 at commit
|
Test build #122810 has finished for PR 28540 at commit
|
Test build #122816 has finished for PR 28540 at commit
|
Test build #122809 has finished for PR 28540 at commit
|
retest this please |
Test build #122820 has finished for PR 28540 at commit
|
Test build #122821 has finished for PR 28540 at commit
|
Test build #122843 has finished for PR 28540 at commit
|
Test build #122901 has finished for PR 28540 at commit
|
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
Show resolved
Hide resolved
retest this please |
Test build #123064 has finished for PR 28540 at commit
|
right: LogicalPlan, | ||
joinType: JoinType, | ||
hint: JoinHint, | ||
onlyLookingAtHint: Boolean, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: selectOnlyFromHint
? I couldn't tell what this meant just by the name onlyLookingAtHint
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we want to be super clear, maybe "pickJoinSideByHintOnly"? or simply "hintOnly".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, hintOnly
looks nice to me.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM if the tests passed.
Test build #123185 has finished for PR 28540 at commit
|
thanks, merging to master! |
oops, the latest commit hasn't passed tests yet. I'll monitor the jenkins status. |
Test build #123190 has finished for PR 28540 at commit
|
What changes were proposed in this pull request?
This PR extracts the logic for selecting the planned join type out of the
JoinSelection
rule and moves it toJoinSelectionHelper
in Catalyst.Why are the changes needed?
This change both cleans up the code in
JoinSelection
and allows the logic to be in one place and be used from other rules that need to make decision based on the join type before the planning time.Does this PR introduce any user-facing change?
BuildSide
,BuildLeft
, andBuildRight
are moved fromorg.apache.spark.sql.execution
to Catalyst inorg.apache.spark.sql.catalyst.optimizer
.How was this patch tested?
This is a refactoring, passes existing tests.