New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-9546][SQL] Centralize orderable data type checking. #7880
Conversation
Test build #39473 has finished for PR 7880 at commit
|
Test build #39482 has finished for PR 7880 at commit
|
Test build #39483 has finished for PR 7880 at commit
|
@@ -111,7 +99,7 @@ private[sql] abstract class SparkStrategies extends QueryPlanner[SparkPlan] { | |||
// If the sort merge join option is set, we want to use sort merge join prior to hashjoin | |||
// for now let's support inner join first, then add outer join | |||
case ExtractEquiJoinKeys(Inner, leftKeys, rightKeys, condition, left, right) | |||
if sqlContext.conf.sortMergeJoinEnabled && isValidSort(leftKeys, rightKeys) => | |||
if sqlContext.conf.sortMergeJoinEnabled && RowOrdering.isOrderable(leftKeys) => |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing && RowOrdering.isOrderable(rightKeys)
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
left/right should be the same data types -- otherwise they cannot be joined together
LGTM |
Thanks - merging this in. |
This pull request creates two isOrderable functions in RowOrdering that can be used to check whether a data type or a sequence of expressions can be used in sorting.