Split apart partition and join in ShuffleRegionJoin #1175

Closed
fnothaft opened this Issue Sep 19, 2016 · 2 comments

Comments

Projects
None yet
2 participants
@fnothaft
Member

fnothaft commented Sep 19, 2016

This would be useful for some stuff that I am doing: in ShuffleRegionJoin.partitionAndJoin, we partition and sort the input data and then we do a join. If we've already got the data sorted and want to do another join, it's unnecessary to do another sort and repartition first. We should make it optional to do that by splitting apart the two tasks.

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Sep 20, 2016

Member

+1

Member

heuermh commented Sep 20, 2016

+1

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Mar 3, 2017

Member

This is done in #1324. For cleanliness, I am going to close this ticket in favor of #1216, which is the main shuffle join ticket.

Member

fnothaft commented Mar 3, 2017

This is done in #1324. For cleanliness, I am going to close this ticket in favor of #1216, which is the main shuffle join ticket.

@fnothaft fnothaft closed this Mar 3, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment