Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SPARK-983. Support external sorting in sortByKey() #1677

Closed
wants to merge 1 commit into from
Closed

SPARK-983. Support external sorting in sortByKey() #1677

wants to merge 1 commit into from

Conversation

mateiz
Copy link
Contributor

@mateiz mateiz commented Jul 31, 2014

This patch simply uses the ExternalSorter class from sort-based shuffle.

Closes #931 and Closes #1090

@mateiz
SPARK-983. Support external sorting in sortByKey()
96b3fda 
This patch simply uses the ExternalSorter class from sort-based shuffle.
@mateiz mateiz mentioned this pull request Jul 31, 2014
Closed
@SparkQA
Copy link

SparkQA commented Jul 31, 2014

QA tests have started for PR 1677. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17532/consoleFull

@SparkQA
Copy link

SparkQA commented Jul 31, 2014

QA results for PR 1677:
- This patch FAILED unit tests.
- This patch merges cleanly
- This patch adds no public classes

For more information see test ouptut:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17532/consoleFull

@pwendell
Copy link
Contributor

@mateiz could you put "Closes #931 and Closes #1090 in the description?" This will clean up the earlier work on this feature.

@pwendell
Copy link
Contributor

Jenkins, test this please.

@SparkQA
Copy link

SparkQA commented Jul 31, 2014

QA tests have started for PR 1677. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17552/consoleFull

@SparkQA
Copy link

SparkQA commented Jul 31, 2014

QA results for PR 1677:
- This patch PASSES unit tests.
- This patch merges cleanly
- This patch adds no public classes

For more information see test ouptut:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17552/consoleFull

@mateiz
Copy link
Contributor Author

mateiz commented Jul 31, 2014

Added "closes" comment

@mateiz
Copy link
Contributor Author

mateiz commented Aug 1, 2014

@pwendell is this okay to merge?

@pwendell
Copy link
Contributor

pwendell commented Aug 1, 2014

Go for it

@asfgit asfgit closed this in 72e3369 Aug 1, 2014
xiliu82 pushed a commit to xiliu82/spark that referenced this pull request Sep 4, 2014
@mateiz @conviva-zz
SPARK-983. Support external sorting in sortByKey()
4624004 
This patch simply uses the ExternalSorter class from sort-based shuffle.

Closes apache#931 and Closes apache#1090

Author: Matei Zaharia <matei@databricks.com>

Closes apache#1677 from mateiz/spark-983 and squashes the following commits:

96b3fda [Matei Zaharia] SPARK-983. Support external sorting in sortByKey()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
3 participants
@mateiz @SparkQA @pwendell