Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-10379] preserve first page in UnsafeShuffleExternalSorter #8543

Closed
wants to merge 2 commits into from
Closed

[SPARK-10379] preserve first page in UnsafeShuffleExternalSorter #8543

wants to merge 2 commits into from

Conversation

davies
Copy link
Contributor

@davies davies commented Aug 31, 2015

No description provided.

@SparkQA
Copy link

SparkQA commented Sep 1, 2015

Test build #41845 has finished for PR 8543 at commit 56e746c.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • class DCT(JavaTransformer, HasInputCol, HasOutputCol):
    • class SQLTransformer(JavaTransformer):

@andrewor14
Copy link
Contributor

@davies looks like the test needs to be modified.

@SparkQA
Copy link

SparkQA commented Sep 1, 2015

Test build #41891 has finished for PR 8543 at commit 666373b.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@andrewor14
Copy link
Contributor

Merged into master and 1.5, thanks @davies

asfgit pushed a commit that referenced this pull request Sep 3, 2015
[SPARK-10379] preserve first page in UnsafeShuffleExternalSorter
b846a9d 
Author: Davies Liu <davies@databricks.com>

Closes #8543 from davies/preserve_page.

(cherry picked from commit 62b4690)
Signed-off-by: Andrew Or <andrew@databricks.com>
@asfgit asfgit closed this in 62b4690 Sep 3, 2015
@davies davies mentioned this pull request Oct 30, 2015
Closed
asfgit pushed a commit that referenced this pull request Oct 30, 2015
@davies
[SPARK-11423] remove MapPartitionsWithPreparationRDD
45029bf 
Since we do not need to preserve a page before calling compute(), MapPartitionsWithPreparationRDD is not needed anymore.

This PR basically revert #8543, #8511, #8038, #8011

Author: Davies Liu <davies@databricks.com>

Closes #9381 from davies/remove_prepare2.
ashangit pushed a commit to ashangit/spark that referenced this pull request Oct 19, 2016
[SPARK-10379] preserve first page in UnsafeShuffleExternalSorter
aa75b4c 
Author: Davies Liu <davies@databricks.com>

Closes apache#8543 from davies/preserve_page.

(cherry picked from commit 62b4690)
Signed-off-by: Andrew Or <andrew@databricks.com>
(cherry picked from commit b846a9d)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
3 participants
@davies @SparkQA @andrewor14