Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-12089] [SQL] Fix memory corrupt due to freeing a page being referenced #10142

Closed
wants to merge 1 commit into from
Closed

[SPARK-12089] [SQL] Fix memory corrupt due to freeing a page being referenced #10142

wants to merge 1 commit into from

Conversation

nongli
Copy link
Contributor

@nongli nongli commented Dec 4, 2015

When the spillable sort iterator was spilled, it was mistakenly keeping
the last page in memory rather than the current page. This causes the
current record to get corrupted.

[SPARK-12089] [SQL] Fix memory corrupt due to freeing a page being re…
7ea40a7 
…ferenced.

When the spillable sort iterator was spilled, it was mistakenly keeping
the last page in memory rather than the current page. This causes the
current record to get corrupted.
@davies
Copy link
Contributor

davies commented Dec 4, 2015

LGTM

@SparkQA
Copy link

SparkQA commented Dec 4, 2015

Test build #47189 has finished for PR 10142 at commit 7ea40a7.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):\n * class SparkPlanInfo(\n * class SQLMetricInfo(\n * case class SparkListenerSQLExecutionStart(\n * case class SparkListenerSQLExecutionEnd(executionId: Long, time: Long)\n

@yhuai
Copy link
Contributor

yhuai commented Dec 4, 2015

test this please

@SparkQA
Copy link

SparkQA commented Dec 4, 2015

Test build #47200 has finished for PR 10142 at commit 7ea40a7.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@davies
Copy link
Contributor

davies commented Dec 4, 2015

Merged into master and 1.6 branch, thanks!

@asfgit asfgit closed this in 95296d9 Dec 4, 2015
asfgit pushed a commit that referenced this pull request Dec 4, 2015
@davies
[SPARK-12089] [SQL] Fix memory corrupt due to freeing a page being re…
3fd757c 
…ferenced

When the spillable sort iterator was spilled, it was mistakenly keeping
the last page in memory rather than the current page. This causes the
current record to get corrupted.

Author: Nong <nong@cloudera.com>

Closes #10142 from nongli/spark-12089.

(cherry picked from commit 95296d9)
Signed-off-by: Davies Liu <davies.liu@gmail.com>
@nongli nongli deleted the spark-12089 branch December 10, 2015 05:29
@davies davies mentioned this pull request Aug 18, 2016
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
4 participants
@nongli @davies @SparkQA @yhuai