Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-10351][SQL] Fixes UTF8String.fromAddress to handle off-heap memory #8523

Closed
wants to merge 3 commits into from

Conversation

feynmanliang
Copy link
Contributor

@rxin
Copy link
Contributor

rxin commented Aug 30, 2015

Actually then it looks like the proper fix should make fromAddress handle base = null. Are there existing code that would break from that?

@SparkQA
Copy link

SparkQA commented Aug 30, 2015

Test build #41791 has finished for PR 8523 at commit a967eb8.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@feynmanliang
Copy link
Contributor Author

AFAIK only callers of UTF8String.fromAddress are UnsafeRow.getUTF8String and UnsafeArrayData.getUTF8String, both of which do null checking prior to calling so we can just remove the null checking within fromAddress to fix this

@SparkQA
Copy link

SparkQA commented Aug 31, 2015

Test build #41814 has finished for PR 8523 at commit 198daa7.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@feynmanliang feynmanliang changed the title [SPARK-10351][SQL] Fixes UnsafeRow.getUTF8String to handle off-heap backed UnsafeRows [SPARK-10351][SQL] Fixes UTF8String.fromAddress to handle off-heap backed UnsafeRows Aug 31, 2015
@feynmanliang feynmanliang changed the title [SPARK-10351][SQL] Fixes UTF8String.fromAddress to handle off-heap backed UnsafeRows [SPARK-10351][SQL] Fixes UTF8String.fromAddress to handle off-heap memory Aug 31, 2015
@rxin
Copy link
Contributor

rxin commented Aug 31, 2015

Thanks - I've merged this.

@asfgit asfgit closed this in 8694c3a Aug 31, 2015
@feynmanliang feynmanliang deleted the SPARK-10351 branch August 31, 2015 16:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants