Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-13663] Upgrade Snappy Java to 1.1.2.1 #11524

Closed
wants to merge 3 commits into from
Closed

[SPARK-13663] Upgrade Snappy Java to 1.1.2.1 #11524

wants to merge 3 commits into from

Conversation

yy2016
Copy link

@yy2016 yy2016 commented Mar 4, 2016

What changes were proposed in this pull request?

The JVM memory leaky problem reported in xerial/snappy-java#131 has been resolved in 1.1.2.1 release.
This PR upgrades snappy-java to 1.1.2.1 release.

How was this patch tested?

Through unit tests.

@srowen
Copy link
Member

srowen commented Mar 4, 2016

Given it's a sub-maintenance release targeting this memory leak problem only, let's back-port for master/1.6. We already worked around it and can leave that change in place, but still helpful.

@srowen
Copy link
Member

srowen commented Mar 4, 2016

Jenkins test this please

@SparkQA
Copy link

SparkQA commented Mar 4, 2016

Test build #52473 has finished for PR 11524 at commit 7e51f1c.

  • This patch fails build dependency tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@srowen
Copy link
Member

srowen commented Mar 4, 2016

Oops, you need to run the script to update the dependencies file to reflect this change.

@JoshRosen
Copy link
Contributor

By the way, one important reason to leave the workaround in place is the fact that we've seen snappy-java get downgraded by user code when the user classpath takes precedence. For this reason, I imagine that in the long run we'll eventually want to shade this dep so that it can't be pulled lower.

@JoshRosen
Copy link
Contributor

Ping. Can you run ./dev/test-dependencies.sh --replace-manifest and commit the result?

@srowen srowen mentioned this pull request Mar 10, 2016
Closed
@srowen
Copy link
Member

srowen commented Mar 10, 2016

@yy2016 I went ahead and made the change at #11631 You should close this PR. (You still get credit.)

@yy2016
Copy link
Author

yy2016 commented Mar 10, 2016

Sorry, I wasn't checking github notifications.

@yy2016 yy2016 closed this Mar 10, 2016
asfgit pushed a commit that referenced this pull request Mar 10, 2016
@srowen
[SPARK-13663][CORE] Upgrade Snappy Java to 1.1.2.1
927e22e 
## What changes were proposed in this pull request?

Update snappy to 1.1.2.1 to pull in a single fix -- the OOM fix we already worked around.
Supersedes #11524

## How was this patch tested?

Jenkins tests.

Author: Sean Owen <sowen@cloudera.com>

Closes #11631 from srowen/SPARK-13663.
asfgit pushed a commit that referenced this pull request Mar 10, 2016
@srowen
[SPARK-13663][CORE] Upgrade Snappy Java to 1.1.2.1
07ace27 
Update snappy to 1.1.2.1 to pull in a single fix -- the OOM fix we already worked around.
Supersedes #11524

Jenkins tests.

Author: Sean Owen <sowen@cloudera.com>

Closes #11631 from srowen/SPARK-13663.

(cherry picked from commit 927e22e)
Signed-off-by: Sean Owen <sowen@cloudera.com>
roygao94 pushed a commit to roygao94/spark that referenced this pull request Mar 22, 2016
@srowen @roygao94
[SPARK-13663][CORE] Upgrade Snappy Java to 1.1.2.1
5e04a40 
## What changes were proposed in this pull request?

Update snappy to 1.1.2.1 to pull in a single fix -- the OOM fix we already worked around.
Supersedes apache#11524

## How was this patch tested?

Jenkins tests.

Author: Sean Owen <sowen@cloudera.com>

Closes apache#11631 from srowen/SPARK-13663.
@viirya viirya mentioned this pull request Aug 16, 2017
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
5 participants
@yy2016 @srowen @SparkQA @JoshRosen @tedyu