Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-31889][BUILD] Docker release script does not allocate enough memory to reliably publish #28698

Conversation

holdenk
Copy link
Contributor

@holdenk holdenk commented Jun 1, 2020

What changes were proposed in this pull request?

Allow overriding the zinc options in the docker release and set a higher so the publish step can succeed consistently.

Why are the changes needed?

The publish step experiences memory pressure.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Running test locally with fake user to see if publish step (besides svn part) succeeds

@holdenk
Copy link
Contributor Author

holdenk commented Jun 1, 2020

cc @srowen @cloud-fan

@srowen
Copy link
Member

srowen commented Jun 1, 2020

Also seems fine.

@SparkQA
Copy link

SparkQA commented Jun 1, 2020

Test build #123388 has finished for PR 28698 at commit a599699.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@holdenk
Copy link
Contributor Author

holdenk commented Jun 1, 2020

Two failures are spurious (failures in Java unit tests).
Jenkins retest this please.

@holdenk
Copy link
Contributor Author

holdenk commented Jun 1, 2020

Ran locally as "fake@apache.org" seems to work fine.

@holdenk holdenk changed the title [WIP][SPARK-31889][BUILD] Docker release script does not allocate enough memory to reliably publish [SPARK-31889][BUILD] Docker release script does not allocate enough memory to reliably publish Jun 1, 2020
@asfgit asfgit closed this in ab9e5a2 Jun 1, 2020
asfgit pushed a commit that referenced this pull request Jun 1, 2020
…emory to reliably publish

### What changes were proposed in this pull request?
Allow overriding the zinc options in the docker release and set a higher so the publish step can succeed consistently.

### Why are the changes needed?

The publish step experiences memory pressure.

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?
Running test locally with fake user to see if publish step (besides svn part) succeeds

Closes #28698 from holdenk/SPARK-31889-docker-release-script-does-not-allocate-enough-memory-to-reliably-publish.

Authored-by: Holden Karau <hkarau@apple.com>
Signed-off-by: Holden Karau <hkarau@apple.com>
(cherry picked from commit ab9e5a2)
Signed-off-by: Holden Karau <hkarau@apple.com>
asfgit pushed a commit that referenced this pull request Jun 1, 2020
…emory to reliably publish

### What changes were proposed in this pull request?
Allow overriding the zinc options in the docker release and set a higher so the publish step can succeed consistently.

### Why are the changes needed?

The publish step experiences memory pressure.

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?
Running test locally with fake user to see if publish step (besides svn part) succeeds

Closes #28698 from holdenk/SPARK-31889-docker-release-script-does-not-allocate-enough-memory-to-reliably-publish.

Authored-by: Holden Karau <hkarau@apple.com>
Signed-off-by: Holden Karau <hkarau@apple.com>
(cherry picked from commit ab9e5a2)
Signed-off-by: Holden Karau <hkarau@apple.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants