[SPARK-8289] Specify stack size for consistency with Java tests - resolves test failures #6727
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Increase stack size - defaults vary between Java vendors. This remedies test failures observed with JavaALSSuite with IBM & Oracle Java owing to lower default size in comparison to OpenJDK.
Member
|
Sounds OK; does this need to go in the SBT build too? also I am probably wrong about this but I thought there was another place this was specified in the Maven build. |
Member
|
ok to test |
Contributor
Author
|
Sean, excellent point about the SBT build, we've been using Maven exclusively for testing - if there's also, say, an -Xmx3g (e.g. an argline equivalent as in the pom file for Maven) then indeed such an addition will need to be made too. I'll have a look, suspecting the answer is yes; it would make sense for such additional options to be mirrored |
Contributor
Author
|
You're right, at project/SparkBuild.scala it's clear we'll need to increase the stack size here too - should I create a new pull request or can we add to the code changed here? |
Contributor
|
I'd say you should just add the changes to the sbt build here as well |
|
Test build #34521 has finished for PR 6727 at commit
|
Contributor
|
Can you make a JIRA for this? |
|
Test build #34526 has finished for PR 6727 at commit
|
a-roberts
changed the title
Member
|
LGTM |
asfgit
pushed a commit
that referenced
this pull request
Jun 11, 2015
…olves test failures This change is a simple one and specifies a stack size of 4096k instead of the vendor default for Java tests (the defaults vary between Java vendors). This remedies test failures observed with JavaALSSuite with IBM and Oracle Java owing to a lower default size in comparison to the size with OpenJDK. 4096k is a suitable default where the tests pass with each Java vendor tested. The alternative is to reduce the number of iterations in the test (no observed failures with 5 iterations instead of 15). -Xss works with Oracle's HotSpot VM, IBM's J9 VM and OpenJDK (IcedTea). I have ensured this does not have any negative implications for other tests. Author: Adam Roberts <aroberts@uk.ibm.com> Author: a-roberts <aroberts@uk.ibm.com> Closes #6727 from a-roberts/IncJavaStackSize and squashes the following commits: ab40aea [Adam Roberts] Specify stack size for SBT builds 5032d8d [a-roberts] Update pom.xml (cherry picked from commit 6b68366) Signed-off-by: Sean Owen <sowen@cloudera.com>
nemccarthy
pushed a commit
to nemccarthy/spark
that referenced
this pull request
Jun 19, 2015
…olves test failures This change is a simple one and specifies a stack size of 4096k instead of the vendor default for Java tests (the defaults vary between Java vendors). This remedies test failures observed with JavaALSSuite with IBM and Oracle Java owing to a lower default size in comparison to the size with OpenJDK. 4096k is a suitable default where the tests pass with each Java vendor tested. The alternative is to reduce the number of iterations in the test (no observed failures with 5 iterations instead of 15). -Xss works with Oracle's HotSpot VM, IBM's J9 VM and OpenJDK (IcedTea). I have ensured this does not have any negative implications for other tests. Author: Adam Roberts <aroberts@uk.ibm.com> Author: a-roberts <aroberts@uk.ibm.com> Closes apache#6727 from a-roberts/IncJavaStackSize and squashes the following commits: ab40aea [Adam Roberts] Specify stack size for SBT builds 5032d8d [a-roberts] Update pom.xml
mtbrandy
pushed a commit
to ibmsoe/spark
that referenced
this pull request
Jun 23, 2015
…olves test failures This change is a simple one and specifies a stack size of 4096k instead of the vendor default for Java tests (the defaults vary between Java vendors). This remedies test failures observed with JavaALSSuite with IBM and Oracle Java owing to a lower default size in comparison to the size with OpenJDK. 4096k is a suitable default where the tests pass with each Java vendor tested. The alternative is to reduce the number of iterations in the test (no observed failures with 5 iterations instead of 15). -Xss works with Oracle's HotSpot VM, IBM's J9 VM and OpenJDK (IcedTea). I have ensured this does not have any negative implications for other tests. Author: Adam Roberts <aroberts@uk.ibm.com> Author: a-roberts <aroberts@uk.ibm.com> Closes apache#6727 from a-roberts/IncJavaStackSize and squashes the following commits: ab40aea [Adam Roberts] Specify stack size for SBT builds 5032d8d [a-roberts] Update pom.xml (cherry picked from commit 6b68366) Signed-off-by: Sean Owen <sowen@cloudera.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This change is a simple one and specifies a stack size of 4096k instead of the vendor default for Java tests (the defaults vary between Java vendors). This remedies test failures observed with JavaALSSuite with IBM and Oracle Java owing to a lower default size in comparison to the size with OpenJDK. 4096k is a suitable default where the tests pass with each Java vendor tested. The alternative is to reduce the number of iterations in the test (no observed failures with 5 iterations instead of 15).
-Xss works with Oracle's HotSpot VM, IBM's J9 VM and OpenJDK (IcedTea).
I have ensured this does not have any negative implications for other tests.