-
Notifications
You must be signed in to change notification settings - Fork 309
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added the port erasure to SparkFunSuite's cleanup. #104
Added the port erasure to SparkFunSuite's cleanup. #104
Conversation
All automated tests passed. |
Hold off on merging this for now ... Carl and I are still tracking something down. |
@@ -27,3 +27,6 @@ Trunk (not yet released) | |||
|
|||
* Fixed issues where VCF header was not being written correctly. This prevented variant calls from being | |||
written after conversion. This was fixed in PR#85. | |||
* ISSUE 103: Added a call to clearProperty('spark.driver.port') in the cleanup from a sparkTest, so that |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: there should be a space between entries.
Yeah, we're still looking-- this isn't quite fixing all our problems yet. We're going to have to run scalatest by hand, since we can't get it to stop truncating our stack traces through maven :-/ On Thu, Feb 13, 2014 at 3:58 PM, Frank Austin Nothaft
|
All automated tests passed. |
We've been getting intermittent errors, with respect to Spark being unable to bind to a port, in the context of repeated unit tests. This apparently is a known problem, see the thread here: http://blog.quantifind.com/posts/spark-unit-test/ and the follow-up from Matei here: https://groups.google.com/forum/#!topic/spark-users/MeVzgoJXm8I The upshot is that we need to clearProperty('spark.driver.port') when we shut down our sparkContext after a sparkTest.
Okay, once the Jenkins build succeeds, this request should be ready (again) for review. |
All automated tests passed. |
Added the port erasure to SparkFunSuite's cleanup.
Thanks Frank! |
Fixes ISSUE #103
We've been getting intermittent errors, with respect to Spark being unable to bind to a port,
in the context of repeated unit tests. This apparently is a known problem, see the thread here:
http://blog.quantifind.com/posts/spark-unit-test/
and the follow-up from Matei here:
https://groups.google.com/forum/#!topic/spark-users/MeVzgoJXm8I
The upshot is that we need to clearProperty('spark.driver.port') when we shut down our sparkContext
after a sparkTest.