New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-8977][Streaming] Defines the RateEstimator interface, and implements the ReceiverRateController #16
Conversation
758b9bc
to
397b8b7
Compare
6cd2b46
to
78219dd
Compare
Refer to this link for build results (access rights to CI server needed): Build Log
Test FAILed. |
|
||
protected def publish(rate: Long): Unit = () | ||
|
||
private def computeAndPublishExecutionContext = ExecutionContext.fromExecutorService( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't this be a val
, or maybe even an implicit val
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That would pose a serializability problem.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@transient
would fix it. Wouldn't the current code create a new thread on each rate computation?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, yes, you're right !
unrelated SQL tests failing: https://ci.typesafe.com/job/ghprb-spark-multi-conf/label=Spark-Ora-JDK7-PV,scala_version=2.10/42/console |
99a40fe
to
8479165
Compare
Refer to this link for build results (access rights to CI server needed): Build Log
Test FAILed. |
Refer to this link for build results (access rights to CI server needed): Build Log
Test FAILed. |
…ements the ReceiverRateController
Refer to this link for build results (access rights to CI server needed): Build Log
Test FAILed. |
Refer to this link for build results (access rights to CI server needed): Build Log
Test FAILed. |
13ada97
to
0c51959
Compare
Depends on #15.