Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-29879][CORE]Support unbounded waiting for RPC reply #26504

Closed
wants to merge 1 commit into from

Conversation

wenxuanguan
Copy link
Contributor

What changes were proposed in this pull request?

Support unbounded waiting for RPC reply with the call RpcEndpointRef.askSync[T: ClassTag](message: Any, timeout: RpcTimeout)

Why are the changes needed?

At present, we hard code long duration to represent never timeout as below

timeout = new RpcTimeout(365.days, "barrierTimeout")

with the support of unbounded waiting, we can do as below

timeout = new RpcTimeout(Duration.Inf, "barrierTimeout")

Does this PR introduce any user-facing change?

No

How was this patch tested?

N/A

@wenxuanguan wenxuanguan changed the title [SPARK-29879]Support unbounded waiting for RPC reply [SPARK-29879][CORE]Support unbounded waiting for RPC reply Nov 13, 2019
@AmplabJenkins
Copy link

Can one of the admins verify this patch?

@jiangxb1987
Copy link
Contributor

Do you really have a training job that runs over a year?

@wenxuanguan
Copy link
Contributor Author

@jiangxb1987 Thanks for your reply.

Do you really have a training job that runs over a year?

No, but what I try to do is a way to not set a specific timeout value with RpcEndpointRef.askSync(), in the situation where never timeout until replied or interrupted by other event.
Such as BarrierTaskContext.barrier() set a long timeout value which would not be reached, and a SparkException will be thrown by BarrierCoordinator on timeout.

@jiangxb1987
Copy link
Contributor

sorry I'm not convinced this change is necessary.

@wenxuanguan
Copy link
Contributor Author

@jiangxb1987 ok, and I will close the pr. Thanks for your review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants