[FLINK-4362] [rpc] Auto generate rpc gateways via Java proxies #2357

tillrohrmann · 2016-08-11T14:15:09Z

This PR introduces a generic AkkaRpcActor which receives rpc calls as a
RpcInvocation message. The RpcInvocation message is generated by the
AkkaInvocationHandler which gets them from automatically generated Java Proxies.
The Java proxies are generated in the AkkaRpcService class upon connection or
starting of a rpc server.

R @mxm, @StephanEwen

StephanEwen · 2016-08-11T15:01:16Z

Good stuff!

I am not sure we should throw an Exception in the AkkaRpcActor upon receiving an incorrect message. In the past we decided to only log those cases. Seems to easy to bring down the system with a poison message otherwise.

StephanEwen · 2016-08-11T15:02:33Z

flink-runtime/src/main/java/org/apache/flink/runtime/rpc/akka/AkkaRpcActor.java

+	 */
+	private void handleCallAsync(CallAsync callAsync) {
+		if (callAsync.getCallable() == null) {
+			final String result = "Received a " + callAsync.getClass().getName() + " message with an empty " +


This should in practice never occur, because the runAsync method is not on any gateway, but only on the hidden self gateway type, correct?

Yes exactly. This branch should never be reached but I included it to treat the case that local messages get serialized (if Akka is instructed to do so).

Once we can detect whether a communication is local or remote, we can even let the call async and run async messages fail on the invocation side if it's a remote connection.

tillrohrmann · 2016-08-11T15:24:28Z

Yes you're right @StephanEwen. We shouldn't stop the actor upon receiving an unknown message type. I'll change the behaviour.

StephanEwen · 2016-08-11T15:39:22Z

flink-runtime/src/main/java/org/apache/flink/runtime/rpc/RpcEndpoint.java

+	 *
+	 * @return Class of the self gateway type
+	 */
+	public abstract Class<C> getSelfGatewayType();


I think one can find this out via reflection. The generic gateway type parameter is stored in the class signature.

Yes we can do this. I thought about this and then went for the solution which saves some compute cycles. I can change it to the reflection alternative if it's pre-mature optimization.

But we can actually do this in the constructor and then store the type.

StephanEwen · 2016-08-11T17:15:40Z

+1 from my side

I would suggest as a followup to find out the gateway type for reflection. The ReflectionUtil helps with that.

tillrohrmann · 2016-08-12T08:14:29Z

Yes good point with the automatic computation of the gateway type. Will add it and then merge the PR. Thanks for your review @StephanEwen :-)

This PR introduces a generic AkkaRpcActor which receives rpc calls as a RpcInvocation message. The RpcInvocation message is generated by the AkkaInvocationHandler which gets them from automatically generated Java Proxies. Add documentation for proxy based akka rpc service Log unknown message type in AkkaRpcActor but do not fail actor Use ReflectionUtil to extract RpcGateway type from RpcEndpoint This closes apache#2357.

This PR introduces a generic AkkaRpcActor which receives rpc calls as a RpcInvocation message. The RpcInvocation message is generated by the AkkaInvocationHandler which gets them from automatically generated Java Proxies. Add documentation for proxy based akka rpc service Log unknown message type in AkkaRpcActor but do not fail actor Use ReflectionUtil to extract RpcGateway type from RpcEndpoint This closes #2357.

This PR introduces a generic AkkaRpcActor which receives rpc calls as a RpcInvocation message. The RpcInvocation message is generated by the AkkaInvocationHandler which gets them from automatically generated Java Proxies. Add documentation for proxy based akka rpc service Log unknown message type in AkkaRpcActor but do not fail actor Use ReflectionUtil to extract RpcGateway type from RpcEndpoint This closes apache#2357.

This PR introduces a generic AkkaRpcActor which receives rpc calls as a RpcInvocation message. The RpcInvocation message is generated by the AkkaInvocationHandler which gets them from automatically generated Java Proxies. Add documentation for proxy based akka rpc service Log unknown message type in AkkaRpcActor but do not fail actor Use ReflectionUtil to extract RpcGateway type from RpcEndpoint This closes #2357.

This PR introduces a generic AkkaRpcActor which receives rpc calls as a RpcInvocation message. The RpcInvocation message is generated by the AkkaInvocationHandler which gets them from automatically generated Java Proxies. Add documentation for proxy based akka rpc service Log unknown message type in AkkaRpcActor but do not fail actor Use ReflectionUtil to extract RpcGateway type from RpcEndpoint This closes apache#2357.

This PR introduces a generic AkkaRpcActor which receives rpc calls as a RpcInvocation message. The RpcInvocation message is generated by the AkkaInvocationHandler which gets them from automatically generated Java Proxies. Add documentation for proxy based akka rpc service Log unknown message type in AkkaRpcActor but do not fail actor Use ReflectionUtil to extract RpcGateway type from RpcEndpoint This closes #2357.

This PR introduces a generic AkkaRpcActor which receives rpc calls as a RpcInvocation message. The RpcInvocation message is generated by the AkkaInvocationHandler which gets them from automatically generated Java Proxies. Add documentation for proxy based akka rpc service Log unknown message type in AkkaRpcActor but do not fail actor Use ReflectionUtil to extract RpcGateway type from RpcEndpoint This closes apache#2357.

This PR introduces a generic AkkaRpcActor which receives rpc calls as a RpcInvocation message. The RpcInvocation message is generated by the AkkaInvocationHandler which gets them from automatically generated Java Proxies. Add documentation for proxy based akka rpc service Log unknown message type in AkkaRpcActor but do not fail actor Use ReflectionUtil to extract RpcGateway type from RpcEndpoint This closes #2357.

This PR introduces a generic AkkaRpcActor which receives rpc calls as a RpcInvocation message. The RpcInvocation message is generated by the AkkaInvocationHandler which gets them from automatically generated Java Proxies. Add documentation for proxy based akka rpc service Log unknown message type in AkkaRpcActor but do not fail actor Use ReflectionUtil to extract RpcGateway type from RpcEndpoint This closes apache#2357.

StephanEwen reviewed Aug 11, 2016
View reviewed changes

tillrohrmann mentioned this pull request Aug 11, 2016

[FLINK-4382] Buffer rpc calls until the RpcEndpoint has been started #2358

Closed

StephanEwen mentioned this pull request Aug 11, 2016

[FLINK-4384] [rpc] Add "scheduleRunAsync()" to the RpcEndpoint #2360

Closed

tillrohrmann force-pushed the proxyRpc branch from 32dbb07 to a7c3802 Compare August 12, 2016 09:01

tillrohrmann force-pushed the proxyRpc branch from a7c3802 to 4cde591 Compare August 12, 2016 09:04

asfgit merged commit 4cde591 into apache:flip-6 Aug 12, 2016

tillrohrmann deleted the proxyRpc branch August 19, 2016 12:22

rmetzger added the component=Runtime/Coordination label Mar 14, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[FLINK-4362] [rpc] Auto generate rpc gateways via Java proxies #2357

[FLINK-4362] [rpc] Auto generate rpc gateways via Java proxies #2357

Uh oh!

tillrohrmann commented Aug 11, 2016

Uh oh!

StephanEwen commented Aug 11, 2016

Uh oh!

StephanEwen Aug 11, 2016

Uh oh!

tillrohrmann Aug 11, 2016

Uh oh!

tillrohrmann Aug 11, 2016

Uh oh!

tillrohrmann commented Aug 11, 2016

Uh oh!

StephanEwen Aug 11, 2016 •

edited

Loading

Uh oh!

tillrohrmann Aug 12, 2016

Uh oh!

tillrohrmann Aug 12, 2016

Uh oh!

StephanEwen commented Aug 11, 2016

Uh oh!

tillrohrmann commented Aug 12, 2016

Uh oh!

Uh oh!

[FLINK-4362] [rpc] Auto generate rpc gateways via Java proxies #2357

[FLINK-4362] [rpc] Auto generate rpc gateways via Java proxies #2357

Uh oh!

Conversation

tillrohrmann commented Aug 11, 2016

Uh oh!

StephanEwen commented Aug 11, 2016

Uh oh!

StephanEwen Aug 11, 2016

Choose a reason for hiding this comment

Uh oh!

tillrohrmann Aug 11, 2016

Choose a reason for hiding this comment

Uh oh!

tillrohrmann Aug 11, 2016

Choose a reason for hiding this comment

Uh oh!

tillrohrmann commented Aug 11, 2016

Uh oh!

StephanEwen Aug 11, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tillrohrmann Aug 12, 2016

Choose a reason for hiding this comment

Uh oh!

tillrohrmann Aug 12, 2016

Choose a reason for hiding this comment

Uh oh!

StephanEwen commented Aug 11, 2016

Uh oh!

tillrohrmann commented Aug 12, 2016

Uh oh!

Uh oh!

StephanEwen Aug 11, 2016 •

edited

Loading