Need a way to stop server side streaming when client disconnected

[pfwang80s]

here is pseudo code to impl a server side streaming:

@Override
public void someMethod(XXRequest request,  StreamObserver<XXEvent> responseObserver) {
	try {
	            while (!Context.current().isCancelled()) {
	                XXEvent event = heavyCompute();
	                if (event != null) {
	                    responseObserver.onNext(event);
	                }
	            }
	} finally {
	       doClean();
	}
}

On client side, if everything is ok, CancellableContext.cancel() or ManagedChannel.shutdownNow() will terminate someMethod call gracefully, but if client crashes && heavyCompute always returns null, someMethod call will run forever...

so is there a way to fix this?

Thanks.

[sanjaypujare]

Couple of ways:

• you need another way to break out of your while loop (either number of iterations or a timer)
• how about using a CancellableContext with a deadline in your while loop? Look at the Javadoc of io.grpc.Context.withDeadline for sample usage.

[pfwang80s]

Couple of ways:

• you need another way to break out of your while loop (either number of iterations or a timer)
• how about using a CancellableContext with a deadline in your while loop? Look at the Javadoc of io.grpc.Context.withDeadline for sample usage.

Thank you for your reply. In my logic, the server returns an infinite event stream that a healthy client consumes event from, so withDeadline may not work, did I misunderstand it?

A workaround is change someMethod to bi-stream, and treat client stream as Ping, but it looks not good...

[sanjaypujare]

If the client has crashed and your event is always null then you go into an infinite loop because of the structure of your while loop. When event is null you can check the state of the responseObserver by casting to ServerCallStreamObserver and calling isCancelled assuming that works when client has crashed. Or you can limit the number of iterations you will perform when event is consecutively null.

[voidzcy]

What do you mean by "crashes"? If the client process closes the socket after it "crashes", the stream will be closed and server RPC context will be cancelled. What you are currently checking in your while loop can handle it.

[pfwang80s]

If the client has crashed and your event is always null then you go into an infinite loop because of the structure of your while loop. When event is null you can check the state of the responseObserver by casting to ServerCallStreamObserver and calling isCancelled assuming that works when client has crashed. Or you can limit the number of iterations you will perform when event is consecutively null.

Thank you. most of heavyCompute() call returns null, no matter if client gone, I just want to quit as soon as possible to save resources.

[pfwang80s]

What do you mean by "crashes"? If the client process closes the socket after it "crashes", the stream will be closed and server RPC context will be cancelled. What you are currently checking in your while loop can handle it.

"crashes" means that client has no chance doing any clean operation, e.g.: network connections broken, or a segmentation exception.

Here is a simple logic with pseudo code:
server.start();
channel := new channel(svr_endpoint);
client := newStub(channel);
iterator := client.someMethod(request);
iterator.nextSomeTimes();
channel.showdown();
Thread.sleep(infinite);

I've waited for someMethod at server-side for 100 seconds, but it didn't terminated as Context.current().isCancelled() returned false.
I can't understand when channel.shutdown(), why Context.current().isCanceled() was returning false continuously for such a long time.

[zhangkun83]

If client or network failed so badly that it has not chance notifying the server side, the TCP connection could stall for very long time until it times out. You can try setting keepAliveTime() in NettyServerBuilder to a smaller number (default 2 hours). Combined with keepAliveTimeout() (default 20s), that would make the server wait at most keepAliveTime + keepAliveTimeout to notice the bad connection.

I can't understand when channel.shutdown(), why Context.current().isCanceled() was returning false continuously for such a long time.

I thought you were complaining the case when client crashes, not when the client calls channel.shutdown() normally.

[pfwang80s]

Thank you for your reply, but I found NettyServerBuilder was introduced at 2016, until now, it has being an experimental feature. Do we plan to turn it normal?

If client or network failed so badly that it has not chance notifying the server side, the TCP connection could stall for very long time until it times out. You can try setting keepAliveTime() in NettyServerBuilder to a smaller number (default 2 hours). Combined with keepAliveTimeout() (default 20s), that would make the server wait at most keepAliveTime + keepAliveTimeout to notice the bad connection.

I can't understand when channel.shutdown(), why Context.current().isCanceled() was returning false continuously for such a long time.

I thought you were complaining the case when client crashes, not when the client calls channel.shutdown() normally.

[voidzcy]

NettyServerBuilder is pretty stable, but it depends on Netty's feature, we reserve the right to change its interface if necessary (e.g., some breaking changes in Netty). In general, we will avoid such a change as much as possible.

[dapengzhang0]

@pengfeiwang-cn If you are developing libraries that other people are depending on, do not use ExperimentalAPI; if you are developing applications, the keepalive API in NettyServerBuilder is pretty safe to use in production.