gRPC Client Threads Concept

[abhilash93]

Client Thread Consuming & Queue Logic Questions

I have a server stream rpc call

example
rpc telemetrySubscribe(SubscriptionRequest) returns (stream OpenConfigData)
(https://github.com/nileshsimaria/jtimon/blob/master/telemetry/telemetry.proto)

I have grpc Client request created using java and used the StreamObserver onNext

I am trying to understand threading and queuing concept on the client side.

• What happens if message on the server side are queuing up fast and the client is been consuming it slow. After 2 hour the TCP connection closed, as the server message are getting queued (GRPC_ARG_KEEPALIVE_TIME_MS is 7200000 (2 hours))

• I have written my processing logic in onNext, will this cause it to slow down the consuming of messages?

• How is the order of the message maintained in the client, if cachedPool is used?

• Will overriding the executor with to more number of thread pool withExecutor(Executors.newFixedThreadPool(20) help?

• I have also tried using MoreExecutors.newSequentialExecutor(Executors.newSingleThreadExecutor()), will this slow down the client consuming of messages?

• What is the efficient executor in MoreExecutors to process messages sequential and fast?

• Will client host device CPU/Memory constrains restrict the number messages being consumed per min?

[sanjaypujare]

Ideally what is needed in this case is the server should see the backpressure and slow down message sending by using isReady.

But it looks like you have no control on the server side so you are looking to tweak the client. Putting your processing logic in onNext will slow down the consumption of messages but the alternative is for onNext to add to a work-queue that is processed by worker threads. If the processing is really time-consuming this queue is going to grow so you are just transferring the server queue to the client side.

With multiple threads you can potentially take advantage of multiple cores/CPUs on the client host assuming your processing logic (in onNext) does not run into resource contention issues.

[abhilash93]

Thank you for your response @sanjaypujare.

I am trying to look for documentation that can help me understand the threading concepts used in gRPC client.

Any gRPC OnNext will be having the processing logic, any efficient way fo handling it?
adding work-queue will the switch from server queue to client side

What happens if withWaitForReady is set for CallOptions, in client ?

[sanjaypujare]

Some open/closed issues on this repo that discuss client side threading could be of help:
#3057
#2424
#2247

I talked about using a queue on the client side and using worker threads to do the work. But ultimately your client should be able to process/consume messages at the rate the server is producing them to address the rate imbalance.

withWaitForReady deals with channel states (ready vs non ready) and does not help with congestion AFAIK.