Allow specifying maximum number of connections on Server

[ejona86]

This will probably be implemented as part of a larger piece of work to control resources on server-side.

http://stackoverflow.com/questions/37338038/how-to-configure-maximum-number-of-simultaneous-connections-in-grpc

Note that slightly different metrics (like maximum number of streams across the server) may be better; I'm not trying to dictate the exact metric, just the need for this sort of control.

[asif31iqbal]

Thanks for creating this issue.

[rmichela]

What's the status of gRPC resource constraints?

[abaldeva]

+1 for this (C++).

[hehe717]

any updates for this issues?

[ejona86]

No updates. Such a feature should be cross-language, and I confirmed that with the other languages today. I guess it would be helpful to know the more precise use cases (I know use cases for this, but I want to know your reason for asking) and expected behavior when the limit is exceeded. We would also need to decide how "big" of a feature this becomes, since I do think there are various levels of completeness and their associated complexity that the solutions could have.

[Daniel-B-Smith]

I can comment about why we would like it at Datadog for Golang. We're actively working on throttlers to prevent OOMs when a server becomes overloaded. We can go a long way via throttling inside our application, but there is still the risk that memory usage at the OS/connection layer will push us over the limit. Ideally in my opinion, when the server hits too many connections, failed connection attempts will return RESOURCE_EXHAUSTED to the client.

[ejona86]

Ideally in my opinion, when the server hits too many connections, failed connection attempts will return RESOURCE_EXHAUSTED to the client.

This is not feasible. Failed connections cause UNAVAILABLE on the client. To cause RESOURCE_EXHAUSTED we'd have to accept the connection and respond with RESOURCE_EXHAUSTED for each inbound RPC.

[Daniel-B-Smith]

Yeah, that makes sense. Are there separate timeouts for connection establishment and the full request? I would rather establish the connection and fail as early as possible than sit around and timeout.

[ejona86]

@Daniel-B-Smith wrote:

Are there separate timeouts for connection establishment and the full request?

Yes. Connection timeouts and RPC timeouts are separate. An RPC will wait if any connection attempts are in progress, but if the connections are in a known-bad state then the RPC can fail immediately. You can get a rough view with https://github.com/grpc/grpc/blob/master/doc/connectivity-semantics-and-api.md . Roughly speaking (load balancing makes things complicated), RPCs will fail if the Channel is in TRANSIENT_FAILURE.

https://github.com/grpc/grpc/blob/master/doc/connection-backoff.md is for connection establishment. It is supposed to start with a 20 second timeout, and in extreme cases increase as the backoff increases over 20 seconds. But I know Java isn't implementing that; it's more of hard-coded timeouts.

[joseret]

Any news on this? or workaround?

[khsahaji]

Facing the same issue with GoLang. Context is getting cancelled due to max number of parallel connections being hit at gRpc stream server. I need to get that error in client side and handle it but the client right now uses "SendMsg" function and finishes sending message before server is hitting the snag. All this leads to lost messages. How to handle this now ? should the client wait for the number of active connection to go down ? How does it get the number of active parallel connection to gRpc stream server ?