Control MAX_CONCURRENT_STREAMS server-side and account for it on client-side

[utrack]

Right now it seems that we're hitting MAX_CONCURRENT_STREAMS limits between client and server.
Is there any way to tweak limits server-side and get the client to create an additional H2 connection if current streams' count is close to the limit?

[dfawley]

Is there any way to tweak limits server-side

You can use this to adjust the limits: https://godoc.org/google.golang.org/grpc#MaxConcurrentStreams

and get the client to create an additional H2 connection if current streams' count is close to the limit?

The client application can't observe the limit, currently. You would either have to communicate it another way or hard-code it. Creating extra connections can be done by having a custom balancer that creates more SubConns when load is heavy. The balancer could measure in-flight RPCs (increment on Pick, decrement on done) and adjust connections accordingly.

There are discussions going on about detecting this situation and automatically creating new connections, but this seems like a pretty significant feature so I wouldn't expect anything for some time.

[adanteng]

@dfawley

Creating extra connections can be done by having a custom balancer that creates more SubConns when load is heavy.

How heavy, do you have got any benchmark. H2 still need multi connections to improve performance?

There are discussions going on about detecting this situation and automatically creating new connections, but this seems like a pretty significant feature so I wouldn't expect anything for some time.

Got any achievement or plan?

[dfawley]

How heavy, do you have got any benchmark. H2 still need multi connections to improve performance?

We don't have any benchmarks for this. Multiple connections might be needed in this case to overcome the max concurrent streams limit (e.g. if it's being set by an L7 proxy in the middle) -- gRPC can configure this limit for direct connections, and so this kind of thing shouldn't be necessary. There are other reasons a single connection could be a bottleneck, but often one connection should be able to maximize throughput.

Got any achievement or plan?

No. There are no current plans or a schedule to implement anything, just some high level discussions.

[dfawley]

For future visitors of this issue:

From #2412 (comment) above:

Creating extra connections can be done by having a custom balancer that creates more SubConns when load is heavy. The balancer could measure in-flight RPCs (increment on Pick, decrement on done) and adjust connections accordingly.

There is an implementation that does this, and can be used as a reference:

https://github.com/GoogleCloudPlatform/grpc-gcp-go/tree/master/grpcgcp

Also: there are still no plans to implement this feature in gRPC itself, and it's unlikely anything will happen without strong enough need behind it. If you're interested in having this as an out-of-the-box feature, please file an issue in the grpc/grpc repo as a cross-language feature request and cc me (I will update this comment with a reference).

EDIT: grpc/grpc#21386 requests this feature.