Best practices for reusing connections, concurrency

[pboyer]

Thank you for this great library! I wanted to inquire about an issue I'm having.

I have a fixed number of machines I wish to make potentially concurrent RPCs to.

I would like maintain a pool of connections (*grpc.ClientConn) to these machines.

Otherwise, for every RPC call I need to make a new connection, which has overhead (as I understand it) and also can exhaust the number of file descriptors when a large number of concurrent RPC's are taking place.

1. What are the restrictions on using clients and connections concurrently?
2. Can I simply maintain a single connection and attach multiple client stubs to it dynamically (more flexible), or can I only have a single stub of a given type for a connection?
3. My expectation is that I need to maintain my own pool with occasional heartbeats to maintain the health of the connection. Is that correct?

[mwitkow]

I'm not the gRPC maintainer, but have used gRPC extensively.

1. AFAIK you can use the connections concurrently between clients. The only problem may be the limitation of number of concurrent HTTP2 streams you can do, but these count in thousands.
2. Yes.
3. Yes and no.
   • gRPC go has a Picker interface which deals with choosing a channel (tcp connection) to send it down to. Ideally you'd do all your pooling here. Don't know what the state of the Picker implementation is. @iamqizhao ?
   • gRPC 1.1 will have a way of healthchecking a backend, making Picker implementations better. See grpc/grpc-java#1648 (comment)

[mwitkow]

I'm not the gRPC maintainer, but have used gRPC extensively.

1. AFAIK you can use the connections concurrently between clients. The only problem may be the limitation of number of concurrent HTTP2 streams you can do, but these count in thousands.
2. Yes.
3. Yes and no.
   • gRPC go has a Picker interface which deals with choosing a channel (tcp connection) to send it down to. Ideally you'd do all your pooling here. Don't know what the state of the Picker implementation is. @iamqizhao ?
   • gRPC 1.1 will have a way of healthchecking a backend, making Picker implementations better. See grpc/grpc-java#1648 (comment)

[dhowden]

1. AFAIK you can use the connections concurrently between clients. The only problem may be the limitation of number of concurrent HTTP2 streams you can do, but these count in thousands.

I am interested to know the official line on this. I made a similar assumption as it appears to work for small messages, however I have found that larger messages are corrupted when multiple goroutines (each with their own client) use the same underlying *grpc.ClientConn.

[dhowden]

1. AFAIK you can use the connections concurrently between clients. The only problem may be the limitation of number of concurrent HTTP2 streams you can do, but these count in thousands.

I am interested to know the official line on this. I made a similar assumption as it appears to work for small messages, however I have found that larger messages are corrupted when multiple goroutines (each with their own client) use the same underlying *grpc.ClientConn.

[iamqizhao]

You do not need to make a new connection for every RPC. A connection can be multiplexed by multiple clients or RPCs.

We have health check service (https://github.com/grpc/grpc/blob/master/doc/health-checking.md) so that a client can detect the health of the services. We are also designing the connection level story (L7 counterpart of TCP keepalive).

[iamqizhao]

You do not need to make a new connection for every RPC. A connection can be multiplexed by multiple clients or RPCs.

We have health check service (https://github.com/grpc/grpc/blob/master/doc/health-checking.md) so that a client can detect the health of the services. We are also designing the connection level story (L7 counterpart of TCP keepalive).

[iamqizhao]

grpc.io email list would be the better place for these questions. close this now.

[iamqizhao]

grpc.io email list would be the better place for these questions. close this now.

[timbunce]

You do not need to make a new connection for every RPC. A connection can be multiplexed by multiple clients or RPCs.

I'm unclear exactly what's meant by "multiplexed" here. Using a pool to avoid concurrent use?

Where is this covered in the documentation? Perhaps I'm missing it.

I've not been able to find docs that address sharing of connections, or clients, between goroutines.

grpc.io email list would be the better place for these questions.

(In my projects I've found it helpful to view questions in issues, and on the mailing list, as implicit bug reports on the documentation. That's reinforced by trying to answer questions with links to useful docs. If I can't then I'd create a PR for a doc change that would answer the question and reply with a link to that.)

[timbunce]

You do not need to make a new connection for every RPC. A connection can be multiplexed by multiple clients or RPCs.

I'm unclear exactly what's meant by "multiplexed" here. Using a pool to avoid concurrent use?

Where is this covered in the documentation? Perhaps I'm missing it.

I've not been able to find docs that address sharing of connections, or clients, between goroutines.

grpc.io email list would be the better place for these questions.

(In my projects I've found it helpful to view questions in issues, and on the mailing list, as implicit bug reports on the documentation. That's reinforced by trying to answer questions with links to useful docs. If I can't then I'd create a PR for a doc change that would answer the question and reply with a link to that.)

[timbunce]

grpc-go-pool is a gRPC pool implementation. As a novice I look at it and think that's maybe what I want. But I also have a nagging doubt that it's not needed at all.

[timbunce]

grpc-go-pool is a gRPC pool implementation. As a novice I look at it and think that's maybe what I want. But I also have a nagging doubt that it's not needed at all.

[dfawley]

ClientConns can safely be accessed concurrently, and RPCs will be sent in parallel.

The only reason I can think of to use a pool of grpc clients to the same backend(s) is if you're running into stream limits or per-connection throughput limits, possibly imposed by a proxy outside your control.

[dfawley]

ClientConns can safely be accessed concurrently, and RPCs will be sent in parallel.

The only reason I can think of to use a pool of grpc clients to the same backend(s) is if you're running into stream limits or per-connection throughput limits, possibly imposed by a proxy outside your control.

[seungryulchoisc]

ClientConns can safely be accessed concurrently, and RPCs will be sent in parallel.

We are planning to use gRpc without TLS. But without TLS, most of HTTP2 does not support concurrent access. Does grpc-go implementation support h2c? (I found @bradfitz mentioned a related issue at golang/go#14141 but I am still not clear about it.)

[seungryulchoisc]

ClientConns can safely be accessed concurrently, and RPCs will be sent in parallel.

We are planning to use gRpc without TLS. But without TLS, most of HTTP2 does not support concurrent access. Does grpc-go implementation support h2c? (I found @bradfitz mentioned a related issue at golang/go#14141 but I am still not clear about it.)

[dfawley]

gRPC implements http2 discovery using external means as described in Section 3.4 of the HTTP/2 RFC, and not the version identification method described in sections 3.1-3.3. I.e. you tell a gRPC client to connect to a known-gRPC server and they speak the http2 wire protocol from there. TLS is off by default, but can be enabled with some options.

This is not true if you are using the Go http.Handler server implementation instead of grpc-go's server. In this case, TLS would need to be enabled, because h2c isn't supported by x/net/http2.

[dfawley]

gRPC implements http2 discovery using external means as described in Section 3.4 of the HTTP/2 RFC, and not the version identification method described in sections 3.1-3.3. I.e. you tell a gRPC client to connect to a known-gRPC server and they speak the http2 wire protocol from there. TLS is off by default, but can be enabled with some options.

This is not true if you are using the Go http.Handler server implementation instead of grpc-go's server. In this case, TLS would need to be enabled, because h2c isn't supported by x/net/http2.

[myles-mcdonnell]

@dfawley

ClientConns can safely be accessed concurrently, and RPCs will be sent in parallel.

So I should create a single connection when a client process bootstraps and close it when the process terminates? Could anything environmental 'break' a connection such that I would need to detect and re-connect?

[myles-mcdonnell]

@dfawley

ClientConns can safely be accessed concurrently, and RPCs will be sent in parallel.

So I should create a single connection when a client process bootstraps and close it when the process terminates? Could anything environmental 'break' a connection such that I would need to detect and re-connect?

[dfawley]

So I should create a single connection when a client process bootstraps and close it when the process terminates? Could anything environmental 'break' a connection such that I would need to detect and re-connect?

Technically, a ClientConn (inappropriately named IMO) can consistent of many connections that come and go for various reasons. However, the ClientConn should manage the connections itself, so if a connection is broken, it will reconnect automatically. And if you have multiple backends, it's possible to connect to multiple of them and load balance between them. You can learn a lot more about this at

https://github.com/grpc/grpc/blob/master/doc/load-balancing.md and
grpc/proposal#30

[dfawley]

So I should create a single connection when a client process bootstraps and close it when the process terminates? Could anything environmental 'break' a connection such that I would need to detect and re-connect?

Technically, a ClientConn (inappropriately named IMO) can consistent of many connections that come and go for various reasons. However, the ClientConn should manage the connections itself, so if a connection is broken, it will reconnect automatically. And if you have multiple backends, it's possible to connect to multiple of them and load balance between them. You can learn a lot more about this at

https://github.com/grpc/grpc/blob/master/doc/load-balancing.md and
grpc/proposal#30