New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Best practices for reusing connections, concurrency #682

Closed
pboyer opened this Issue May 13, 2016 · 11 comments

Comments

Projects
None yet
8 participants
@pboyer
Copy link

pboyer commented May 13, 2016

Thank you for this great library! I wanted to inquire about an issue I'm having.

I have a fixed number of machines I wish to make potentially concurrent RPCs to.

I would like maintain a pool of connections (*grpc.ClientConn) to these machines.

Otherwise, for every RPC call I need to make a new connection, which has overhead (as I understand it) and also can exhaust the number of file descriptors when a large number of concurrent RPC's are taking place.

  1. What are the restrictions on using clients and connections concurrently?
  2. Can I simply maintain a single connection and attach multiple client stubs to it dynamically (more flexible), or can I only have a single stub of a given type for a connection?
  3. My expectation is that I need to maintain my own pool with occasional heartbeats to maintain the health of the connection. Is that correct?
@mwitkow

This comment has been minimized.

Copy link
Contributor

mwitkow commented May 16, 2016

I'm not the gRPC maintainer, but have used gRPC extensively.

  1. AFAIK you can use the connections concurrently between clients. The only problem may be the limitation of number of concurrent HTTP2 streams you can do, but these count in thousands.
  2. Yes.
  3. Yes and no.
    • gRPC go has a Picker interface which deals with choosing a channel (tcp connection) to send it down to. Ideally you'd do all your pooling here. Don't know what the state of the Picker implementation is. @iamqizhao ?
    • gRPC 1.1 will have a way of healthchecking a backend, making Picker implementations better. See grpc/grpc-java#1648 (comment)
@dhowden

This comment has been minimized.

Copy link

dhowden commented May 17, 2016

  1. AFAIK you can use the connections concurrently between clients. The only problem may be the limitation of number of concurrent HTTP2 streams you can do, but these count in thousands.

I am interested to know the official line on this. I made a similar assumption as it appears to work for small messages, however I have found that larger messages are corrupted when multiple goroutines (each with their own client) use the same underlying *grpc.ClientConn.

@iamqizhao

This comment has been minimized.

Copy link
Contributor

iamqizhao commented May 17, 2016

You do not need to make a new connection for every RPC. A connection can be multiplexed by multiple clients or RPCs.

We have health check service (https://github.com/grpc/grpc/blob/master/doc/health-checking.md) so that a client can detect the health of the services. We are also designing the connection level story (L7 counterpart of TCP keepalive).

@iamqizhao

This comment has been minimized.

Copy link
Contributor

iamqizhao commented May 17, 2016

grpc.io email list would be the better place for these questions. close this now.

@timbunce

This comment has been minimized.

Copy link

timbunce commented Jun 30, 2017

You do not need to make a new connection for every RPC. A connection can be multiplexed by multiple clients or RPCs.

I'm unclear exactly what's meant by "multiplexed" here. Using a pool to avoid concurrent use?

Where is this covered in the documentation? Perhaps I'm missing it.

I've not been able to find docs that address sharing of connections, or clients, between goroutines.

grpc.io email list would be the better place for these questions.

(In my projects I've found it helpful to view questions in issues, and on the mailing list, as implicit bug reports on the documentation. That's reinforced by trying to answer questions with links to useful docs. If I can't then I'd create a PR for a doc change that would answer the question and reply with a link to that.)

@timbunce

This comment has been minimized.

Copy link

timbunce commented Jun 30, 2017

grpc-go-pool is a gRPC pool implementation. As a novice I look at it and think that's maybe what I want. But I also have a nagging doubt that it's not needed at all.

@dfawley

This comment has been minimized.

Copy link
Contributor

dfawley commented Jul 19, 2017

ClientConns can safely be accessed concurrently, and RPCs will be sent in parallel.

The only reason I can think of to use a pool of grpc clients to the same backend(s) is if you're running into stream limits or per-connection throughput limits, possibly imposed by a proxy outside your control.

@seungryulchoisc

This comment has been minimized.

Copy link

seungryulchoisc commented Aug 23, 2017

ClientConns can safely be accessed concurrently, and RPCs will be sent in parallel.

We are planning to use gRpc without TLS. But without TLS, most of HTTP2 does not support concurrent access. Does grpc-go implementation support h2c? (I found @bradfitz mentioned a related issue at golang/go#14141 but I am still not clear about it.)

@dfawley

This comment has been minimized.

Copy link
Contributor

dfawley commented Aug 23, 2017

gRPC implements http2 discovery using external means as described in Section 3.4 of the HTTP/2 RFC, and not the version identification method described in sections 3.1-3.3. I.e. you tell a gRPC client to connect to a known-gRPC server and they speak the http2 wire protocol from there. TLS is off by default, but can be enabled with some options.

This is not true if you are using the Go http.Handler server implementation instead of grpc-go's server. In this case, TLS would need to be enabled, because h2c isn't supported by x/net/http2.

@myles-mcdonnell

This comment has been minimized.

Copy link

myles-mcdonnell commented Jan 25, 2018

@dfawley

ClientConns can safely be accessed concurrently, and RPCs will be sent in parallel.

So I should create a single connection when a client process bootstraps and close it when the process terminates? Could anything environmental 'break' a connection such that I would need to detect and re-connect?

@dfawley

This comment has been minimized.

Copy link
Contributor

dfawley commented Jan 25, 2018

So I should create a single connection when a client process bootstraps and close it when the process terminates? Could anything environmental 'break' a connection such that I would need to detect and re-connect?

Technically, a ClientConn (inappropriately named IMO) can consistent of many connections that come and go for various reasons. However, the ClientConn should manage the connections itself, so if a connection is broken, it will reconnect automatically. And if you have multiple backends, it's possible to connect to multiple of them and load balance between them. You can learn a lot more about this at

https://github.com/grpc/grpc/blob/master/doc/load-balancing.md and
grpc/proposal#30

@lock lock bot locked as resolved and limited conversation to collaborators Sep 26, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.