Do I need to use common-pools to wrap ManagedChannel

[thefallentree]

Hi,

Do we need to implement a pool of ManagedChannel on the client side?

We are currently sharing a single ManagedChannel to a single target on whole application, but we are seeing some pretty weird behavior that channel went into permanent DEADLINE_EXCEEDED mode.

We havn't been able to replicated it locally and we are trying anything we can..

Thanks

[buchgr]

what gRPC version are you using?

we are seeing some pretty weird behavior that channel went into permanent DEADLINE_EXCEEDED mode.

I don't understand. Could you please elaborate?

Also, we have a better chance of fixing your problem if you could share some code and additional details.

Thanks!

[buchgr]

Also, is the garbage collector running a lot and are the pause times similar or longer to your call deadlines?

[thefallentree]

we are using 0.13.2 latest release. We don't seem to be triggering GC , our call deadlines are pretty big, mostly 30s.

Cheers

[buchgr]

Do we need to implement a pool of ManagedChannel on the client side?

no that should not be necessary. also deadlines aren't per channel, but per client call.

hmm. not sure we can help without additional details...

[thefallentree]

Thanks for the information, we will try to gather some more data on our side.

It's been pretty random, as we have been hit by this phantom issue several times in the last several days.

[buchgr]

@thefallentree

we are seeing some pretty weird behavior that channel went into permanent DEADLINE_EXCEEDED mode.

what does that mean exactly? After some point in time every RPC fails with DEADLINE_EXCEEDED?

[ejona86]

@thefallentree, what sort of networking environment are you in? Could it be possible that the network (NAT, Load Balancer, etc) is breaking the TCP connection but not informing your client? We aren't currently doing manual keepalive so there may be some cases still which may cause O(minutes) to detect the connection failure. (I'm working on fixing that though.)

The only reason I would suggest having a pool of Channels is if it is difficult for your application to share Channels more precisely; your application doesn't actually know which Channels it wants to share so you have a common pool to try to reuse Channels as much as possible. All other cases of pooling I'd be suspicious of in that either 1) it is working around a bug or 2) there is a more appropriate place for the logic (like in a LoadBalancer).

[ejona86]

Closing for now. Will reopen if we get some more information. May be partially related to #1648.