feat: use multiple gRPC channels using connection pool

[girishc13]

Describe the feature

Using a single channel for all gRPC requests can become a performance bottleneck and a single point of failure in the cloud environment. Network issues are the most common root cause in the cloud. Using multiple connections will reduce the impact if one connection is facing issues. Recreating connections periodically will help to avoid intermittent network issues.

References:

• https://grpc.io/docs/guides/performance/#general
• load balancing: support creating additional connections when MAX_CONCURRENT_STREAM is reached grpc/grpc#21386
• Question: Python GRPC client, is there a way to create multiple connections with in the same channel? grpc/grpc#20985

Your proposal

A potential implementation might be available in the google-cloud-python library. A custom solution is required since gRPC Python doesn't have the support yet.

---

Environment

Screenshots

[girishc13]

Give the limitation in the Python library, I'm not sure if this feature can be implemented.
grpc/grpc#20985 (comment)

[JoanFM]

Should we go all the way and migrate to golang?

[JoanFM]

wrap the golang client in Python?

[girishc13]

I'm not sure about the packing and version conflicts. This would make it harder for the user to depend on something other than python. Let's think about improving the client-go library and verify if there are major improvements. If not then let's stick with the plain python library.

[jina-bot]

@jina-ai/product This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 14 days