Thick client side load balancing without using load balancer

[archit-harness]

What is the best practice to use client-side load balancing?
I read through this thread - #428 and various options provided.
I found one example - https://github.com/grpc/grpc-java/blob/master/examples/src/main/java/io/grpc/examples/customloadbalance/CustomLoadBalanceClient.java

But don't think its dynamically updating the endpoints of the servers based on scale up/down.

Trying to follow - https://github.com/sercand/kuberesolver where they are using Kubernetes API to watch for IPs and updating the endpoints to do round-robin load balancing and making server as headless service.

Found other blogs having different approaches- https://medium.com/jamf-engineering/how-three-lines-of-configuration-solved-our-grpc-scaling-issues-in-kubernetes-ca1ff13f7f06

But if I close the gRPC connection I do not benefit from the long-lived connection provided by gRPC.

Let me know your thoughts, what is the preferred approach for client-side load balancing?

[ejona86]

For k8s assuming you want L7 load balancing, it is normal to use a headless service with the round_robin load balancer. That can be done by calling channelBuilder.defaultLoadBalancingPolicy("round_robin").

When using DNS to resolve addresses, yes, you will want to configure serverBuilder.maxConnectionAge() on your server to occasionally cycle connections so that the client re-resolves DNS addresses. I'd suggest using an age of some number of minutes.

(That approach would work well for L4 load balancing as well, but you'd use pick_first with shuffleAddressList enabled.)

As a slightly more advanced alternative, using the k8s watch API can work well, and with that approach you don't need to use max connection age. We don't have a built-in implementation of the k8s watch, but there are examples floating around. The NameResolver API is experimental and we know we will change it in some ways before marking it stable, but such changes we work to make easy to absorb, as we know people are using the API.

But if I close the gRPC connection I do not benefit from the long-lived connection provided by gRPC.

This isn't really a problem in practice. The amortized cost of the connection is pretty low, as long as you don't get very aggressive on the max connection age.

[archit-harness]

Thanks for the recommendation, we are trying this out and update here.

[ejona86]

Seems like this is resolved. If you end up having trouble, comment, and it can be reopened.

[archit-harness]

@ejona86 one more thing was to check with you, when we are using round robin and headless service, the client will refer to server as DNS:///headless-service: or just headless-service: ?

[ejona86]

@archit-harness, those are generally equivalent. gRPC detects headless-service: is incomplete and prefixes it with "dns:///" (the default for most systems). The "canonical" form is "dns:///headless-service" (with or without port). No scheme prefix is a short form.

(In the olden days we only supported host:port, but when we added name resolver support which used the scheme we tried to detect if it was old-form and convert it into new-form. But the old-form is also useful as a shorthand.)

[archit-harness]

@ejona86 as per this blog - https://itnext.io/grpc-name-resolution-load-balancing-everything-you-need-to-know-and-probably-a-bit-more-77fc0ae9cd6c
It says the default scheme used is passthrough,
The definition - Passthrough (default): Just returns the target provided by the ClientConn without any specific logic.

So i didn't get as its specifically mentioned DNS separately, so wanted to confirm will both return same results?

[ejona86]

That is grpc-go-specific. Go (against my recommendations) didn't use the new form (there were some technical issues, but in my mind they had easy solutions). Although Go now does do what I mentioned if using grpc.NewClient() instead of grpc.Dial() (deprecated). That is a very recent development.

[archit-harness]

@ejona86 thanks for the clarification, so without DNS prefix also it will resolve the same. 👍