Introduce load balancer for client-go

[chenk008]

Enhancement Description

• One-line enhancement description (can be used as a release note): Introduce load balancer for client-go
• Kubernetes Enhancement Proposal:

Background:

1. client-go use http2/gRPC to call APIServer. For GRPC and HTTP2, the client will only try to establish 1 single tcp connection.
2. High Availability Kubernetes clusters can be achieved by putting the API Server instances behind a load balancer. It is usually an L4 proxy.
3. Connection-level balancing isn't very useful. Once the connection is established, there's no more balancing to be done. All requests will get pinned to a single APIServer.

Proposal:

1. Add feature-gate 'CLIENT_LOAD_BALANCE', it will introduce load balancer which is similar to gRPC round_robin load balancer
2. We set multiple endpoint in client. The load balancer will connects to all the addresses it sees, and sends an request to each backend one at a time in order
3. Optional: client-go could get service kubernetes endpoints, and add all endpoints to load balancer.
4. Optional: Add a new headless service kubernetes-headless, client-go could get all endpoints from dns server.

• Discussion Link:
• Primary contact (assignee): @chenk008
• Responsible SIGs:
• Enhancement target (which target equals to which milestone):
  • Alpha release target (x.y):
  • Beta release target (x.y):
  • Stable release target (x.y):
• Alpha
  • KEP (k/enhancements) update PR(s):
  • Code (k/k) update PR(s):
  • Docs (k/website) update PR(s):

Please keep this description up to date. This will help the Enhancement Team to track the evolution of the enhancement efficiently.

[k8s-ci-robot]

@chenk008: The label(s) sig/sig-api-machinery cannot be applied, because the repository doesn't have them.

In response to this:

/sig sig-api-machinery

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[chenk008]

/sig api-machinery

[aojea]

There is a long discussion about this problem here kubernetes/kubernetes#18174.

A client side loadbalancer KEP was proposed and the decision was to avoid it because it does not address completely all the existing problems and increase the client complexity, see @deads2k comment here #3034 (comment)

[chenk008]

@aojea Thanks for your reminder.

May I ask, is it possible to only provide the capability of client load balancing, and leave the implementation of discovering multiple endpoints up to the developers?

case 1: An operator developer want to distribute the requests across multiple servers .
case 2: When a cloud vendor supports to expose multiple apiserver endpoints through DNS, the kubelet/kube-proxy could distribute the requests.

[chenk008]

Currently, we have not found an appropriate method to solve the dependency issues. I closes this issue for now.

I hope there's a way to address the problem of unbalanced request loads for multiples apiserver. @deads2k @aojea