Support Network Endpoint Groups as a load balancer back end

[nrb]

/kind feature

Describe the solution you'd like
[A clear and concise description of what you want to happen.]

Right now, CAPG uses unmanaged instance groups to group VMs for load balancing. This mostly works, but has the following limitation, quoted from their docs, emphasis mine:

Don't put a VM in more than one load-balanced instance group. If a VM is a member of two or more unmanaged instance groups, or a member of one managed instance group and one or more unmanaged instance groups, Google Cloud limits you to only using one of those instance groups at a time as a backend for a particular backend service.

This limitation means it is difficult to have nodes participating in multiple load balancers based on Instance Groups. This could happen in the case of workload cluster worker nodes participating in a load balancer managed by a CNI solution for an application hosted on a CAPG cluster, for example.

Network Endpoint Groups, by contrast, allow load balancing by software service rather than the entire VM. This allows more flexibility for end user application load balancing.

Anything else you would like to add:

This came out of a discussion in the GCP cloud provider about re-using an instance group for load balancing. Our (non-CAPG) installation was trying to associate nodes with different instance groups, which resulted in errors.

As we move to Cluster API-based solutions, it would be nice if the cloud provider's more flexible solutions were supported.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale