operator: Add LB-IPAM

[dylandreimerink]

This PR introduces a new feature called LoadBalancer IP Address Management, LB-IPAM for short. The goal of LB-IPAM is to provide a mechanism to allocate and assign IP addresses to services of type LoadBalancer, to be used when no external IPAM is available. The primary use-case we are targeting is a private cloud/bare metal deployment where one or more CIDRs are assigned to the cluster to be used. Additional Cilium features should be utilized to direct traffic for the assigned IPs to the cluster and the correct nodes(for now just BGP, but not exclusively).

LB-IPAM has the concept of an IP Pool which is a list of CIDRs and an optional service label selector. Users can add, modify or remove these at will via the CiliumLoadBalancerIPPool custom resource (formally the CiliumBGPLoadBalancerIPPool, which was renamed to make it more generic). The service label selector can be used to limit which services can get IPs from the given pool.

LB-IPAM can allocate and assign both IPv4 and IPv6 addresses in SingleStack and DualStack operation. LB-IPAM can be deployed in a cluster with multiple load balancers since it respects the loadBalancerClass values of services. Lastly LB-IPAM allows services to request one or more specific IPs.

Add LoadBalancer IP address management (LB-IPAM)

[lazedo]

Hello,
if this is to replace the metallb loadbalancer then can you ensure that something like metallb.universe.tf/allow-shared-ip: "my-shared-ip" is implemented (sharing a loadbalancer ip) ? I do see that we can request a specific ip, but requesting by a known name is very useful and allows to deploy service manifests without knowing the shared ip in advance

[dylandreimerink]

if this is to replace the metallb loadbalancer

That is the eventual intent combined with some other work that is still in progress.

then can you ensure that something like metallb.universe.tf/allow-shared-ip: "my-shared-ip" is implemented (sharing a loadbalancer ip) ?

The LB-IPAM version in this PR doesn't support that feature. I am not going for full feature parity with MetalLB, at least not in the initial release. The feature seems quite involved, not easily added to the PR as it stands, though I am open to adding it in a future iteration. I have created issue #21776 to track it.

[dylandreimerink]

@ldelossa I have now refactored and processed feedback, replaced the cell-wide context+cancel+done channel with a workerpool as discussed offline and plumbed the context via params so there is no longer a ctx field in the cell.

[ldelossa]

thanks @dylandreimerink - LGTM.

[dylandreimerink]

/test

[dylandreimerink]

/test

[dylandreimerink]

Alright got 5 approvals covering all requested teams. All feedback is addresses. All tests are green except for:

ConformaceAKS: fails with Running test client-egress-to-echo-expression-deny: setting up test: applying network policies: policies were not applied on all Cilium nodes in time: error dialing backend: read unix @->/tunnel-uds/proxysocket: read: connection reset by peer. This seems unrelated to any changes made in this PR but is also isn't a known flake, I will be making a new issue for this one. Edit: #22367

ConformanceGKE: fails at install time with "Unable to install Cilium: timeout while waiting for status to become successful: context deadline exceeded". However, by the time we collect the sysdump all agents are up and cilium status is happy, so it looks like something is waiting to short. In any case, it doesn't look related to this PR.

ConformanceKind1.19 / installation-and-connectivity: Fails due to a curl timeout in the client-egress-to-echo-service-account-deny senario. I am unable to find an existing GH issue for this flake, but it again doesn't seem to have much to do with this PR since it is in policy enforcement.

Travis CI hits #21730.

None of these failures seem to be caused by changed made in this PR since they are known flakes. So I will add the ready-to-merge label.