Creating a loadbalancer is not correctly taking the topology for metalnetlet into account

[balpert89]

Describe the bug
When we see a loadbalancer & additional loadbalancerrouting created, we can see that regardless of the topology corresponding metalnet loadbalancers are created in every metalnetlet hosted compute cluster.

To Reproduce

1. setup several compute clusters
2. deploy metalnetlet controllers in each compute cluster with different zone labels
3. create a loadbalancer & loadbalancerrouting which targets only a subset of the topology

Expected behavior
the resulting metalnet loadbalancers should only be created by metalnetlet instances which are in sync with the anti-affinity shown in the loadbalancer.

[balpert89]

The actual problem is that too many Instance objects are being created for the loadbalancer & loadbalancerrouting. Assume this loadbalancer & loadbalancerrouting combination:

apiVersion: networking.ironcore.dev/v1alpha1
kind: LoadBalancer
metadata:
  annotations:
    meta.helm.sh/release-name: lb
    meta.helm.sh/release-namespace: bmilka
  creationTimestamp: "2025-05-16T21:10:24Z"
  finalizers:
  - apinet.ironcore.dev/loadbalancer
  labels:
    app.kubernetes.io/managed-by: Helm
  name: bmilka-lb-lb
  namespace: bmilka
  resourceVersion: "635881963"
  uid: bfa84d3e-a11e-4423-9c3c-046f9fc035ef
spec:
  ipFamilies:
  - IPv4
  networkRef:
    name: bmilka-vip-network
  ports:
  - port: 80
    protocol: TCP
  type: Public
status:
  ips:
  - 160.44.67.112

apiVersion: networking.ironcore.dev/v1alpha1
destinations:
- ip: 192.168.108.0
  targetRef:
    name: bmilka-vip-nic
    providerID: ironcore-net://ironcore-net-system/0e5f0289-bab9-5849-a233-ace6d196e79a/ironcore1-vm1.ironcore1-vm1-n1/bec871e3-fb5e-4768-8ffe-2e5f857ba2ef
    uid: 334a5c69-30b8-4b16-919e-bab9d3fe0cf8
kind: LoadBalancerRouting
metadata:
  annotations:
    meta.helm.sh/release-name: lb
    meta.helm.sh/release-namespace: bmilka
  creationTimestamp: "2025-05-16T21:10:24Z"
  labels:
    app.kubernetes.io/managed-by: Helm
  name: bmilka-lb-lb
  namespace: bmilka
  resourceVersion: "635881964"
  uid: 8a72f1f2-7d73-45ff-a483-ee1d5e4cf0d6
networkRef:
  name: bmilka-vip-network
  uid: 31f9c132-cdd7-4f2e-be8f-ed1ebc533e43

the loadbalancer only has 1 NIC it selects for creating the destinations. Until here this is fine.

However, in ironcore-net we can see too many Instance objects. We are currently running 13 machinepools (and therefore 13 metalnetlet instances). From our expectation we would see exactly 1 Instance object with the nodeRef set to the compute cluster & node where the virtual machine with its NIC is placed. But we can see 13 Instance objects:

k get instances -n ironcore-net-system -l apinetlet.ironcore.dev/loadbalancer-name=bmilka-lb-lb --no-headers |wc -l
13

[afritzler]

We should try to reproduce this in the test suite (https://github.com/ironcore-dev/ironcore-net/blob/main/internal/controllers/daemonset_controller_test.go) and eventually fix the problem.

[sujeet01]

We should try to reproduce this in the test suite (https://github.com/ironcore-dev/ironcore-net/blob/main/internal/controllers/daemonset_controller_test.go) and eventually fix the problem.

After reproducing this scenario in the test suite, here are the observations:

• The controller creates an Instance for each eligible node.
• Based on topology constraints, it correctly assigns the nodeRef to scheduled Instances.
• As a result, we see a mix of Instance objects with and without a nodeRef.
• Metalnetlet correctly processes only scheduled Instances (those with a nodeRef) to create metalnet loadbalancers.
• The unscheduled Instances (those without a nodeRef) are currently persisted and not removed.

The open question is whether to remove or retain these unscheduled Instances.

[balpert89]

potentially fixed with #414