Skip to content

CiliumLoadBalancerIPPool serviceSelector doesn't match Gateway resources #42994

Description

@MacLeodMike

Is there an existing issue for this?

  • I have searched the existing issues

Version

equal or higher than v1.18.4 and lower than v1.19.0

What happened?

I'm trying to migrate from ingress-nginx to Cilium's Gateway API but I have two CiliumLoadBalancerIPPool's - one for internal and one for external IPs - and the serviceSelector isn't matching the label on the Gateway resource. If I remove the serviceSelector from one of the pools the Gateway and Service resources acquire an IP from the pool, but that defeats the purpose of having two pools and using serviceSelector.

How can we reproduce the issue?

  1. Deploy kubernetes (I'm using Talos)
  2. Install the Gateway API CRDs (experimental channel):
    kubectl apply --server-side -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.4.0/experimental-install.yaml
  3. Install cilium with helm. I'm deploying a dual-stack configuration using the BGP control plane and native routing. The relevant helm values are:
routingMode: "native"
bgpControlPlane:
  enabled: true
enableIPv4Masquerade: false
enableIPv6Masquerade: false
autoDirectNodeRoutes: true
gatewayAPI:
  enabled: true
  enableAlpn: true
  enableAppProtocol: true
  gatewayClass:
    create: "true"
ipam:
  mode: "kubernetes"
k8s-require-ipv4-pod-cidr: true
k8s-require-ipv6-pod-cidr: true
ipv4:
  enabled: true
ipv6:
  enabled: true
k8s:
  requireIPv4PodCIDR: true
  requireIPv6PodCIDR: true
bpf:
  datapathMode: "netkit"
  1. Deploy the BGP Control Plane resources, including a CiliumBGPAdvertisement and two CiliumLoadBalancerIPPool's:
apiVersion: cilium.io/v2
kind: CiliumBGPAdvertisement
metadata:
  namespace: cilum
  name: bgp-advertisements
  labels:
    advertise: bgp
spec:
  advertisements:
    - advertisementType: "PodCIDR"
    - advertisementType: "Service"
      service:
        addresses:
          - ExternalIP
          - LoadBalancerIP
      selector:
        matchExpressions:
          - {key: somekey, operator: NotIn, values: ['never-used-value']}
      attributes:
        communities:
          standard: [ "64514:100" ]
---
apiVersion: cilium.io/v2alpha1
kind: CiliumLoadBalancerIPPool
metadata:
  name: public-pool
spec:
  blocks:
    - cidr: 203.0.113.244/29
    - start: 2001:db8:beef:cafe::244
      stop: 2001:db8:beef:cafe::247
  serviceSelector:
    matchLabels:
      network: public
---
apiVersion: cilium.io/v2alpha1
kind: CiliumLoadBalancerIPPool
metadata:
  name: internal-pool
spec:
  allowFirstLastIPs: "No"
  blocks:
    - cidr: 10.46.0.0/16
    - cidr: 3fff:dead:beef:cafe::/112
  serviceSelector:
    matchLabels:
      network: internal
  1. Deploy an example workload (registry, in this case), including Service, Gateway, and HTTPRoute resources:
apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/name: registry
    app.kubernetes.io/instance: registry
  name: registry
spec:
  ports:
  - name: registry
    port: 5000
    protocol: TCP
  selector:
    app.kubernetes.io/instance: registry
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway
metadata:
  name: registry
  labels:
    network: internal
spec:
  infrastructure:
    annotations:
      loadBalancerClass: io.cilium/bgp-control-plane
  gatewayClassName: cilium
  listeners:
  - name: http
    hostname: gateway.lan.example.com
    port: 80
    protocol: HTTP
    allowedRoutes:
      namespaces:
        from: All
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
  name: registry
  labels:
    network: internal
spec:
  hostnames:
  - gateway.lan.example.com
  parentRefs:
  - name: registry
  rules:
  - backendRefs:
    - name: registry
      port: 5000
  1. Review the Service and Gateways to see that no external IP address has been assigned:
user@workstation:~/talos$ kubectl get gateway registry -o wide
NAME       CLASS    ADDRESS   PROGRAMMED   AGE
registry   cilium             False        90s
user@workstation:~/talos$ kubectl get svc cilium-gateway-registry -o wide
NAME                      TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE     SELECTOR
cilium-gateway-registry   LoadBalancer   172.31.113.51   <pending>     80:32053/TCP   2m24s   <none>

We can see in detail that the Gateway is waiting for an IP:

user@workstation:~/talos$ kubectl get gateway registry -o json | jq -r .status.conditions
[
  {
    "lastTransitionTime": "2025-11-26T16:19:59Z",
    "message": "Gateway successfully scheduled",
    "observedGeneration": 1,
    "reason": "Accepted",
    "status": "True",
    "type": "Accepted"
  },
  {
    "lastTransitionTime": "2025-11-26T16:19:59Z",
    "message": "Gateway waiting for address",
    "observedGeneration": 1,
    "reason": "AddressNotAssigned",
    "status": "False",
    "type": "Programmed"
  }
]
  1. Patch the internal CiliumLoadBalancerIPPool to remove the serviceSelector:
    kubectl patch ciliumloadbalancerippools.cilium.io internal-pool --type=json -p='[{"op": "remove", "path": "/spec/serviceSelector"}]'
  2. Verify that IPs have been assigned to the Service and Gateway:
user@workstation:~/talos$ kubectl get gateway registry -o wide
NAME       CLASS    ADDRESS     PROGRAMMED   AGE
registry   cilium   10.46.0.2   True         31m
user@workstation:~/talos$ kubectl get svc cilium-gateway-registry -o wide
NAME                      TYPE           CLUSTER-IP      EXTERNAL-IP                        PORT(S)        AGE   SELECTOR
cilium-gateway-registry   LoadBalancer   172.31.113.51   10.46.0.2,3fff:dead:beef:cafe::2   80:32053/TCP   32m   <none>

Cilium Version

$ cilium -n kube-system version
cilium-cli: v0.18.3 compiled with go1.24.2 on linux/amd64
cilium image (default): v1.17.2
cilium image (stable): v1.18.4
cilium image (running): 1.18.4

Kernel Version

Linux version 6.12.57-talos (root@buildkitsandbox) (gcc (GCC) 14.3.0, GNU ld (GNU Binutils) 2.44) #1 SMP Wed Nov 5 12:59:35 UTC 2025

Kubernetes Version

Client Version: v1.34.1
Kustomize Version: v5.7.1
Server Version: v1.34.1

Regression

No response

Sysdump

cilium-sysdump-20251126-120736.zip

Relevant log output

Anything else?

This same configuration works fine with an ingress-nginx Ingress resource (tested with the seviceSelector restored):

$ kubectl get svc -n internal-nginx internal-nginx-controller
NAME                        TYPE           CLUSTER-IP       EXTERNAL-IP                        PORT(S)                      AGE
internal-nginx-controller   LoadBalancer   172.31.142.184   10.46.0.1,3fff:dead:beef:cafe::1   80:32448/TCP,443:31133/TCP   66m
$ kubectl get svc -n internal-nginx internal-nginx-controller -o yaml
apiVersion: v1
kind: Service
metadata:
  annotations:
    meta.helm.sh/release-name: internal-nginx
    meta.helm.sh/release-namespace: internal-nginx
  creationTimestamp: "2025-11-26T16:13:20Z"
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: internal-nginx
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.14.0
    helm.sh/chart: ingress-nginx-4.14.0
    network: internal
  name: internal-nginx-controller
  namespace: internal-nginx
  resourceVersion: "117786"
  uid: 1b009c44-0b4e-469d-bb63-6cfac4487a82
spec:
  allocateLoadBalancerNodePorts: true
  clusterIP: 172.31.142.184
  clusterIPs:
  - 172.31.142.184
  - fd00:beef:cafe::198c
  externalTrafficPolicy: Cluster
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  - IPv6
  ipFamilyPolicy: PreferDualStack
  ports:
  - appProtocol: http
    name: http
    nodePort: 32448
    port: 80
    protocol: TCP
    targetPort: http
  - appProtocol: https
    name: https
    nodePort: 31133
    port: 443
    protocol: TCP
    targetPort: https
  selector:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: internal-nginx
    app.kubernetes.io/name: ingress-nginx
  sessionAffinity: None
  type: LoadBalancer
status:
  conditions:
  - lastTransitionTime: "2025-11-26T16:13:20Z"
    message: ""
    reason: satisfied
    status: "True"
    type: cilium.io/IPAMRequestSatisfied
  loadBalancer:
    ingress:
    - ip: 10.46.0.1
      ipMode: VIP
    - ip: 3fff:dead:beef:cafe::1
      ipMode: VI

Cilium Users Document

  • Are you a user of Cilium? Please add yourself to the Users doc

Code of Conduct

  • I agree to follow this project's Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/agentCilium agent related.area/servicemeshGH issues or PRs regarding servicemeshfeature/k8s-gateway-apikind/bugThis is a bug in the Cilium logic.kind/community-reportThis was reported by a user in the Cilium community, eg via Slack.needs/triageThis issue requires triaging to establish severity and affected area.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions