pkg/k8s: fallback on retrieving CiliumNode from kube-apiserver

[aanm]

pkg/k8s: fallback on retrieving CiliumNode from kube-apiserver

Retrieving objects from caches can be useful to prevent doing useless
requests to kube-apiserver. In the unlikely event that the object
doesn't exist in the local cache Cilium can try to retrieve it from
kube-apiserver directly. For this particular case, with CiliumNode, it
is causing Cilium to fatal as it is unable to retrieve CiliumNode from
the cache, due subsystem initialization issues, thus we will fallback on
retrieving the object directly from kube-apiserver.

In this case, the subsystem initialization issue happened due to the
fact that CiliumNode watcher is blocked on its event handler by the
egressGatewayManager [1] which is blocked by the initialization of the
identity allocator [2]. Unfortunately, the identity allocator is only
initialized at a later stage causing the CiliumNode cache from being
populated with all of its nodes.

[1]

cilium/pkg/k8s/watchers/cilium_node.go

Line 56 in 933bdcb

k.egressGatewayManager.OnUpdateNode(n)

[2]

cilium/pkg/egressgateway/manager.go

Line 83 in 933bdcb

if err := manager.identityAllocator.WaitForInitialGlobalIdentities(identityCtx); err != nil {

Fixes: 69e4c69 ("k8s: optimize API calls made to kube-apiserver")

Fix Cilium fatal "Could not create or update CiliumNode resource, despite retries" on environments with `enable-ipv4-egress-gateway`

[aanm]

/test

[michi-covalent]

a magical one-line fix

[aanm]

Travis hit #21730