nodeinit pods failing in 1.15.5

[dlahn]

Is there an existing issue for this?

• I have searched the existing issues

What happened?

We have recently tried to upgrade to 1.15.5 and the latest pre-release, but our nodeinit pods are failing with the following error:

nsenter: cannot open /proc/1/ns/ipc: Permission denied
!!! startup-script failed! exit code '1'

Reverting to 1.15.4 resolves the issue.

Cilium Version

1.15.5

Kernel Version

.

Kubernetes Version

v1.30.0-gke.145700

Regression

1.15.4

Sysdump

No response

Relevant log output

nsenter: cannot open /proc/1/ns/ipc: Permission denied
!!! startup-script failed! exit code '1'

Anything else?

No response

Cilium Users Document

• Are you a user of Cilium? Please add yourself to the Users doc

Code of Conduct

• I agree to follow this project's Code of Conduct

[dlahn]

Actually, I think it may have been introduced here:
https://github.com/cilium/cilium/pull/31641/files#diff-0ea42ad21164b19bec1732225e254d3096d1e4040481c00053669287d81015fe

[lmb]

Do you have any custom helm config related to the nodeinit pod?

[dlahn]

@lmb

  nodeinit:
    enabled: true
    reconfigureKubelet: true
    removeCbrBridge: true

[aanm]

@dlahn can you provide the steps you used for both 1.15.4 and 1.15.5? Thank you

[dlahn]

@aanm I think it may have happened here, https://github.com/cilium/cilium/pull/31641/files#diff-0ea42ad21164b19bec1732225e254d3096d1e4040481c00053669287d81015fe, so I mispoke, and I think the last working verison was 1.15.3. If we simply upgrade the helm chart to the newest version, we receive these errors.

The only way to get 1.15.4 to work is to add this to the nodeinit section:

  nodeinit:
    enabled: true
    reconfigureKubelet: true
    removeCbrBridge: true
    image:
      tag: "62093c5c233ea914bfa26a10ba41f8780d9b737f"

However, this doesn't work in 1.15.5

[dlahn]

Any ideas here?

[jbmolle]

Hi!
I raised this issue on K8s Github kubernetes/kubernetes#125069
I don't know if your error is related to that but I couldn't start Cilium either with version 1.15.5 because the pod annotations were removed and replaced by appArmorProfile type Unconfined.
But the appArmorProfile Unconfined doesn't work for me with containerd. So if you also use containerd you can try to reput the annotations like on 1.15.4:
container.apparmor.security.beta.kubernetes.io/cilium-agent: "unconfined"
container.apparmor.security.beta.kubernetes.io/clean-cilium-state: "unconfined"
container.apparmor.security.beta.kubernetes.io/mount-cgroup: "unconfined"
container.apparmor.security.beta.kubernetes.io/apply-sysctl-overwrites: "unconfined"

[dlahn]

Adding these annotations seems to have resolved the issue for us.