Enable validation of nginx.conf back in chroot mode

[sepich]

What this PR does / why we need it:

Due to CVE-2025-1974 validation of nginx.conf was disabled here:
#13070
Here is a more technical CVE description I was able to find:

• Use NGINX Client Body Buffering to upload a shared library to the target pod
• Send an AdmissionReview request to the NGINX admission controller with the ssl_engine load_module directive, causing NGINX to load the previously uploaded shared library
• Get the shared library being executed using the file descriptor reference from the /proc filesystem (in particular, by searching for it in /proc/*/fd/*)

So to exploit it, attacker need no connect simultaneously to nginx and ingress-controller sharing the same FS. It works for non-chroot mode, as nginx can then access service-account token in Pod at /var/run/secrets.

But that does not valid for chroot image, where nginx is running scoped to /chroot folder in Pod, so cannot access /var/run/secrets above. At time of Admission Webhook controller would call nginx -t:

ingress-nginx/internal/ingress/controller/util.go

Line 145 in dcfa450

return exec.Command(nc.Binary, "-c", cfg, "-t").CombinedOutput()

but in chroot image nginx executable is actually a wrapper:

ingress-nginx/rootfs/Dockerfile-chroot

Line 69 in 4038211

COPY --chown=www-data:www-data nginx-chroot-wrapper.sh /usr/bin/nginx

Which in turn would run nginx -t scoped to the same /chroot.

Validation of nginx.conf is very important in muti-team, multi-tenant clusters, where users learning k8s and often try to submit Ingress objects with errors. Currently that requires manual work to find out broken Ingress, fix it and communicate the problem.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• CVE Report (Scanner found CVE and adding report)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation only

Which issue/s this PR fixes

kubernetes/kubernetes#131009

How Has This Been Tested?

Unit tests are re-enabled back.
Also manual tests of resulting image in KIND by submitting invalid Ingress object.

Checklist:

• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I've read the CONTRIBUTION guide
• I have added unit and/or e2e tests to cover my changes.
• All new and existing tests passed.

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: sepich / name: Alex R (61bd529)

[k8s-ci-robot]

This issue is currently awaiting triage.

If Ingress contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[k8s-ci-robot]

Hi @sepich. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: sepich
Once this PR has been reviewed and has the lgtm label, please assign tao12345666333 for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[sepich]

Please tell if I missing something in the CVE workflow. As the main part regarding "code execution in the context of the ingress-nginx controller" is actually code execution in the context of nginx, and there is actually no issues with Admission Review where files are exposed by the controller itself.
I'm also ready to submit PR for v1.12 if it is ok.

[Gacko]

We won't accept this change for two reasons:

There's way more to exploit than just secrets. The vulnerability gives unauthenticated users the possibility to execute any code. Accessing secrets is just an example.

Even with the other CVEs fixed we can not safely enable the configuration check as there potentially still are other yet unknown vulnerabilities.

So in the end chroot does not effectively solve this vulnerability and rather gives a wrong feeling of security as it also requires the pod to be executed with extendend privileges.

[sepich]

Thank you for additional information!
Could you please comment on the following idea:

• Run ingress-nginx pods with admission-webhook disabled, they are only serving traffic
• Separate from those pods, run another group of pods which are only run admission-webhook but not accepting traffic to nginx (i.e networkpolicy, or additional cli arg)

Can we have working nginx.conf validation in this case?
Will that be accepted?

[Gacko]

No, as the current validation implementation checks if the required secrets exist and therefore needs API access to them.

chroot is not a security device, so I'd not consider the NGINX process isolated from the service account token.

[sepich]

How about:

• We add additional cli-arg to make possible to run controller in webhook only mode without access to secrets at all (separate SA and ClusterRole). I.e it would generate blank files for nginx certs, or re-use fake-cert content. That should be enough to validate nginx.conf (but obviously we cannot serve traffic from it, as all certs are invalid)
• We run such Pods with this cli-arg enabled and only point Admission there
• Another group of pods running with admission-webhook disabled, but under SA having access to secrets, they are only serving traffic