Introduce value to run proxy-init as privileged #9873

mateiidavid · 2022-11-21T16:59:42Z

This change aims to solve two distinct issues that have cropped up in the proxy-init configuration.

First, it decouples allowPrivilegeEscalation from running proxy-init as root. At the moment, whenever the container is run as root, privilege escalation is also allowed. In more restrictive environments, this will prevent the pod from coming up (e.g security policies may complain about allowPrivilegeEscalation=true). Worth noting that privilege escalation is not necessary in many scenarios since the capabilities are passed to the iptables child process at build time.

Second, it introduces a new privileged value that will allow users to run the proxy-init container without any restrictions (meaning all capabilities are inherited). This is essentially the same as mapping root on host to root in the container. This value may solve issues in distributions that run security enhanced linux, since iptables will be able to load kernel modules that it may otherwise not be able to load (privileged mode allows the container nearly the same privileges as processes running outside of a container on a host, this further allows the container to set configurations in AppArmor or SELinux).

Privileged mode is independent from running the container as root. This gives users more control over the security context in proxy-init. The value may still be used with runAsRoot: false.

Fixes #9718

Signed-off-by: Matei David matei@buoyant.io

Relevant literature:

My hunch is that coupled with SELinux options at a Pod level, this should also fix #7767

Tests

I tested only with the control plane which makes use of the proxy-init partial. I tested four distinct scenarios:

Default (privileged=false, runAsRoot=false)
Privileged-only (privileged=true)
Privileged and root (privileged=true, runAsRoot=true)
Root only (privileged=false, runAsRoot=true)

I tested that the pods are running and we see what we are supposed to in the security context. All yaml output is taken from the destination pod in all 4 cases.

:; kgp -n linkerd
NAME                                      READY   STATUS    RESTARTS   AGE
linkerd-identity-b578dcd7-wzfft           2/2     Running   0          8m12s
linkerd-destination-77858c7cdf-4t84d      4/4     Running   0          8m12s
linkerd-proxy-injector-8467f465d9-m2hrg   2/2     Running   0          8m12s

    securityContext:
      allowPrivilegeEscalation: false
      capabilities:
        add:
        - NET_ADMIN
        - NET_RAW
      privileged: false
      readOnlyRootFilesystem: true
      runAsNonRoot: true
      runAsUser: 65534
    terminationMessagePath: /dev/termination-log

:; kgp -n linkerd
NAME                                      READY   STATUS    RESTARTS   AGE
linkerd-identity-b5b966fc8-pw55t          2/2     Running   0          66s
linkerd-destination-5797b48dd6-dx829      4/4     Running   0          66s
linkerd-proxy-injector-6b77c67598-jcgld   2/2     Running   0          66s

    securityContext:
      allowPrivilegeEscalation: true
      capabilities:
        add:
        - NET_ADMIN
        - NET_RAW
      privileged: true
      readOnlyRootFilesystem: true
      runAsNonRoot: true
      runAsUser: 65534
    terminationMessagePath: /dev/termination-log

:; kgp -n linkerd
NAME                                      READY   STATUS    RESTARTS   AGE
linkerd-identity-5cfbcd6bbd-w4gvf         2/2     Running   0          66s
linkerd-destination-57799fc85d-mb6kq      4/4     Running   0          66s
linkerd-proxy-injector-7446c779cb-sc7pv   2/2     Running   0          66s

    securityContext:
      allowPrivilegeEscalation: true
      capabilities:
        add:
        - NET_ADMIN
        - NET_RAW
      privileged: true
      readOnlyRootFilesystem: true
      runAsNonRoot: false
      runAsUser: 0
    terminationMessagePath: /dev/termination-log

:; kgp -n linkerd
NAME                                      READY   STATUS    RESTARTS   AGE
linkerd-identity-759d77d5c4-9vnfw         2/2     Running   0          62s
linkerd-destination-5f997cdf64-jbkp8      4/4     Running   0          62s
linkerd-proxy-injector-6465df8b58-4k9hq   2/2     Running   0          62s


    securityContext:
      allowPrivilegeEscalation: false
      capabilities:
        add:
        - NET_ADMIN
        - NET_RAW
      privileged: false
      readOnlyRootFilesystem: true
      runAsNonRoot: false
      runAsUser: 0
    terminationMessagePath: /dev/termination-log

This change aims to solve two distinct issues that have cropped up in the proxy-init configuration. First, it decouples `allowPrivilegeEscalation` from running proxy-init as root. At the moment, whenever the container is run as root, privilege escalation is also allowed. In more restrictive environments, this will prevent the pod from coming up (e.g security policies may complain about `allowPrivilegeEscalation=true`). Worth noting that privilege escalation is not necessary in many scenarios since the capabilities are passed to the iptables child process at build time. Second, it introduces a new `privileged` value that will allow users to run the proxy-init container without any restrictions (meaning all capabilities are inherited). This is essentially the same as mapping root on host to root in the container. This value may solve issues in distributions that run security enhanced linux, since iptables will be able to load kernel modules that it may otherwise not be able to load (privileged mode allows the container nearly the same privileges as processes running outside of a container on a host, this further allows the container to set configurations in AppArmor or SELinux). Privileged mode is independent from running the container as root. This gives users more control over the security context in proxy-init. The value may still be used with `runAsRoot: false`. Fixes #9718 Signed-off-by: Matei David <matei@buoyant.io>

stevej · 2022-11-21T18:14:27Z

charts/partials/templates/_proxy-init.tpl

@@ -41,7 +41,7 @@ imagePullPolicy: {{.Values.proxyInit.image.pullPolicy | default .Values.imagePul
 name: linkerd-init
 {{ include "partials.resources" .Values.proxyInit.resources }}
 securityContext:
-  {{- if or .Values.proxyInit.closeWaitTimeoutSecs .Values.proxyInit.runAsRoot }}
+  {{- if or .Values.proxyInit.closeWaitTimeoutSecs .Values.proxyInit.privileged }}


I'm curious: what is the relationship between .Values.proxyInit.closeWaitTimeoutSecs and this feature?

closeWaitTimeoutSecs is there to address long-running requests (predates me, #4276 should have info about it). The feature can't be used without running in privileged mode, which is why it acts as a conditional in our templates. When the change to run the init container as non-root was made, we've chosen to tackle the issue through templates (linkerd/linkerd2-proxy-init#49 (comment)) and this is since tied to anything that has to do with init container privileges.

alpeb

Logic sounds good to me, just some nits 👇

charts/linkerd-control-plane/values.yaml

charts/partials/templates/_proxy-init.tpl

alpeb · 2022-11-21T19:30:34Z

cli/cmd/testdata/install_custom_domain.golden

@@ -907,6 +909,7 @@ spec:
        - mountPath: /var/run/secrets/tokens
          name: linkerd-identity-token
      initContainers:
+


Any idea what is causing these empty lines?

They don't seem to be introduced by this PR 🤔 I've looked around and I have a bit of an idea about what's going on.

On latest main if I build the CNI and print out the install manifests:

:; git branch -v dependabot/cargo/clap-4.0.19 c430080e9 build(deps): bump clap from 4.0.18 to 4.0.19 eliza/issue-9671 9b777750b update goldenfiles * main 4ea8ab21d edge-22.11.3 change notes (#9884) matei/add-init-priv-escalation 3c0635b7f PR feedback :; bin/linkerd install --ignore-cluster| rg 'initContainer' -A 2 initContainers: - args: -- initContainers: - args: -- initContainers: - args: - --incoming-proxy-port

Culprit seems to be this line:

linkerd2/charts/linkerd-control-plane/templates/destination.yaml

Line 313 in 4ea8ab2

{{ if .Values.cniEnabled -}}

(and the equivalent line for identity, injector doesn't seem to suffer from this).

To fix, I've added - at the start of the if for destination and identity.

Result:

:; git branch dependabot/cargo/clap-4.0.19 eliza/issue-9671 main * matei/add-init-priv-escalation :; bin/linkerd install --ignore-cluster| rg 'initContainer' -A 2 initContainers: - args: - --incoming-proxy-port -- initContainers: - args: - --incoming-proxy-port -- initContainers: - args: - --incoming-proxy-port

The templates don't seem to want to update themselves through go test ./... -update though

kleimkuhler

Looks good after Alejandro's comments.

Signed-off-by: Matei David <matei@buoyant.io>

mateiidavid · 2022-11-24T15:01:47Z

Thanks for the feedback, addressed all of it. Will wait for #9889 (review) to merge first to have the emptyline fix.

alpeb

Thanks @mateiidavid , it seems there's still is some funkiness with the empty lines even after having merged #9889, but that's unrelated to this PR.

…t-priv-escalation

This change aims to solve two distinct issues that have cropped up in the proxy-init configuration. First, it decouples `allowPrivilegeEscalation` from running proxy-init as root. At the moment, whenever the container is run as root, privilege escalation is also allowed. In more restrictive environments, this will prevent the pod from coming up (e.g security policies may complain about `allowPrivilegeEscalation=true`). Worth noting that privilege escalation is not necessary in many scenarios since the capabilities are passed to the iptables child process at build time. Second, it introduces a new `privileged` value that will allow users to run the proxy-init container without any restrictions (meaning all capabilities are inherited). This is essentially the same as mapping root on host to root in the container. This value may solve issues in distributions that run security enhanced linux, since iptables will be able to load kernel modules that it may otherwise not be able to load (privileged mode allows the container nearly the same privileges as processes running outside of a container on a host, this further allows the container to set configurations in AppArmor or SELinux). Privileged mode is independent from running the container as root. This gives users more control over the security context in proxy-init. The value may still be used with `runAsRoot: false`. Fixes #9718 Signed-off-by: Matei David <matei@buoyant.io>

This change aims to solve two distinct issues that have cropped up in the proxy-init configuration. First, it decouples `allowPrivilegeEscalation` from running proxy-init as root. At the moment, whenever the container is run as root, privilege escalation is also allowed. In more restrictive environments, this will prevent the pod from coming up (e.g security policies may complain about `allowPrivilegeEscalation=true`). Worth noting that privilege escalation is not necessary in many scenarios since the capabilities are passed to the iptables child process at build time. Second, it introduces a new `privileged` value that will allow users to run the proxy-init container without any restrictions (meaning all capabilities are inherited). This is essentially the same as mapping root on host to root in the container. This value may solve issues in distributions that run security enhanced linux, since iptables will be able to load kernel modules that it may otherwise not be able to load (privileged mode allows the container nearly the same privileges as processes running outside of a container on a host, this further allows the container to set configurations in AppArmor or SELinux). Privileged mode is independent from running the container as root. This gives users more control over the security context in proxy-init. The value may still be used with `runAsRoot: false`. Fixes linkerd#9718 Signed-off-by: Matei David <matei@buoyant.io> Signed-off-by: Szymon Grzemski <sz.grzemski@gmail.com>

mateiidavid requested a review from a team as a code owner November 21, 2022 16:59

stevej reviewed Nov 21, 2022

View reviewed changes

alpeb reviewed Nov 21, 2022

View reviewed changes

adleong approved these changes Nov 21, 2022

View reviewed changes

kleimkuhler approved these changes Nov 22, 2022

View reviewed changes

mateiidavid added 2 commits November 24, 2022 11:06

PR feedback

3c0635b

Signed-off-by: Matei David <matei@buoyant.io>

Revert whitespace fix

a65f9df

Signed-off-by: Matei David <matei@buoyant.io>

alpeb approved these changes Nov 24, 2022

View reviewed changes

Merge branch 'main' of github.com:linkerd/linkerd2 into matei/add-ini…

65a8387

…t-priv-escalation

mateiidavid merged commit d45f733 into main Nov 25, 2022

mateiidavid deleted the matei/add-init-priv-escalation branch November 25, 2022 10:58

This was referenced Nov 25, 2022

iptables Chain already exists #7767

Closed

Docker container runtime can run as non-root with allowPrivilegeEscalation set to true #9552

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce value to run proxy-init as privileged #9873

Introduce value to run proxy-init as privileged #9873

mateiidavid commented Nov 21, 2022

stevej Nov 21, 2022

mateiidavid Nov 21, 2022

alpeb left a comment

alpeb Nov 21, 2022

mateiidavid Nov 24, 2022

mateiidavid Nov 24, 2022

kleimkuhler left a comment

mateiidavid commented Nov 24, 2022

alpeb left a comment

Introduce value to run proxy-init as privileged #9873

Introduce value to run proxy-init as privileged #9873

Conversation

mateiidavid commented Nov 21, 2022

Tests

stevej Nov 21, 2022

Choose a reason for hiding this comment

mateiidavid Nov 21, 2022

Choose a reason for hiding this comment

alpeb left a comment

Choose a reason for hiding this comment

alpeb Nov 21, 2022

Choose a reason for hiding this comment

mateiidavid Nov 24, 2022

Choose a reason for hiding this comment

mateiidavid Nov 24, 2022

Choose a reason for hiding this comment

kleimkuhler left a comment

Choose a reason for hiding this comment

mateiidavid commented Nov 24, 2022

alpeb left a comment

Choose a reason for hiding this comment