Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kubernetes securityContext sysctl settings not applied in pod #87198

Open
jeffdesc opened this issue Jan 14, 2020 · 1 comment
Open

Kubernetes securityContext sysctl settings not applied in pod #87198

jeffdesc opened this issue Jan 14, 2020 · 1 comment

Comments

@jeffdesc
Copy link

@jeffdesc jeffdesc commented Jan 14, 2020

What happened:
When configuring all components to use sysctl override configuration, the pod is not being configured properly. The settings are configured per deployment (even tried a single pod) but the specific '/proc' is still not what I was needed. This is a rootless pod, we're running a custom user in this container.

The configured deployment, Pod Security Policy and Kubelet config is attached here: https://gist.github.com/jeffdesc/d0a727a118d2e526a210b45253851a8c

When the following sysctl is configured...:

sysctls:
  - name: net.ipv4.ip_forward
    value: "1"

... then the setting is not propagated properly:

custom-user@test-sysctl-1-7c58dff94c-c9s2d:/$ cat /proc/sys/net/ipv4/ip_forward
0
custom-user@test-sysctl-1-7c58dff94c-c9s2d:/$ sysctl -a | grep -i ipv4.ip_forward
net.ipv4.ip_forward = 0

What you expected to happen:
When the following sysctl is configured...:

sysctls:
  - name: net.ipv4.ip_forward
    value: "1"

... then the setting is not propagated properly:

custom-user@test-sysctl-1-7c58dff94c-c9s2d:/$ cat /proc/sys/net/ipv4/ip_forward
1
custom-user@test-sysctl-1-7c58dff94c-c9s2d:/$ sysctl -a | grep -i ipv4.ip_forward
net.ipv4.ip_forward = 1

How to reproduce it (as minimally and precisely as possible):
Configure deployment, Pod Security Policy and Kubelet like this: https://gist.github.com/jeffdesc/d0a727a118d2e526a210b45253851a8c

Then spin up the deployment and verify if the setting is configured with: cat /proc/sys/net/ipv4/ip_forward.

Anything else we need to know?:

  • The container is configured rootless using a custom user (uid 10000)
  • A Pod Security Policy is applied and not allowing any privileged containers (which has to remain)

The flag is set in Docker, but when it's spin-up in Kubernetes the flag is being rewritten. Example:

$ docker run --rm -it test-local-user:1.10.3.27 /bin/bash
user-agent@ebbc6cd9b52b:/$ cat /proc/sys/net/ipv4/ip_forward
1
user-agent@ebbc6cd9b52b:/$ exit

$ kex test-sysctl-1-7c58dff94c-c9s2d /bin/bash
user-agent@test-sysctl-1-7c58dff94c-c9s2d:/$ cat /proc/sys/net/ipv4/ip_forward
0

Environment:

  • Kubernetes version (use kubectl version): v1.16.3
  • Cloud provider or hardware configuration: Virtual machines on VMware
  • OS (e.g: cat /etc/os-release):
NAME="Container Linux by CoreOS"
ID=coreos
VERSION=2303.3.0
VERSION_ID=2303.3.0
BUILD_ID=2019-12-02-2049
PRETTY_NAME="Container Linux by CoreOS 2303.3.0 (Rhyolite)"
ANSI_COLOR="38;5;75"
HOME_URL="https://coreos.com/"
BUG_REPORT_URL="https://issues.coreos.com"
COREOS_BOARD="amd64-usr"
  • Kernel (e.g. uname -a):
Linux <FQDN> 4.19.86-coreos #1 SMP Mon Dec 2 20:13:38 -00 2019 x86_64 Intel(R) Xeon(R) Gold 6142 CPU @ 2.60GHz GenuineIntel GNU/Linux
  • Install tools: Kubespray
  • Network plugin and version (if this is a network-related bug): Calico (v3.7.3)
  • Others: none
@jeffdesc

This comment has been minimized.

Copy link
Author

@jeffdesc jeffdesc commented Jan 14, 2020

/sig node

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.