[KEP-2400] Mount tmpfs memory-backed volumes with a noswap option if supported

[iholder101]

What type of PR is this?

/kind feature

What this PR does / why we need it:

This PR addresses #105978 which is about preventing memory-backed volumes from being swapped.

This PR brings the following behavior:

• When kubelet starts running with the fail-swap-on=false argument, it will check if the tmpfs noswap option is supported. If it's not - a warning log will be raised.
  • To understand if tmpfs noswap is supported or not, kubelet would try to mount a dummy tmpfs with the option and check for success. The directory is obviously cleaned up afterwards.
• If the tmpfs noswap option [1] is supported, use it by default for every tmpfs volume mount. With this options, memory-backed volumes will not be able to swap.

[1] https://www.kernel.org/doc/html/latest/filesystems/tmpfs.html

Example

After running kubelet with the --fail-swap-on=false argument, I've created the following pod and secret:

apiVersion: v1
kind: Secret
metadata:
  name: mysecret
data:
  username: dXNlcm5hbWU=  # Base64 encoded value of 'username'
  password: cGFzc3dvcmQ=  # Base64 encoded value of 'password'
---
apiVersion: v1
kind: Pod
metadata:
  name: mypod
spec:
  volumes:
  - name: my-volume
    emptyDir:
      medium: "Memory"
  - name: secret-volume
    secret:
      secretName: mysecret
  containers:
  - image: k8s.gcr.io/e2e-test-images/agnhost:2.9
    volumeMounts:
    - name: my-volume
      mountPath: /tmpfsdir
    - name: secret-volume
      mountPath: "/etc/secrets"
    command:
      - sleep
      - "36000"
    name: mycontainer

Then on the node we can see tmpfs noswap being used:

> grep noswap /proc/mounts
tmpfs /var/lib/kubelet/pods/a79290c3-8565-4d05-b065-e6fe3d471beb/volumes/kubernetes.io~projected/kube-api-access-dfw72 tmpfs rw,seclabel,relatime,size=51200k,inode64,noswap 0 0
tmpfs /var/lib/kubelet/pods/1b756da6-531f-40d0-aa44-df79de14ad04/volumes/kubernetes.io~projected/kube-api-access-gh9tk tmpfs rw,seclabel,relatime,size=394492600k,inode64,noswap 0 0
tmpfs /var/lib/kubelet/pods/1fe60428-4258-481a-99b6-bdc4ae671c71/volumes/kubernetes.io~empty-dir/my-volume tmpfs rw,seclabel,relatime,size=524288k,inode64,noswap 0 0
tmpfs /var/lib/kubelet/pods/1fe60428-4258-481a-99b6-bdc4ae671c71/volumes/kubernetes.io~secret/secret-volume tmpfs rw,seclabel,relatime,size=524288k,inode64,noswap 0 0
tmpfs /var/lib/kubelet/pods/1fe60428-4258-481a-99b6-bdc4ae671c71/volumes/kubernetes.io~projected/kube-api-access-sfr4d tmpfs rw,seclabel,relatime,size=524288k,inode64,noswap 0 0

Which issue(s) this PR fixes:

Fixes #105978

Does this PR introduce a user-facing change?

NONE

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

- [KEP]: https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/2400-node-swap/README.md

Note: I keep the functest and its revert commit for future reference, as in the future it would be a great candidate for a functional test (when noswap is widely supported enough)

/sig node

[iholder101]

/cc @liggitt @kannon92

[bart0sh]

/triage accepted
/priority important-longterm

[kannon92]

Nice work @iholder101!

can you update the swap KEP with some of our discoveries around this?

[iholder101]

Nice work @iholder101!

can you update the swap KEP with some of our discoveries around this?

Absolutely, will do!

[liggitt]

( for explicit /lgtm from either @liggitt or @mrunalp )

one comment about avoiding the global rootDir in the swap utility package, I'll defer to @mrunalp for lgtm after that's addressed

[mrunalp]

@iholder101 Can we make the commits purely additive?

[iholder101]

@iholder101 Can we make the commits purely additive?

@mrunalp Definitely.
I've now folded two commits (regarding error message improvement + address dim's request to organize imports) into previous ones.

PTAL

[k8s-ci-robot]

@iholder101: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-kubernetes-e2e-gce-providerless	9f7631f	link	false	/test pull-kubernetes-e2e-gce-providerless
pull-kubernetes-conformance-kind-ipv6-parallel	9f7631f	link	false	/test pull-kubernetes-conformance-kind-ipv6-parallel
pull-kubernetes-e2e-gce-storage-snapshot	a6b971f	link	false	/test pull-kubernetes-e2e-gce-storage-snapshot

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dims, iholder101, kwilczynski, mrunalp

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• pkg/kubelet/OWNERS [dims,mrunalp]
• pkg/util/kernel/OWNERS [dims,mrunalp]
• pkg/volume/emptydir/OWNERS [dims]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[haircommander]

/lgtm

great work here!

[k8s-ci-robot]

LGTM label has been added.

Git tree hash: 7ee51bea411226326f5247a0081b6e8681284089

[iholder101]

Thank you everyone for your fruitful reviews!

Unholding based on #124060 (comment)
/unhold

[AkihiroSuda]

Regression:

• ci-kubernetes-e2e-kind-rootless began to fail on 2024-05-24: Network plugin returns error: cni plugin not initialized, caused due to Turning off swap in unprivileged tmpfs mounts unsupported #125137

Fix:

• kubelet: fix Turning off swap in unprivileged tmpfs mounts unsupported #125138