-
Notifications
You must be signed in to change notification settings - Fork 14.9k
Documentation for Swap memory management #51303
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Signed-off-by: Itamar Holder <iholder@redhat.com>
Signed-off-by: Itamar Holder <iholder@redhat.com>
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
✅ Pull request preview available for checkingBuilt without sensitive environment variables
To edit notification comments on pull requests, go to your Netlify project configuration. |
/cc @ajaysundark |
|
||
<!-- body --> | ||
|
||
## In which cases swap can be valuable? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@lmktfy I was hesitating to whether this fits to documentation.
What do you think? If it is I can gladly fill this section up.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about saving the Q&A for a blog article (which can be evergreen)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for working on this - it's really needed.
Here's quite a lot of feedback.
|
||
<!-- overview --> | ||
|
||
Swap is a fundamental and an invaluable Linux feature. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Watch out for implying that Windows doesn't support paging. It's not that backwards!
memory by swapping out unused data, | ||
shielding nodes from system-level memory spikes, | ||
preventing Pods from crashing when they hit their memory limits, | ||
and [much more](https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/2400-node-swap/README.md#user-stories). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Avoid hyperlinking to KEPs from the running text of docs.
As a result, the node special interest group within the Kubernetes project | ||
has invested significant effort into supporting swap on Linux nodes. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As a result, the node special interest group within the Kubernetes project | |
has invested significant effort into supporting swap on Linux nodes. |
True, but not right for our docs.
|
||
<!-- body --> | ||
|
||
## In which cases swap can be valuable? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about saving the Q&A for a blog article (which can be evergreen)?
{{< note >}} | ||
|
||
If configuration for `memorySwap` is not specified, | ||
by default the kubelet will apply the same behaviour as the `NoSwap` setting. | ||
|
||
{{< /note >}} | ||
|
||
{{< note >}} | ||
|
||
On Linux nodes, Kubernetes only supports running with swap enabled for hosts that use cgroup v2. | ||
On cgroup v1 systems, all Kubernetes workloads are not allowed to use swap memory. | ||
|
||
{{< /note >}} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Avoid consecutive notes.
The main note to add is that Kubernetes only supports configurable swap for Linux nodes; Windows nodes just have it unconditionally.
priority to system critical daemons. See the relevant section in the | ||
[recommended practices](#good-practice-for-using-swap-in-a-kubernetes-cluster) section below. | ||
|
||
### Memory-backed volumes |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
-
I recommend adding a page (or stub) that is like https://kubernetes.io/docs/concepts/security/windows-security/ but for Linux. That page should hyperlink here.
-
https://kubernetes.io/docs/concepts/security/secrets-good-practices/ should link here
* If the `noswap` option is not supported, kubelet will emit a warning log entry, | ||
then continue its execution. | ||
|
||
It is deeply encouraged to encrypt the swap space. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Put this in a caution block, at the start of the Risks and caveats section, and then hyperlink to this section. It's really important.
### Use of a dedicated disk for swap | ||
|
||
It is recommended to use a separate, encrypted disk for the swap partition. | ||
If swap resides on a partition or the root filesystem, workloads may interfere | ||
with system processes that need to write to disk. | ||
When they share the same disk, processes can overwhelm swap, | ||
disrupting the I/O of kubelet, container runtime, and systemd, which would impact other workloads. | ||
Since swap space is located on a disk, it is crucial to ensure the disk is fast enough for the intended use cases. | ||
Alternatively, one can configure I/O priorities between different mapped areas of a single backing device. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd tweak this. Using a dedicated device is one option, but it's not universally a good practice.
I suppose we should caution people to avoid rotational swap.
## Looking ahead | ||
|
||
Swap was intentionally scoped as a minimal feature for the initial release, | ||
which can and should be incrementally extended in the future. | ||
|
||
In the near future, additional features are planned to further improve swap capabilities, | ||
including better eviction mechanisms, extended API support, increased customizability, | ||
better debug abilities and more! | ||
|
||
Our future plans will depend on user feedback and the community's needs. | ||
Therefore, we encourage you to try out the current implementation and share your experiences with us. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We explicitly avoid statements about the future
## How can I learn more? | ||
|
||
You can check out the latest [blog-post about swap](/content/en/blog/_posts/2025-03-25-swap-fresh-improvements.md). | ||
|
||
For more information, please see [KEP-2400](https://github.com/kubernetes/enhancements/issues/4128) and its | ||
[design proposal](https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/2400-node-swap/README.md). | ||
|
||
## How do I get involved? | ||
|
||
Your feedback is always welcome! SIG Node [meets regularly](https://github.com/kubernetes/community/tree/master/sig-node#meetings) | ||
and [can be reached](https://github.com/kubernetes/community/tree/master/sig-node#contact) | ||
via [Slack](https://slack.k8s.io/) (channel **#sig-node**), or the SIG's | ||
[mailing list](https://groups.google.com/forum/#!forum/kubernetes-sig-node). A Slack | ||
channel dedicated to swap is also available at **#sig-node-swap**. | ||
|
||
Feel free to reach out to me, Itamar Holder (**@iholder101** on Slack and GitHub) | ||
if you'd like to help or ask further questions. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
## How can I learn more? | |
You can check out the latest [blog-post about swap](/content/en/blog/_posts/2025-03-25-swap-fresh-improvements.md). | |
For more information, please see [KEP-2400](https://github.com/kubernetes/enhancements/issues/4128) and its | |
[design proposal](https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/2400-node-swap/README.md). | |
## How do I get involved? | |
Your feedback is always welcome! SIG Node [meets regularly](https://github.com/kubernetes/community/tree/master/sig-node#meetings) | |
and [can be reached](https://github.com/kubernetes/community/tree/master/sig-node#contact) | |
via [Slack](https://slack.k8s.io/) (channel **#sig-node**), or the SIG's | |
[mailing list](https://groups.google.com/forum/#!forum/kubernetes-sig-node). A Slack | |
channel dedicated to swap is also available at **#sig-node-swap**. | |
Feel free to reach out to me, Itamar Holder (**@iholder101** on Slack and GitHub) | |
if you'd like to help or ask further questions. |
Use a whatsnext section (search docs for whatsnext
to see how we do this).
Description
Add documentation for KEP-2400 NodeSwap GA.
Basically, this PR does 2 things:
content/en/docs/concepts/cluster-administration/swap-memory-management.md
. This documentation is heavily based on the contents of the latest blog-post about swap memory: https://kubernetes.io/blog/2025/03/25/swap-linux-improvements/. While the content is very similar, there are minor updates.content/en/docs/concepts/architecture/nodes.md
.This documentation should contain everything the user needs to know for swap GA.
Issue
Fixes #51153