New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ERROR Swap]: running with swap on is not supported. Please disable swap #610

Closed
cjdcordeiro opened this Issue Dec 21, 2017 · 10 comments

Comments

Projects
None yet
9 participants
@cjdcordeiro
Copy link

cjdcordeiro commented Dec 21, 2017

Is this a BUG REPORT or FEATURE REQUEST?

BUG REPORT

Versions

kubeadm version (use kubeadm version): 1.9.0

Environment:

  • Kubernetes version (use kubectl version): 1.9.0
  • Cloud provider or hardware configuration: Raspberry Pi
  • OS (e.g. from /etc/os-release): Raspbian GNU/Linux 8 (jessie)
  • Kernel (e.g. uname -a): Linux 4.9.35-v7+
  • Others:

What happened?

When doing kubeadm init the error pops up [ERROR Swap]: running with swap on is not supported. Please disable swap. The suggested fix is to use kubelet's flag so basically:

kubeadm reset 
echo 'Environment="KUBELET_EXTRA_ARGS=--fail-swap-on=false"' >> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf

systemctl daemon-reload
systemctl restart kubelet

kubeadm init

But in the end, the error is still there. Maybe this is a Kubelet issue instead or maybe I'm misconfiguring something...

@luxas

This comment has been minimized.

Copy link
Member

luxas commented Dec 22, 2017

If you do set the kubelet flag, you're safe to do this kubeadm init --ignore-preflight-errors Swap, but I'd recommend you to just turn off swap instead, as you're going into unsupported territory here.

@luxas luxas closed this Dec 22, 2017

@cjdcordeiro

This comment has been minimized.

Copy link
Author

cjdcordeiro commented Jan 2, 2018

no need to re-open this but I still feel this is a bit counter intuitive. If you set the kubelet flag, then I would expect I don't need to ignore the preflight errors. If --ignore-preflight-errors is not passed, the error message remains the same whether the kubelet flag is enabled or not, and that's misleading.

@cli0

This comment has been minimized.

Copy link

cli0 commented Jan 13, 2018

What is the command to turn off swap?

@cjdcordeiro

This comment has been minimized.

Copy link
Author

cjdcordeiro commented Jan 15, 2018

swapoff -a

@kachkaev

This comment has been minimized.

Copy link

kachkaev commented Feb 27, 2018

It's true that swapoff -a is a silver bullet in most cases, however, certain k8s setups may really require swap. For instance, I've got a very small and cheap VM with just 1GB RAM, which I use for a personal GitLab Runner that rarely handles short CI/CD tasks. If I increase the size of the machine, I'll be paying more for a resource that's 99% idle. If I disable swap, npm install and other scripts inside the buid pods may hang because they require quite a lot of memory, although for short periods of time. Thus, a single-node kubeadm cluster with gitlab runner chart and swap is what suits me best.

Here is how I could get my mini-cluster up and running:

UPD: Below workaround applies to k8s 1.10- only - for 1.11+ see https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ (TLDR: you can specify kubeletConfiguration:\n failSwapOn: false in your kubeadm's config.yaml and then kubeadm init --config config.yaml --ignore-preflight-errors Swap).

kubeadm reset 

## ↓ see explanation below
sed -i '9s/^/Environment="KUBELET_EXTRA_ARGS=--fail-swap-on=false"\n/' /etc/systemd/system/kubelet.service.d/10-kubeadm.conf

systemctl daemon-reload
systemctl restart kubelet

echo "
kind: MasterConfiguration
apiVersion: kubeadm.k8s.io/v1alpha1
api:
  bindPort: ${K8S_API_PORT}
apiServerCertSANs: ${K8S_API_EXTRA_HOSTS}
" > /tmp/config.yaml

kubeadm init --config /tmp/config.yaml --ignore-preflight-errors Swap

## make possible to run workload on master
kubectl taint nodes --all node-role.kubernetes.io/master-

Below paragraphs apply to k8s 1.10- only
The reason why I used sed -i '9s/^/... instead of echo 'Environment="..."' >> ... as mentioned by @cjdcordeiro is because in the latter case the lines in 10-kubeadm.conf stacked in the wrong order:

...
Environment="KUBELET_CERTIFICATE_ARGS=--rotate-certificates=true --cert-dir=/var/lib/kubelet/pki"
ExecStart=
ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_SYSTEM_PODS_ARGS $KUBELET_NETWORK_ARGS $KUBELET_DNS_ARGS $KUBELET_AUTHZ$
Environment="KUBELET_EXTRA_ARGS=--fail-swap-on=false"

Because KUBELET_EXTRA_ARGS appeared after ExecStart, it looked like it was not picking up. With sed -i '9s/^/..., file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf ends up like so and works:

...
Environment="KUBELET_CERTIFICATE_ARGS=--rotate-certificates=true --cert-dir=/var/lib/kubelet/pki"
Environment="KUBELET_EXTRA_ARGS=--fail-swap-on=false"
ExecStart=
ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_SYSTEM_PODS_ARGS $KUBELET_NETWORK_ARGS $KUBELET_DNS_ARGS $KUBELET_AUTHZ$

It'd be great if enabling swap in kubeadm was easier than now – this would save people across the world tones of hours. Making my mini-cluster work after upgrading to 1.8 was a real pain because I'm quite inexperienced in Linux administration and I think it'd be great if others did not have to take the same path. The ideal solution would look like this IMO:

echo "
kind: MasterConfiguration
apiVersion: kubeadm.k8s.io/v1alpha1
kubeletConfiguration:
  allowSwap: true
" > /tmp/config.yaml

kubeadm init --config /tmp/config.yaml

Of course, enabling swap should remain an edge case because it may bite in lots of circumstances. However, it'd be great if kubeadm users had a choice. Until then, it'd be great if there was an opened issue about enabling swap.

@mcronce

This comment has been minimized.

Copy link

mcronce commented Mar 12, 2018

+1 for having an option for an experimental/alpha/etc "feature" flag to enable swap that adds it to the kubeadm drop-in in kubelet.service.d and disables that pre-flight check. Definitely an edge case, agreed, but it would be nice if it were easier.

(Googling and being able to quickly find this issue helps a lot, though.)

@geerlingguy

This comment has been minimized.

Copy link

geerlingguy commented May 9, 2018

@yeison

This comment has been minimized.

Copy link

yeison commented Oct 2, 2018

I turned my swap off but I'm still getting this error.

@rcmorano

This comment has been minimized.

Copy link

rcmorano commented Nov 5, 2018

Just for the record, at least in an ubuntu-based system, I guess the place to add the --fail-swap-on=false flag is in /etc/default/kubelet file; not in the systemd conf file itself.

@neolit123

This comment has been minimized.

Copy link
Member

neolit123 commented Nov 5, 2018

Just for the record, at least in an ubuntu-based system, I guess the place to add the --fail-swap-on=false flag is in /etc/default/kubelet file; not in the systemd conf file itself.

for 1.11+ this is true.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment