Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot remove disk pressure taint #83668

Open
davidq2q opened this issue Oct 9, 2019 · 4 comments

Comments

@davidq2q
Copy link

commented Oct 9, 2019

What happened:
Worker nodes were all under disk pressure. New pods were being evicted. After freeing up space and disk inodes disk pressure state was not refreshed. Removing the taint manually was not working either. Full restart of everything was required.

What you expected to happen:
Disk pressure state to be changed to false when inodes and space are freed up.

How to reproduce it (as minimally and precisely as possible):
Fill up your disk space get disk pressure state to true on specific node. Then cleanup disk under pressure treshold and try to remove taint.

Anything else we need to know?:

Environment:

  • Kubernetes version (use kubectl version):
    v1.13.5
  • Cloud provider or hardware configuration:
    Bare metal
    2x Master node
    3x Worker node
  • OS (e.g: cat /etc/os-release):
NAME="Debian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"```
- Kernel (e.g. `uname -a`):
```Linux node1 4.19.0-0.bpo.2-amd64 #1 SMP Debian 4.19.16-1~bpo9+1 (2019-02-07) x86_64 GNU/Linux```
- Install tools:
ansible (kubespray) and terraform
- Network plugin and version (if this is a network-related bug):
flannel
- Others:
@neolit123

This comment has been minimized.

Copy link
Member

commented Oct 9, 2019

/sig node

@k8s-ci-robot k8s-ci-robot added sig/node and removed needs-sig labels Oct 9, 2019
@zouyee

This comment has been minimized.

Copy link
Member

commented Oct 10, 2019

/assign

@zouyee

This comment has been minimized.

Copy link
Member

commented Oct 10, 2019

Unable to reproduce the problem. The taint was transient and has been removed after removed the extra file.

@k82cn

This comment has been minimized.

Copy link
Member

commented Oct 12, 2019

please help to share the log of controller and kubelet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.