Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kube-scheduler doesn't retry rescheduling the pod #87145

Open
systm32 opened this issue Jan 13, 2020 · 1 comment
Open

Kube-scheduler doesn't retry rescheduling the pod #87145

systm32 opened this issue Jan 13, 2020 · 1 comment

Comments

@systm32
Copy link

@systm32 systm32 commented Jan 13, 2020

What happened: Pod remains in a Pending state. In the kubernetes yaml file we have one daemonSet, one PV and one PVC to be created. Daemonset pods are using the PVC. Deleting and recreating these resources in a loop creates this issue intermittently. Even in success case in the events section I do see few initial warning messages "pod has unbound immediate PersistentVolumeClaims" where scheduling fails which is expected as PVC , PV and pods are being created at same time but after few retries, it succeeds. But sometimes pod stucks in "Pending" state forever unless kube-scheduler is restarted.

Events:
Type Reason Age From Message


Warning FailedScheduling 28m (x2 over 28m) default-scheduler pod has unbound immediate PersistentVolumeClaims

x2 count remains the same.

What you expected to happen: Kube-scheduler should retry scheduling and pod status should be changed to Running.

How to reproduce it (as minimally and precisely as possible): Deleting and re-creating the resources using the yaml file in loop using a shell file.
#/bin/sh

while :
do
kubectl delete -f resources.yaml
envsubst < resources.yaml | kubectl apply -f -
/give some time to start the process/
sleep 5
kubectl get pods | grep 'resources' | grep "Pending"
rc=$?; if [ $rc != 1 ]; then exit $rc; fi
done

Anything else we need to know?: Kubernetes cluster is running in HA mode.

Environment:

  • Kubernetes version (use kubectl version):Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.2", GitCommit:"cff46ab41ff0bb44d8584413b598ad8360ec1def", GitTreeState:"clean", BuildDate:"2019-01-10T23:35:51Z", GoVersion:"go1.11.4", Compiler:"gc", Platform:"linux/amd64"}
    Server Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.2", GitCommit:"cff46ab41ff0bb44d8584413b598ad8360ec1def", GitTreeState:"clean", BuildDate:"2019-01-10T23:28:14Z", GoVersion:"go1.11.4", Compiler:"gc", Platform:"linux/amd64"}
  • Cloud provider or hardware configuration:
  • OS (e.g: cat /etc/os-release):
  • Kernel (e.g. uname -a):Linux cvp146.sjc.aristanetworks.com 3.10.0-957.1.3.el7.x86_64 #1 SMP Thu Nov 29 14:49:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
  • Install tools:
  • Network plugin and version (if this is a network-related bug):
  • Others:
@neolit123

This comment has been minimized.

Copy link
Member

@neolit123 neolit123 commented Jan 13, 2020

/remove-kind bug
/triage unresolved

Warning FailedScheduling 28m (x2 over 28m) default-scheduler pod has unbound immediate PersistentVolumeClaims

/sig storage scheduling

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.