Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kubelet plugin registration should retry on failures #71487

Closed
saad-ali opened this issue Nov 28, 2018 · 7 comments · Fixed by #72873
Closed

Kubelet plugin registration should retry on failures #71487

saad-ali opened this issue Nov 28, 2018 · 7 comments · Fixed by #72873
Assignees
Labels
kind/bug Categorizes issue or PR as related to a bug. sig/storage Categorizes an issue or PR as relevant to SIG Storage.
Milestone

Comments

@saad-ali
Copy link
Member

What happened:

If kubelet plugin registration fails during the registration process, the kubelet never attempts to register the socket again (assuming the error was transient), unless the socket for the driver is deleted & recreated to trigger a new registration.

What you expected to happen:
Maybe retry with exponential backoff when kubelet plugin registration fails instead of just logging and waiting for another filesystem event like it does today.

Of course, if a socket returns a terminal error (does not implement the kubelet plugin registration service, for example), we should not retry.

Follow up on #71440

/milestone v1.14
/sig storage

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Environment:

  • Kubernetes version (use kubectl version):
  • Cloud provider or hardware configuration:
  • OS (e.g. from /etc/os-release):
  • Kernel (e.g. uname -a):
  • Install tools:
  • Others:

/kind bug

@k8s-ci-robot k8s-ci-robot added the sig/storage Categorizes an issue or PR as relevant to SIG Storage. label Nov 28, 2018
@k8s-ci-robot k8s-ci-robot added this to the v1.14 milestone Nov 28, 2018
@k8s-ci-robot k8s-ci-robot added the kind/bug Categorizes issue or PR as related to a bug. label Nov 28, 2018
@taragu
Copy link
Contributor

taragu commented Jan 13, 2019

@saad-ali I'd like to work on this issue

@taragu
Copy link
Contributor

taragu commented Jan 24, 2019

/assign

@saad-ali
Copy link
Member Author

saad-ali commented May 9, 2019

Re opening.

#72873 was a temporary work around for this.

#73891 is the permanent fix.

@saad-ali saad-ali reopened this May 9, 2019
@saad-ali saad-ali modified the milestones: v1.14, v1.15 May 9, 2019
@dstrebel
Copy link
Contributor

Hello! We are starting Bug Freeze for 1.15. Is this issue still planned on being incorporated into 1.15?

@taragu
Copy link
Contributor

taragu commented May 30, 2019

@dstrebel the dependent PR #73891 is not yet merged. @saad-ali can we get this in for 1.15?

@dims
Copy link
Member

dims commented Jun 6, 2019

@saad-ali said that #73891 is the "permanent fix." and that has merged. So let's close this out

/close

@k8s-ci-robot
Copy link
Contributor

@dims: Closing this issue.

In response to this:

@saad-ali said that #73891 is the "permanent fix." and that has merged. So let's close this out

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Categorizes issue or PR as related to a bug. sig/storage Categorizes an issue or PR as relevant to SIG Storage.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants