Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] cloneset reuse terminating pod's name when pvc exists #1099

Closed
willise opened this issue Oct 9, 2022 · 5 comments · Fixed by #1113
Closed

[BUG] cloneset reuse terminating pod's name when pvc exists #1099

willise opened this issue Oct 9, 2022 · 5 comments · Fixed by #1113
Assignees
Labels
kind/bug Something isn't working

Comments

@willise
Copy link
Contributor

willise commented Oct 9, 2022

What happened:
The node is not ready and pod with pvc on this node is evicted. But as the kubelet is unready so it doesn't really delete the pod and the pod stuck in terminating state.
cloneset try to scale out a new pod and it finds there is a pvc exsits, so it reuse the pod name referred by pvc to create the pod. But the terminating pod already exists then it fails.

What you expected to happen:
A new pod can be created successfully.

How to reproduce it (as minimally and precisely as possible):

  1. Create a cloneset with pvc
  2. Stop the kubelet and wait 5min

Anything else we need to know?:

Environment:

  • Kruise version: 1.0.2
  • Kubernetes version (use kubectl version):
  • Install details (e.g. helm install args):
  • Others:
@willise willise added the kind/bug Something isn't working label Oct 9, 2022
@FillZpp
Copy link
Member

FillZpp commented Oct 10, 2022

@willise Yeah, this is the expected behavior for now. But I understand your intention, how about add a new optional strategy field in spec.scaleStrategy to decide whether it should reuse the instanceId of existing free PVCs when creating new Pods?

@willise
Copy link
Contributor Author

willise commented Oct 10, 2022

@willise Yeah, this is the expected behavior for now. But I understand your intention, how about add a new optional strategy field in spec.scaleStrategy to decide whether it should reuse the instanceId of existing free PVCs when creating new Pods?

I think this works, but I have another concern about the old PVC. After the terminating pod is deleted finally, should the old PVC be deleted manually or be processed by cloneset?

@FillZpp
Copy link
Member

FillZpp commented Oct 10, 2022

I think this works, but I have another concern about the old PVC. After the terminating pod is deleted finally, should the old PVC be deleted manually or be processed by cloneset?

With the field enabled, cloneset should cleanup those useless PVCs and then create new Pods with new PVCs.

@willise
Copy link
Contributor Author

willise commented Oct 10, 2022

With the field enabled, cloneset should cleanup those useless PVCs and then create new Pods with new PVCs.

Ok, I am looking forward it.

@FillZpp
Copy link
Member

FillZpp commented Oct 10, 2022

Ok, I am looking forward it.

And you are welcome if interested in contributing this :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants