Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pod don't start after delete. #157

Closed
chemist opened this issue Feb 11, 2020 · 2 comments · Fixed by #162
Closed

Pod don't start after delete. #157

chemist opened this issue Feb 11, 2020 · 2 comments · Fixed by #162
Labels
bug Something isn't working examples Related to the examples

Comments

@chemist
Copy link

chemist commented Feb 11, 2020

I install ember-csi on k8s 1.17, centos 7, only lvm and iscsi (clean system, k8s installed with kubeadm)

And I see next problem, when I made "kubectl delete pod csi-node-***, it can't restart any more.
It always crashes.

After reboot node, it starts.
I attach log from 1 node here.

I found that this problem because of /dev/termination-log

systemctl stop docker containerd Kubelet
umount /dev/termination-log
systemctl start docker kubelet

after it all starts.

I add to ds
terminationMessagePath: /tmp/termination-csi
it helps, now I don't have any problem with permanent crashes.

log here
@Akrog
Copy link
Member

Akrog commented Mar 12, 2020

Thanks for reporting the issue and the solution!
We'll update the examples and the operator.

@Akrog Akrog added bug Something isn't working examples Related to the examples labels Mar 12, 2020
@Akrog
Copy link
Member

Akrog commented Mar 13, 2020

The operator is already doing the right thing the right thing, so this is just a problem with this repository's example:

			TerminationMessagePath: "/tmp/termination-log",

Akrog added a commit to Akrog/ember-csi that referenced this issue Apr 7, 2020
@Akrog
Fix removing pods in examples
2bccb59 
If we delete a pod of the Kubernetes example they won't be able to
properly start again.

This is caused by a missing terminationMessagePath, which we add in this
patch to fix this issue.

Fix embercsi#157
@Akrog Akrog mentioned this issue Apr 7, 2020
Merged
trgill pushed a commit to trgill/ember-csi that referenced this issue Apr 21, 2020
@Akrog @trgill
Fix removing pods in examples
e2e8aa1 
If we delete a pod of the Kubernetes example they won't be able to
properly start again.

This is caused by a missing terminationMessagePath, which we add in this
patch to fix this issue.

Fix embercsi#157
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working examples Related to the examples
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add RWX for block volumes Akrog/ember-csi
2 participants
@chemist @Akrog