Force restart VM expose the risk of data corruption

[xiesheng211]

What happened:
When VM get force restarted with GracePeriod == 0, the virt-launcher will get force deleted.
Code link

This could cause potential corruption if triggered when the node is under partition, which causes two VMs running at same time.

What you expected to happen:
SIGKILL should succeed when tearing down the VM.

One way to workaround is to set GracePeriod = 1, which is similar workaround as kubectl (link)

cc. @rmohr

[xpivarc]

I believe this is expected behavior. What do you mean by SIGKILL should succeed when tearing down the VM.?

[rmohr]

I think the issue is that it behaves like kubectl delete pods --grace-period 0 --force. It is basically not waiting for kubelet confirmation. So, if the kubelet is slow or unresponsive, we risk a split-brain scenario. I don't think this is what we intend to express with a forced restart. I think we just want to express: "Don't wait for a clean shutdown of the guest. I want to press the reset button.".

I think, as outlined by @xiesheng211, going with a grace period of 1 would be more what users would understand as a force-restart: If they system is unhealthy (e.g. node unresponsive), the pod will still be stuck until a force delete, in any other case, it would still restart practically right away.

[xpivarc]

I see your point. We don't wait for confirmation that the Pod was deleted.

[rmohr]

I see your point. We don't wait for confirmation that the Pod was deleted.

@xpivarc Does it sound reasonable to internally on the pod delete transform the 0 request into a 1 request?