Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kubevirt: Ignore completed virt-launcher pods #3990

Merged

Conversation

qinqon
Copy link
Contributor

@qinqon qinqon commented Oct 31, 2023

- What this PR does and why is it needed
When a live migration fails at completed virt-launcher pod with newer
creation timestamp is lingering in the system so the virt-launcher that
is really running the VM is hidden by it and routes are wrongly
reconstructed. This change take into account if the pod is completed to
ignore it marking it as stale.

Closes https://issues.redhat.com/browse/OCPBUGS-22767

- How to verify it
It includes a unit test that emulate the result of failing migration, completed virt-launcher pod with newer creation timestamp.

- Description for the changelog
kubevirt: Ignore completed virt-launcher pods

@qinqon qinqon force-pushed the kubevirt-ignore-completed-virt-launcher-pod branch 2 times, most recently from 04a5f25 to d1e5820 Compare November 3, 2023 05:39
@qinqon qinqon changed the title wip kubevirt: Ignore completed virt-launcher pods Nov 3, 2023
@qinqon qinqon force-pushed the kubevirt-ignore-completed-virt-launcher-pod branch 2 times, most recently from b78d756 to 6740e48 Compare November 3, 2023 09:49
@qinqon qinqon marked this pull request as ready for review November 3, 2023 09:50
@qinqon qinqon force-pushed the kubevirt-ignore-completed-virt-launcher-pod branch from 6740e48 to e2609f4 Compare November 3, 2023 09:50
if !IsPodLiveMigratable(pod) {
return false, nil
return true, nil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is counter-intuitive. Why don't you do the IsPodLiveMigratable check first, and return false if the pod is not migratable.

}
vmPods, err := findVMRelatedPods(client, pod)
if err != nil {
return false, fmt.Errorf("failed finding related pods for pod %s/%s when checking live migration left overs: %v", pod.Namespace, pod.Name, err)
return true, fmt.Errorf("failed finding related pods for pod %s/%s when checking live migration left overs: %v", pod.Namespace, pod.Name, err)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why? Does it matter if we are returning an error? false is the better convention here so it is weird that you had to change this value.

@qinqon qinqon force-pushed the kubevirt-ignore-completed-virt-launcher-pod branch from e2609f4 to 043b146 Compare November 3, 2023 10:05
@qinqon qinqon requested a review from jcaamano November 3, 2023 10:05
@@ -123,11 +123,14 @@ func EnsurePodAnnotationForVM(watchFactory *factory.WatchFactory, kube *kube.Kub
return podAnnotation, nil
}

// IsMigratedSourcePodStale return true if there are other pods related to
// to it and any of them has newer creation timestamp.
// IsMigratedSourcePodStale return false is the pod is live migratable and not completed
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IsMigratedSourcePodStale return false if the pod is live migratable, not completed and is the running VM pod with newest creation timestamp

@qinqon qinqon force-pushed the kubevirt-ignore-completed-virt-launcher-pod branch from 043b146 to ce91a2c Compare November 3, 2023 11:09
@qinqon qinqon force-pushed the kubevirt-ignore-completed-virt-launcher-pod branch 3 times, most recently from 5d03941 to bbe0491 Compare November 3, 2023 12:43
When a live migration fails at completed virt-launcher pod with newer
creation timestamp is lingering in the system so the virt-launcher that
is really running the VM is hidden by it and routes are wrongly
reconstructed. This change take into account if the pod is completed to
ignore it marking it as stale.

Signed-off-by: Enrique Llorente <ellorent@redhat.com>
@qinqon qinqon force-pushed the kubevirt-ignore-completed-virt-launcher-pod branch from bbe0491 to 4f9b4c2 Compare November 3, 2023 13:03
@jcaamano jcaamano merged commit 469c4e8 into ovn-org:master Nov 3, 2023
28 of 29 checks passed
@openshift-merge-robot
Copy link

Fix included in accepted release 4.15.0-0.nightly-2023-11-16-214550

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants