[virt-operator] cope with misscheduled virt-handler pods

Applying a custom `NoSchedule` taint to a nod will flip the `Available`
condition of KubeVirt to `False`:

```
  - lastProbeTime: "2023-08-07T14:15:43Z"
    lastTransitionTime: "2023-08-07T14:15:43Z"
    message: Deploying version devel with registry registry:5000/kubevirt
    reason: DeploymentInProgress
    status: "False"
    type: Available
```

This will only resolve back to `True` if the pod gets manually evicted,
or if the daemonset gets updated.

The background is, that we will see more up-to-date virt-handler pods in
ready state than we actually want.

Relax the readiness check slightly by counting misscheduled but
up-to-date and ready virt-handlers as something which does not trigger a
`DeploymentInProgress` state.

Signed-off-by: Roman Mohr <rmohr@google.com>

pkg/virt-operator/util/readycheck.go

@@ -71,11 +71,13 @@ func DaemonsetIsReady(kv *v1.KubeVirt, daemonset *appsv1.DaemonSet, stores Store
 	}
 
 	if podsReady == 0 {
-		log.Log.Infof("DaemonSet %v not ready yet. Waiting on at least one ready pod", daemonset.Name)
+		log.Log.Infof("DaemonSet %v not ready yet. Waiting for all pods to be ready", daemonset.Name)
 		return false
 	}
 
-	return podsReady == daemonset.Status.DesiredNumberScheduled
+	// Misscheduled but up to date daemonset pods will not be eviceted unless manually deleted or the daemonset gets updated.
+	// Don't force the Available condition to false or block the upgrade on this.
+	return podsReady == daemonset.Status.DesiredNumberScheduled+daemonset.Status.NumberMisscheduled
 }
 
 func DeploymentIsReady(kv *v1.KubeVirt, deployment *appsv1.Deployment, stores Stores) bool {