Race condition between rollback checks and Node state

[qinqon]

What happened:
When applying a policy that deals with primary nic and to test rollback policy was a success state but enacment was progressing and ended with failure (sice a rollback was expected) the issue is related to node counting for policy conditions, since only ready nodes are being counted and playing with primary nic can render one node temporally at NotReady, the comparation between number of nodes and not matching enactments was passing.

To fix this we have to add another probe after apply and after rollback to check that Node is at Ready state so we block there until node is ok again.

What you expected to happen:
Policy to be a Failure state after a rollback from a bad primary nic change.

How to reproduce it (as minimally and precisely as possible):
Apply a bad primary nic policy at multinic env, it has to be exercise multiple time until race appear.

Anything else we need to know?:

Environment:

• NodeNetworkState on affected nodes (use kubectl get nodenetworkstate <node_name> -o yaml):
• Problematic NodeNetworkConfigurationPolicy:
• kubernetes-nmstate image (use kubectl get pods --all-namespaces -l app=kubernetes-nmstate -o jsonpath='{.items[0].spec.containers[0].image}'):
• NetworkManager version (use nmcli --version)
• Kubernetes version (use kubectl version):
• OS (e.g. from /etc/os-release):
• Others: