Fixes multiple NICs OVN/OVS config

Deploying with multiple NICs will end up placing all NICs on the same
default NM connection. If one of these interfaces fails to DHCP or does
not DHCP in time during boot, NM-wait-online will fail (even though 1
NIC may have DHCP'ed correctly). Therefore the ovs-configuration service
should not require NM-wait-online, but we should wait until it is
completed and we do want it. However, if it fails we still try to to
check the system if a default gateway is present.

Additionally, since multiple NICs may share the same NM connection, we
cannot always just bring it down. Instead, change the behavior to just
disconnect the NIC and then bring up the higher priority connection
later to preserve the default NM connection.

Signed-off-by: Tim Rozet <trozet@redhat.com>

templates/common/_base/files/configure-ovs-network.yaml

@@ -77,13 +77,15 @@ contents:
     fi
 
     # bring down any old iface
-    nmcli conn down $old_conn
+    nmcli device disconnect $iface
 
     if ! nmcli connection show ovs-if-phys0 &> /dev/null; then
       nmcli c add type 802-3-ethernet conn.interface ${iface} master ovs-port-phys0 con-name ovs-if-phys0 \
         connection.autoconnect-priority 100 802-3-ethernet.mtu ${iface_mtu}
     fi
 
+    nmcli conn up ovs-if-phys0
+
     if ! nmcli connection show ovs-if-br-ex &> /dev/null; then
       nmcli c add type ovs-interface slave-type ovs-port conn.interface br-ex master ovs-port-br-ex con-name \
         ovs-if-br-ex 802-3-ethernet.mtu ${iface_mtu} 802-3-ethernet.cloned-mac-address ${iface_mac}

templates/common/_base/units/ovs-configuration.service.yaml

@@ -4,7 +4,8 @@ contents: |
   [Unit]
   Description=Configures OVS with proper host networking configuration
   # This service is used to move a physical NIC into OVS and reconfigure OVS to use the host IP
-  Requires=NetworkManager-wait-online.service openvswitch.service
+  Requires=openvswitch.service
+  Wants=NetworkManager-wait-online.service
   After=NetworkManager-wait-online.service openvswitch.service node-valid-hostname.service
   Before=network-online.target kubelet.service crio.service