From f7900ce5873483e9dd056c5dd90414e908469df8 Mon Sep 17 00:00:00 2001 From: Steven Smith Date: Wed, 29 May 2024 14:39:25 -0400 Subject: [PATCH] Adds live migration procedures for sdn to ovnk Updates limited live migration procedure to find various resources Multicast enabled namespaces change Further changes --- ...ter-initiating-limited-live-migration.adoc | 223 ++++++++++++++++++ ...ore-initiating-limited-live-migration.adoc | 136 +++++++++++ .../how-the-live-migration-process-works.adoc | 54 +++++ .../initiating-limited-live-migration.adoc | 61 +++++ .../live-migration-metrics-information.adoc | 54 +++++ .../nw-network-plugin-migration-process.adoc | 53 +---- ...netes-checking-live-migration-metrics.adoc | 75 ++++++ ...w-ovn-kubernetes-live-migration-about.adoc | 80 +++++++ modules/nw-ovn-kubernetes-live-migration.adoc | 9 + .../nw-ovn-kubernetes-migration-about.adoc | 49 ++-- modules/nw-ovn-kubernetes-migration.adoc | 2 +- modules/nw-ovn-kubernetes-rollback-live.adoc | 88 +++++++ modules/patching-ovnk-address-ranges.adoc | 43 ++++ ...ore-initiating-limited-live-migration.adoc | 52 ++++ .../migrate-from-openshift-sdn.adoc | 54 ++++- .../rollback-to-openshift-sdn.adoc | 11 +- 16 files changed, 963 insertions(+), 81 deletions(-) create mode 100644 modules/checking-cluster-resources-after-initiating-limited-live-migration.adoc create mode 100644 modules/checking-cluster-resources-before-initiating-limited-live-migration.adoc create mode 100644 modules/how-the-live-migration-process-works.adoc create mode 100644 modules/initiating-limited-live-migration.adoc create mode 100644 modules/live-migration-metrics-information.adoc create mode 100644 modules/nw-ovn-kubernetes-checking-live-migration-metrics.adoc create mode 100644 modules/nw-ovn-kubernetes-live-migration-about.adoc create mode 100644 modules/nw-ovn-kubernetes-live-migration.adoc create mode 100644 modules/nw-ovn-kubernetes-rollback-live.adoc create mode 100644 modules/patching-ovnk-address-ranges.adoc create mode 100644 modules/removing-egress-router-pods-before-initiating-limited-live-migration.adoc diff --git a/modules/checking-cluster-resources-after-initiating-limited-live-migration.adoc b/modules/checking-cluster-resources-after-initiating-limited-live-migration.adoc new file mode 100644 index 000000000000..3dfedb894b17 --- /dev/null +++ b/modules/checking-cluster-resources-after-initiating-limited-live-migration.adoc @@ -0,0 +1,223 @@ +// Module included in the following assemblies: +// +// * networking/ovn_kubernetes_network_provider/migrate-from-openshift-sdn.adoc + +:_mod-docs-content-type: PROCEDURE +[id="checking-cluster-resources-after-initiating-limited-live-migration_{context}"] += Checking cluster resources after initiating the limited live migration + +The following procedure shows you how to check for egress IP resources, egress firewall resources, multicast enabled namespaces, and network policies when your deploying is using OVN-Kubernetes. If you had these resources on OpenShift SDN, you should check them after migration to ensure that they are working properly. + +.Prerequisites + +* You have access to the cluster as a user with the `cluster-admin` role. +* You have successfully migrated from OpenShift SDN to OVN-Kubernetes by using the limited live migration. + +.Procedure + +. As an {product-title} cluster administrator, check for egress firewall resources. You can do this by using the `oc` CLI, or by using the {product-title} web console. + +.. To check for egress firewall resource by using the `oc` CLI tool: + +... To check for egress firewall resources, enter the following command: ++ +[source,terminal] +---- +$ oc get egressfirewalls.k8s.ovn.org -A +---- ++ +.Example output ++ +[source,terminal] +---- +NAMESPACE NAME AGE + 5d +---- + +... You can check the intended behavior of an egress firewall resource by using the `-o yaml` flag. For example: ++ +[source,terminal] +---- +$ oc get egressfirewall -n -o yaml +---- ++ +.Example output ++ +[source,terminal] +---- +apiVersion: k8s.ovn.org/v1 +kind: EgressFirewall +metadata: + name: + namespace: +spec: + egress: + - type: Allow + to: + cidrSelector: 192.168.0.0/16 + - type: Deny + to: + cidrSelector: 0.0.0.0/0 +---- ++ +Ensure that the behavior of this resource is intended because it could have changed after migration. For more information about egress firewalls, see "Configuring an egress firewall for a project". + +.. To check for egress firewall resources by using the {product-title} web console: + +... On the {product-title} web console, click *Observe* -> *Metrics*. + +... In the *Expression* box, type `ovnkube_controller_num_egress_firewall_rules` and click *Run queries*. If you have egress firewall resources, they are returned in the *Expression* box. + +. Check your cluster for egress IP resources. You can do this by using the `oc` CLI, or by using the {product-title} web console. + +.. To check for egress IPs by using the `oc` CLI tool: + +... To list the namespace with egress IP resources, enter the following command: ++ +[source,terminal] +---- +$ oc get egressip +---- ++ +.Example output ++ +[source,terminal] +---- +NAME EGRESSIPS ASSIGNED NODE ASSIGNED EGRESSIPS +egress-sample 192.0.2.10 ip-10-0-42-79.us-east-2.compute.internal 192.0.2.10 +egressip-sample-2 192.0.2.14 ip-10-0-42-79.us-east-2.compute.internal 192.0.2.14 +---- + +... To provide detailed information about an egress IP, enter the following command: ++ +[source,terminal] +---- +$ oc get egressip -o yaml +---- ++ +.Example output ++ +[source,terminal] +---- +apiVersion: k8s.ovn.org/v1 +kind: EgressIP +metadata: + annotations: + kubectl.kubernetes.io/last-applied-configuration: | + {"apiVersion":"k8s.ovn.org/v1","kind":"EgressIP","metadata":{"annotations":{},"name":"egressip-sample"},"spec":{"egressIPs":["192.0.2.12","192.0.2.13"],"namespaceSelector":{"matchLabels":{"name":"my-namespace"}}}} + creationTimestamp: "2024-06-27T15:48:36Z" + generation: 7 + name: egressip-sample + resourceVersion: "125511578" + uid: b65833c8-781f-4cc9-bc96-d970259a7631 +spec: + egressIPs: + - 192.0.2.12 + - 192.0.2.13 + namespaceSelector: + matchLabels: + name: my-namespace +---- ++ +Repeat this for all egress IPs. Ensure that the behavior of each resource is intended because it could have changed after migration. For more information about EgressIPs, see "Configuring an EgressIP address". + +.. To check for egress IPs by using the {product-title} web console: + +... On the {product-title} web console, click *Observe* -> *Metrics*. + +... In the *Expression* box, type `ovnkube_clustermanager_num_egress_ips` and click *Run queries*. If you have egress firewall resources, they are returned in the *Expression* box. + +. Check your cluster for multicast enabled namespaces. You can only do this by using the `oc` CLI. + +.. To locate namespaces with multicast enabled, enter the following command: ++ +[source,terminal] +---- +$ oc get namespace -o json | jq -r '.items[] | select(.metadata.annotations."k8s.ovn.org/multicast-enabled" == "true") | .metadata.name' +---- ++ +.Example output ++ +[source,terminal] +---- +namespace1 +namespace3 +---- + +.. To describe each multicast enabled namespace, enter the following command: ++ +[source,terminal] +---- +$ oc describe namespace +---- ++ +.Example output ++ +[source,terminal] +---- +Name: my-namespace +Labels: kubernetes.io/metadata.name=my-namespace + pod-security.kubernetes.io/audit=restricted + pod-security.kubernetes.io/audit-version=v1.24 + pod-security.kubernetes.io/warn=restricted + pod-security.kubernetes.io/warn-version=v1.24 +Annotations: k8s.ovn.org/multicast-enabled: true + openshift.io/sa.scc.mcs: s0:c25,c0 + openshift.io/sa.scc.supplemental-groups: 1000600000/10000 + openshift.io/sa.scc.uid-range: 1000600000/10000 +Status: Active +---- ++ +Ensure that multicast functionality is correctly configured and working as expected in each namespace. For more information, see "Enabling multicast for a project". + +. Check your cluster's network policies. You can only do this by using the `oc` CLI. + +.. To obtain information about network policies within a namespace, enter the following command: ++ +[source,terminal] +---- +$ oc get networkpolicy -n +---- ++ +.Example output ++ +[source,terminal] +---- +NAME POD-SELECTOR AGE +allow-multicast app=my-app 11m +---- + +.. To provide detailed information about the network policy, enter the following command: ++ +[source,terminal] +---- +$ oc describe networkpolicy allow-multicast -n +---- ++ +.Example output ++ +[source,terminal] +---- +Name: allow-multicast +Namespace: my-namespace +Created on: 2024-07-24 14:55:03 -0400 EDT +Labels: +Annotations: +Spec: + PodSelector: app=my-app + Allowing ingress traffic: + To Port: (traffic allowed to all ports) + From: + IPBlock: + CIDR: 224.0.0.0/4 + Except: + Allowing egress traffic: + To Port: (traffic allowed to all ports) + To: + IPBlock: + CIDR: 224.0.0.0/4 + Except: + Policy Types: Ingress, Egress +---- ++ +Ensure that the behavior of the network policy is as intended. Optimization for network policies differ between SDN and OVN-K, so users might need to adjust their policies to achieve optimal performance for different CNIs. For more information, see "About network policy". \ No newline at end of file diff --git a/modules/checking-cluster-resources-before-initiating-limited-live-migration.adoc b/modules/checking-cluster-resources-before-initiating-limited-live-migration.adoc new file mode 100644 index 000000000000..9b8618390b62 --- /dev/null +++ b/modules/checking-cluster-resources-before-initiating-limited-live-migration.adoc @@ -0,0 +1,136 @@ +// Module included in the following assemblies: +// +// * networking/ovn_kubernetes_network_provider/migrate-from-openshift-sdn.adoc + +:_mod-docs-content-type: PROCEDURE +[id="checking-cluster-resources-before-initiating-limited-live-migration_{context}"] += Checking cluster resources before initiating the limited live migration + +Before migrating to OVN-Kubernetes by using the limited live migration, you should check for egress IP resources, egress firewall resources, and multicast-enabled namespaces on your OpenShift SDN deployment. You should also review any network policies in your deployment. If you find that your cluster has these resources before migration, you should check their behavior after migration to ensure that they are working as intended. + +The following procedure shows you how to check for egress IP resources, egress firewall resources, multicast-enabled namespaces, network policies, and an NNCP. No action is necessary after checking for these resources. + +.Prerequisites + +* You have access to the cluster as a user with the `cluster-admin` role. + +.Procedure + +. As an {product-title} cluster administrator, check for egress firewall resources. You can do this by using the `oc` CLI, or by using the {product-title} web console. + +.. To check for egress firewall resource by using the `oc` CLI tool: + +... To check for egress firewall resources, enter the following command: ++ +[source,terminal] +---- +$ oc get egressnetworkpolicies.network.openshift.io -A +---- ++ +.Example output ++ +[source,terminal] +---- +NAMESPACE NAME AGE + 5d +---- + +... You can check the intended behavior of an egress firewall resource by using the `-o yaml` flag. For example: ++ +[source,terminal] +---- +$ oc get egressnetworkpolicy -n -o yaml +---- ++ +.Example output ++ +[source,terminal] +---- +apiVersion: network.openshift.io/v1 +kind: EgressNetworkPolicy +metadata: + name: + namespace: +spec: + egress: + - type: Allow + to: + cidrSelector: 0.0.0.0/0 + - type: Deny + to: + cidrSelector: 10.0.0.0/8 +---- + +.. To check for egress firewall resources by using the {product-title} web console: + +... On the {product-title} web console, click *Observe* -> *Metrics*. + +... In the *Expression* box, type `sdn_controller_num_egress_firewalls` and click *Run queries*. If you have egress firewall resources, they are returned in the *Expression* box. + +. Check your cluster for egress IP resources. You can do this by using the `oc` CLI, or by using the {product-title} web console. + +.. To check for egress IPs by using the `oc` CLI tool: + +... To list namespaces with egress IP resources, enter the following command ++ +[source,terminal] +---- +$ oc get netnamespace -A | awk '$3 != ""' +---- ++ +.Example output ++ +[source,terminal] +---- +NAME NETID EGRESS IPS +namespace1 14173093 ["10.0.158.173"] +namespace2 14173020 ["10.0.158.173"] +---- + +.. To check for egress IPs by using the {product-title} web console: + +... On the {product-title} web console, click *Observe* -> *Metrics*. + +... In the *Expression* box, type `sdn_controller_num_egress_ips` and click *Run queries*. If you have egress firewall resources, they are returned in the *Expression* box. + +. Check your cluster for multicast enabled namespaces. You can do this by using the `oc` CLI, or by using the {product-title} web console. + +.. To check for multicast enabled namespaces by using the `oc` CLI tool: + +... To locate namespaces with multicast enabled, enter the following command: ++ +[source,terminal] +---- +$ oc get netnamespace -o json | jq -r '.items[] | select(.metadata.annotations."netnamespace.network.openshift.io/multicast-enabled" == "true") | .metadata.name' +---- ++ +.Example output ++ +[source,terminal] +---- +namespace1 +namespace3 +---- + +.. To check for multicast enabled namespaces by using the {product-title} web console: + +... On the {product-title} web console, click *Observe* -> *Metrics*. + +... In the *Expression* box, type `sdn_controller_num_multicast_enabled_namespaces` and click *Run queries*. If you have multicast enabled namespaces, they are returned in the *Expression* box. + +. Check your cluster for any network policies. You can do this by using the `oc` CLI. + +.. To check for network policies by using the `oc` CLI tool, enter the following command: ++ +[source,terminal] +---- +$ oc get networkpolicy -n +---- ++ +.Example output ++ +[source,terminal] +---- +NAME POD-SELECTOR AGE +allow-multicast app=my-app 11m +---- \ No newline at end of file diff --git a/modules/how-the-live-migration-process-works.adoc b/modules/how-the-live-migration-process-works.adoc new file mode 100644 index 000000000000..3fd6101019b1 --- /dev/null +++ b/modules/how-the-live-migration-process-works.adoc @@ -0,0 +1,54 @@ +// Module included in the following assemblies: +// +// * networking/ovn_kubernetes_network_provider/migrate-from-openshift-sdn.adoc + +ifeval::["{context}" == "migrate-to-openshift-sdn"] +:sdn: OpenShift SDN +:previous-sdn: OVN-Kubernetes +:type: OpenShiftSDN +endif::[] +ifeval::["{context}" == "migrate-from-openshift-sdn"] +:sdn: OVN-Kubernetes +:previous-sdn: OpenShift SDN +:type: OVNKubernetes +endif::[] + +[id="how-the-live-migration-process-works_{context}"] += How the limited live migration process works + +The following table summarizes the limited live migration process by segmenting between the user-initiated steps in the process and the actions that the migration script performs in response. + +.Limited live migration to OVNKubernetes from OpenShiftSDN +[cols="1,1a",options="header"] +|=== +|User-initiated steps|Migration activity +ifdef::openshift-rosa,openshift-dedicated[] +| Add the `unsupported-red-hat-internal-testing` annotation to the cluster-level network configuration. +| The Cluster Network Operator (CNO) acknowledges the unsupported testing environment. +endif::[] + +| Patch the cluster-level networking configuration by changing the `networkType` from `OpenShiftSDN` to `OVNKubernetes`. +| +Cluster Network Operator (CNO):: ++ +-- +* Sets migration related fields in the `network.operator` custom resource (CR) and waits for routable MTUs to be applied to all nodes. +* Patches the `network.operator` CR to set the migration mode to `Live` for OVN-Kubernetes and deploys the OpenShift SDN network plugin in migration mode. +* Deploys OVN-Kubernetes with hybrid overlay enabled, ensuring that no racing conditions occur. +* Waits for the OVN-Kubernetes deployment and updates the conditions in the status of the `network.config` CR. +* Triggers the Machine Config Operator (MCO) to apply the new machine config to each machine config pool, which includes node cordoning, draining, and rebooting. +* OVN-Kubernetes adds nodes to the appropriate zones and recreates pods using OVN-Kubernetes as the default CNI plugin. +* Removes migration-related fields from the network.operator CR and performs cleanup actions, such as deleting OpenShift SDN resources and redeploying OVN-Kubernetes in normal mode with the necessary configurations. +* Waits for the OVN-Kubernetes redeployment and updates the status conditions in the `network.config` CR to indicate migration completion. If your migration is blocked, see "Checking limited live migration metrics" for information on troubleshooting the issue. +-- +|=== + +ifdef::sdn[] +:!sdn: +endif::[] +ifdef::previous-sdn[] +:!previous-sdn: +endif::[] +ifdef::type[] +:!type: +endif::[] \ No newline at end of file diff --git a/modules/initiating-limited-live-migration.adoc b/modules/initiating-limited-live-migration.adoc new file mode 100644 index 000000000000..081beecf08aa --- /dev/null +++ b/modules/initiating-limited-live-migration.adoc @@ -0,0 +1,61 @@ +// Module included in the following assemblies: +// +// * networking/ovn_kubernetes_network_provider/migrate-from-openshift-sdn.adoc + +:_mod-docs-content-type: PROCEDURE +[id="initiating-limited-live-migration_{context}"] += Initiating the limited live migration process + +After you have checked the behavior of egress IP resources, egress firewall resources, and multicast enabled namespaces, and removed any egress router resources, you can initiate the limited live migration process. + +.Prerequisites + +* A cluster has been configured with the OpenShift SDN CNI network plugin in the network policy isolation mode. +* You have installed the OpenShift CLI (`oc`). +* You have access to the cluster as a user with the `cluster-admin` role. +* You have created a recent backup of the etcd database. +* The cluster is in a known good state without any errors. +* Before migration to OVN-Kubernetes, a security group rule must be in place to allow UDP packets on port `6081` for all nodes on all cloud platforms. +* If the `100.64.0.0/16` and `100.88.0.0/16` address ranges were previously in use by OpenShift-SDN, you have patched them. The first step of this procedure checks whether these address ranges are in use. If they are in use, see "Patching OVN-Kubernetes address ranges". +* You have checked for egress IP resources, egress firewall resources, and multicast enabled namespaces. +* You have removed any egress router pods before beginning the limited live migration. For more information about egress router pods, see "Deploying an egress router pod in redirect mode". +* You have reviewed the "Considerations for limited live migration to the OVN-Kubernetes network plugin" section of this document. + +.Procedure + +ifdef::openshift-rosa,openshift-dedicated[] +. To add the `unsupported-red-hat-internal-testing` annotation to the cluster-level network configuration, enter the following command: ++ +[source,terminal] +---- +$ oc patch Network.config.openshift.io cluster --type='merge' --patch '{"metadata":{"annotations":{"unsupported-red-hat-internal-testing": "true"}}}' +---- +endif::[] + +. To patch the cluster-level networking configuration and initiate the migration from OpenShift SDN to OVN-Kubernetes, enter the following command: ++ +[source,terminal] +---- +$ oc patch Network.config.openshift.io cluster --type='merge' --patch '{"metadata":{"annotations":{"network.openshift.io/network-type-migration":""}},"spec":{"networkType":"OVNKubernetes"}}' +---- ++ +After running this command, the migration process begins. During this process, the Machine Config Operator reboots the nodes in your cluster twice. The migration takes approximately twice as long as a cluster upgrade. ++ +[IMPORTANT] +==== +This `oc patch` command checks for overlapping CIDRs in use by OpenShift SDN. If overlapping CIDRs are detected, you must patch them before the limited live migration process can start. For more information, see "Patching OVN-Kubernetes address ranges". +==== + +. Optional: To ensure that the migration process has completed, and to check the status of the `network.config`, you can enter the following commands: ++ +[source,terminal] +---- +$ oc get network.config.openshift.io cluster -o jsonpath='{.status.networkType}' +---- ++ +[source,terminal] +---- +$ oc get network.config cluster -o=jsonpath='{.status.conditions}' | jq . +---- ++ +You can check limited live migration metrics for troubleshooting issues. For more information, see "Checking limited live migration metrics". \ No newline at end of file diff --git a/modules/live-migration-metrics-information.adoc b/modules/live-migration-metrics-information.adoc new file mode 100644 index 000000000000..26bb26bc040e --- /dev/null +++ b/modules/live-migration-metrics-information.adoc @@ -0,0 +1,54 @@ +// Module included in the following assemblies: +// +// * networking/ovn_kubernetes_network_provider/migrate-from-openshift-sdn.adoc + +:_mod-docs-content-type: PROCEDURE +[id="live-migration-metrics-information"] += Information about limited live migration metrics + +The following table shows you the available metrics and the label values populated from the `openshift_network_operator_live_migration_procedure` expression. Use this information to monitor progress or to troubleshoot the migration. + + +.Limited live migration metrics +[cols="1a,1a",options="header"] +|=== +| Metric | Label values +| +*`openshift_network_operator_live_migration_blocked:`*:: ++ +-- +A Prometheus gauge vector metric. A metric that contains a constant `1` value labeled with the reason that the CNI limited live migration might not have started. This metric is available when the CNI limited live migration has started by annotating the `Network` custom resource. + +This metric is not published unless the limited live migration is blocked. +-- +| +The list of label values includes the following:: ++ +-- +* `UnsupportedCNI`: Unable to migrate to the unsupported target CNI. Valid CNI is `OVNKubernetes` when migrating from OpenShift SDN. +* `UnsupportedHyperShiftCluster`: Limited live migration is unsupported within an HCP cluster. +* `UnsupportedSDNNetworkIsolationMode`: OpenShift SDN is configured with an unsupported network isolation mode `Multitenant`. Migrate to a supported network isolation mode before performing limited live migration. +* `UnsupportedMACVLANInterface`: Remove the egress router or any pods which contain the pod annotation `pod.network.openshift.io/assign-macvlan`. +Find the offending pod's namespace or pod name with the following command: + + + +`oc get pods -Ao=jsonpath='{range .items[?(@.metadata.annotations.pod\.network\.openshift\.io/assign-macvlan=="")]}{@.metadata.namespace}{"\t"}{@.metadata.name}{"\n"}'`. +-- + +| +*`openshift_network_operator_live_migration_condition:`*:: ++ +-- +A metric which represents the status of each condition type for the CNI limited live migration. The set of status condition types is defined for `network.config` to support observability of the CNI limited live migration. + +A `1` value represents condition status `true`. A `0` value represents `false`. `-1` represents unknown. This metric is available when the CNI limited live migration has started by annotating the `Network` custom resource (CR). + +This metric is only available when the limited live migration has been triggered by adding the relevant annotation to the `Network` CR cluster, otherwise, it is not published. If the following condition types are not present within the Network CR cluster, the metric and their labels are cleared. +-- +| +The list of label values includes the following:: ++ +-- +* `NetworkTypeMigrationInProgress` +* `NetworkTypeMigrationTargetCNIAvailable` +* `NetworkTypeMigrationTargetCNIInUse` +* `NetworkTypeMigrationOriginalCNIPurged` +* `NetworkTypeMigrationMTUReady` +-- +|=== \ No newline at end of file diff --git a/modules/nw-network-plugin-migration-process.adoc b/modules/nw-network-plugin-migration-process.adoc index c69d1123a464..8610d06915e3 100644 --- a/modules/nw-network-plugin-migration-process.adoc +++ b/modules/nw-network-plugin-migration-process.adoc @@ -15,11 +15,11 @@ ifeval::["{context}" == "migrate-from-openshift-sdn"] endif::[] [id="how-the-migration-process-works_{context}"] -= How the migration process works += How the offline migration process works The following table summarizes the migration process by segmenting between the user-initiated steps in the process and the actions that the migration performs in response. -.Migrating to {sdn} from {previous-sdn} +.Offline migration to {sdn} from {previous-sdn} [cols="1,1a",options="header"] |=== @@ -38,7 +38,7 @@ CNO:: Performs the following actions: -- * Destroys the {previous-sdn} control plane pods. * Deploys the {sdn} control plane pods. -* Updates the Multus objects to reflect the new network plugin. +* Updates the Multus daemon sets and config map objects to reflect the new network plugin. -- | @@ -48,51 +48,6 @@ Cluster:: As nodes reboot, the cluster assigns IP addresses to pods on the {sdn} |=== -ifeval::["{context}" == "migrate-from-openshift-sdn"] -If a rollback to OpenShift SDN is required, the following table describes the process. - -[IMPORTANT] -==== -You must wait until the migration process from OpenShift SDN to OVN-Kubernetes network plugin is successful before initiating a rollback. -==== - -.Performing a rollback to OpenShift SDN -[cols="1,1a",options="header"] -|=== - -|User-initiated steps|Migration activity - -|Suspend the MCO to ensure that it does not interrupt the migration. -|The MCO stops. - -| -Set the `migration` field of the `Network.operator.openshift.io` custom resource (CR) named `cluster` to `OpenShiftSDN`. Make sure the `migration` field is `null` before setting it to a value. -| -CNO:: Updates the status of the `Network.config.openshift.io` CR named `cluster` accordingly. - -|Update the `networkType` field. -| -CNO:: Performs the following actions: -+ --- -* Destroys the OVN-Kubernetes control plane pods. -* Deploys the OpenShift SDN control plane pods. -* Updates the Multus objects to reflect the new network plugin. --- - -| -Reboot each node in the cluster. -| -Cluster:: As nodes reboot, the cluster assigns IP addresses to pods on the OpenShift-SDN network. - -| -Enable the MCO after all nodes in the cluster reboot. -| -MCO:: Rolls out an update to the systemd configuration necessary for OpenShift SDN; the MCO updates a single machine per pool at a time by default, so the total time the migration takes increases with the size of the cluster. - -|=== -endif::[] - ifdef::sdn[] :!sdn: endif::[] @@ -101,4 +56,4 @@ ifdef::previous-sdn[] endif::[] ifdef::type[] :!type: -endif::[] +endif::[] \ No newline at end of file diff --git a/modules/nw-ovn-kubernetes-checking-live-migration-metrics.adoc b/modules/nw-ovn-kubernetes-checking-live-migration-metrics.adoc new file mode 100644 index 000000000000..eb9425738107 --- /dev/null +++ b/modules/nw-ovn-kubernetes-checking-live-migration-metrics.adoc @@ -0,0 +1,75 @@ +// Module included in the following assemblies: +// +// * networking/ovn_kubernetes_network_provider/migrate-from-openshift-sdn.adoc + +:_mod-docs-content-type: PROCEDURE +[id="checking-live-migration-metrics"] += Checking limited live migration metrics + +Metrics are available to monitor the progress of the limited live migration. Metrics can be viewed on the {product-title} web console, or by using the `oc` CLI. + +.Prerequisites + +* You have initiated a limited live migration to OVN-Kubernetes. + +.Procedure + +. To view limited live migration metrics on the {product-title} web console: + +.. Click *Observe* -> *Metrics*. + +.. In the *Expression* box, type *openshift_network* and click the *openshift_network_operator_live_migration_procedure* option. + +. To view metrics by using the `oc` CLI: + +.. Enter the following command to generate a token for the `prometheus-k8s` service account in the `openshift-monitoring` namespace: ++ +[source,terminal] +---- +$ oc create token prometheus-k8s -n openshift-monitoring +---- ++ +.Example output ++ +[source,terminal] +---- +eyJhbGciOiJSUzI1NiIsImtpZCI6IlZiSUtwclcwbEJ2VW9We... +---- + +.. Enter the following command to request information about the `openshift_network_operator_live_migration_condition` metric: ++ +[source,terminal] +---- +$ oc -n openshift-monitoring exec -c prometheus prometheus-k8s-0 -- curl -k -H "Authorization: " "https://" --data-urlencode "query=openshift_network_operator_live_migration_condition" | jq +---- ++ +.Example output ++ +[source,terminal] +---- + "status": "success", + "data": { + "resultType": "vector", + "result": [ + { + "metric": { + "__name__": "openshift_network_operator_live_migration_condition", + "container": "network-operator", + "endpoint": "metrics", + "instance": "10.0.83.62:9104", + "job": "metrics", + "namespace": "openshift-network-operator", + "pod": "network-operator-6c87754bc6-c8qld", + "prometheus": "openshift-monitoring/k8s", + "service": "metrics", + "type": "NetworkTypeMigrationInProgress" + }, + "value": [ + 1717653579.587, + "1" + ] + }, +... +---- + +The table in "Information about limited live migration metrics" shows you the available metrics and the label values populated from the `openshift_network_operator_live_migration_procedure` expression. Use this information to monitor progress or to troubleshoot the migration. \ No newline at end of file diff --git a/modules/nw-ovn-kubernetes-live-migration-about.adoc b/modules/nw-ovn-kubernetes-live-migration-about.adoc new file mode 100644 index 000000000000..c461a78fbddb --- /dev/null +++ b/modules/nw-ovn-kubernetes-live-migration-about.adoc @@ -0,0 +1,80 @@ +// Module included in the following assemblies: +// +// * networking/ovn_kubernetes_network_provider/migrate-from-openshift-sdn.adoc + +[id="nw-ovn-kubernetes-live-migration-about_{context}"] += Limited live migration to the OVN-Kubernetes network plugin overview + +The limited live migration method is the process in which the OpenShift SDN network plugin and its network configurations, connections, and associated resources, are migrated to the OVN-Kubernetes network plugin without service interruption. It is available for {product-title}. +// Azure Red Hat OpenShift deployment types will needed added in .z stream. +It is not available for hosted control plane deployment types. This migration method is valuable for deployment types that require constant service availability and offers the following benefits: + +* Continuous service availability +* Minimized downtime +* Automatic node rebooting +* Seamless transition from the OpenShift SDN network plugin to the OVN-Kubernetes network plugin + +Although a rollback procedure is provided, the limited live migration is intended to be a one-way process. + +include::snippets/sdn-deprecation-statement.adoc[] + +The following sections provide more information about the limited live migration method. + +[id="supported-platforms-live-migrating-ovn-kubernetes"] +== Supported platforms when using the limited live migration method + +The following table provides information about the supported platforms for the limited live migration type. + +.Supported platforms for the limited live migration method +[cols="1,1", options="header"] +|=== +| Platform | Limited Live Migration + +| Bare metal hardware (IPI and UPI) |✓ +| Amazon Web Services (AWS) (IPI and UPI) |✓ +| Google Cloud Platform (GCP) (IPI and UPI) |✓ +| {ibm-cloud-name} (IPI and UPI) |✓ +| Microsoft Azure (IPI and UPI) |✓ +| {rh-openstack-first} (IPI and UPI) |✓ +| VMware vSphere (IPI and UPI) |✓ +| AliCloud (UPI) |✓ +| Nutanix (IPI and UPI) |✓ +|=== + +[id="considerations-live-migrating-ovn-kubernetes-network-provider_{context}"] +== Considerations for limited live migration to the OVN-Kubernetes network plugin + +Before using the limited live migration method to the OVN-Kubernetes network plugin, cluster administrators should consider the following information: + +* The limited live migration procedure is unsupported for clusters with OpenShift SDN multitenant mode enabled. + +* Egress router pods block the limited live migration process. They must be removed before beginning the limited live migration process. + +* During the limited live migration, multicast, egress IP addresses, and egress firewalls are temporarily disabled. They can be migrated from OpenShift SDN to OVN-Kubernetes after the limited live migration process has finished. + +* The migration is intended to be a one-way process. However, for users that want to rollback to OpenShift-SDN, migration from OpenShift-SDN to OVN-Kubernetes must have succeeded. Users can follow the same procedure below to migrate to the OpenShift SDN network plugin from the OVN-Kubernetes network plugin. + +* The limited live migration is not supported on HyperShift clusters. + +* OpenShift SDN does not support IPsec. After the migration, cluster administrators can enable IPsec. + +* OpenShift SDN does not support IPv6. After the migration, cluster administrators can enable dual-stack. + +* The OpenShift SDN plugin allows application of the `NodeNetworkConfigurationPolicy` (NNCP) custom resource (CR) to the primary interface on a node. The OVN-Kubernetes network plugin does not have this capability. + +* The cluster MTU is the MTU value for pod interfaces. It is always less than your hardware MTU to account for the cluster network overlay overhead. The overhead is 100 bytes for OVN-Kubernetes and 50 bytes for OpenShift SDN. ++ +During the limited live migration, both OVN-Kubernetes and OpenShift SDN run in parallel. OVN-Kubernetes manages the cluster network of some nodes, while OpenShift SDN manages the cluster network of others. To ensure that cross-CNI traffic remains functional, the Cluster Network Operator updates the routable MTU to ensure that both CNIs share the same overlay MTU. As a result, after the migration has completed, the cluster MTU is 50 bytes less. + +* Some parameters of OVN-Kubernetes cannot be changed after installation. The following parameters can be set only before starting the limited live migration: + +** `InternalTransitSwitchSubnet` +** `internalJoinSubnet` + +* OVN-Kubernetes reserves the `100.64.0.0/16` and `100.88.0.0/16` IP address ranges. These subnets cannot be overlapped with any other internal or external network. If these IP addresses have been used by OpenShift SDN or any external networks that might communicate with this cluster, you must patch them to use a different IP address range before starting the limited live migration. See "Patching OVN-Kubernetes address ranges" for more information. + +* In most cases, the limited live migration is independent of the secondary interfaces of pods created by the Multus CNI plugin. However, if these secondary interfaces were set up on the default network interface controller (NIC) of the host, for example, using MACVLAN, IPVLAN, SR-IOV, or bridge interfaces with the default NIC as the control node, OVN-Kubernetes might encounter malfunctions. Users should remove such configurations before proceeding with the limited live migration. + +* When there are multiple NICs inside of the host, and the default route is not on the interface that has the Kubernetes NodeIP, you must use the offline migration instead. + +* All `DaemonSet` objects in the `openshift-sdn` namespace, which are not managed by the Cluster Network Operator (CNO), must be removed before initiating the limited live migration. These unmanaged daemon sets can cause the migration status to remain incomplete if not properly handled. \ No newline at end of file diff --git a/modules/nw-ovn-kubernetes-live-migration.adoc b/modules/nw-ovn-kubernetes-live-migration.adoc new file mode 100644 index 000000000000..95a60359b7bd --- /dev/null +++ b/modules/nw-ovn-kubernetes-live-migration.adoc @@ -0,0 +1,9 @@ +// Module included in the following assemblies: +// +// * networking/ovn_kubernetes_network_provider/migrate-from-openshift-sdn.adoc + +:_mod-docs-content-type: PROCEDURE +[id="nw-ovn-kubernetes-live-migration_{context}"] += Migrating to the OVN-Kubernetes network plugin by using the limited live migration method + +Migrating to the OVN-Kubernetes network plugin by using the limited live migration method is a multiple step process that requires users to check the behavior of egress IP resources, egress firewall resources, and multicast enabled namespaces. Administrators must also review any network policies in their deployment and remove egress router resources before initiating the limited live migration process. The following procedures should be used in succession. \ No newline at end of file diff --git a/modules/nw-ovn-kubernetes-migration-about.adoc b/modules/nw-ovn-kubernetes-migration-about.adoc index 9bc35cc62f42..54da0ba7f13f 100644 --- a/modules/nw-ovn-kubernetes-migration-about.adoc +++ b/modules/nw-ovn-kubernetes-migration-about.adoc @@ -3,38 +3,45 @@ // * networking/ovn_kubernetes_network_provider/migrate-from-openshift-sdn.adoc [id="nw-ovn-kubernetes-migration-about_{context}"] -= Migration to the OVN-Kubernetes network plugin += Offline migration to the OVN-Kubernetes network plugin overview -Migrating to the OVN-Kubernetes network plugin is a manual process that includes some downtime during which your cluster is unreachable. +The offline migration method is a manual process that includes some downtime, during which your cluster is unreachable. This method is primarily used for self-managed {product-title} deployments. -[IMPORTANT] -==== -Before you migrate your {product-title} cluster to use the OVN-Kubernetes network plugin, update your cluster to the latest z-stream release so that all the latest bug fixes apply to your cluster. -==== - -Although a rollback procedure is provided, the migration is intended to be a one-way process. +Although a rollback procedure is provided, the offline migration is intended to be a one-way process. +//// [IMPORTANT] ==== Migrating to or from the OVN-Kubernetes network plugin is not supported for managed OpenShift cloud services such as {product-dedicated}, Azure Red Hat OpenShift(ARO), and Red Hat OpenShift Service on AWS (ROSA). - -Migrating from OpenShift SDN network plugin to OVN-Kubernetes network plugin is not supported on Nutanix. ==== +//// +include::snippets/sdn-deprecation-statement.adoc[] -A migration to the OVN-Kubernetes network plugin is supported on the following platforms: +The following sections provide more information about the offline migration method. -* Bare-metal hardware -* {aws-first} -* {gcp-first} -* {ibm-cloud-name} -* {azure-first} -* {rh-openstack-first} -* {vmw-first} +[id="supported-platforms-offline-migrating-ovn-kubernetes"] +== Supported platforms when using the offline migration method -include::snippets/sdn-deprecation-statement.adoc[] +The following table provides information about the supported platforms for the offline migration type. + +.Supported platforms for the offline migration method +[cols="1,1", options="header"] +|=== +| Platform | Offline Migration + +| Bare metal hardware (IPI and UPI) |✓ +| Amazon Web Services (AWS) (IPI and UPI) |✓ +| Google Cloud Platform (GCP) (IPI and UPI) |✓ +| {ibm-cloud-name} (IPI and UPI) |✓ +| Microsoft Azure (IPI and UPI) |✓ +| {rh-openstack-first} (IPI and UPI) |✓ +| VMware vSphere (IPI and UPI) |✓ +| AliCloud (IPI and UPI) |✓ +| Nutanix (IPI and UPI) |✓ +|=== [id="considerations-migrating-ovn-kubernetes-network-provider_{context}"] -== Considerations for migrating to the OVN-Kubernetes network plugin +== Considerations for offline migration to the OVN-Kubernetes network plugin If you have more than 150 nodes in your {product-title} cluster, then open a support case for consultation on your migration to the OVN-Kubernetes network plugin. @@ -160,4 +167,4 @@ For more information on using multicast in OVN-Kubernetes, see "Enabling multica [id="network-policies_{context}"] === Network policies -OVN-Kubernetes fully supports the Kubernetes `NetworkPolicy` API in the `networking.k8s.io/v1` API group. No changes are necessary in your network policies when migrating from OpenShift SDN. +OVN-Kubernetes fully supports the Kubernetes `NetworkPolicy` API in the `networking.k8s.io/v1` API group. No changes are necessary in your network policies when migrating from OpenShift SDN. \ No newline at end of file diff --git a/modules/nw-ovn-kubernetes-migration.adoc b/modules/nw-ovn-kubernetes-migration.adoc index 73fffc33e161..139d7af9bf70 100644 --- a/modules/nw-ovn-kubernetes-migration.adoc +++ b/modules/nw-ovn-kubernetes-migration.adoc @@ -5,7 +5,7 @@ :_mod-docs-content-type: PROCEDURE [id="nw-ovn-kubernetes-migration_{context}"] -= Migrating to the OVN-Kubernetes network plugin += Migrating to the OVN-Kubernetes network plugin by using the offline migration method As a cluster administrator, you can change the network plugin for your cluster to OVN-Kubernetes. During the migration, you must reboot every node in your cluster. diff --git a/modules/nw-ovn-kubernetes-rollback-live.adoc b/modules/nw-ovn-kubernetes-rollback-live.adoc new file mode 100644 index 000000000000..ce0e9a4b5aee --- /dev/null +++ b/modules/nw-ovn-kubernetes-rollback-live.adoc @@ -0,0 +1,88 @@ +// Module included in the following assemblies: +// +// * networking/ovn_kubernetes_network_provider/rollback-to-openshift-sdn.adoc + +:_mod-docs-content-type: PROCEDURE +[id="nw-ovn-kubernetes-rollback-live_{context}"] += Using the limited live migration method to roll back to the OpenShift SDN network plugin + +As a cluster administrator, you can roll back to the OpenShift SDN Container Network Interface (CNI) network plugin by using the limited live migration method. During the migration with this method, nodes are automatically rebooted and service to the cluster is not interrupted. + +[IMPORTANT] +==== +You must wait until the migration process from OpenShift SDN to OVN-Kubernetes network plugin is successful before initiating a rollback. +==== + +If a rollback to OpenShift SDN is required, the following table describes the process. + +.Performing a rollback to OpenShift SDN +[cols="1,1a",options="header"] +|=== + +|User-initiated steps|Migration activity +ifdef::openshift-rosa,openshift-dedicated[] +| Add the `unsupported-red-hat-internal-testing` annotation to the cluster-level network configuration. +| The Cluster Network Operator (CNO) acknowledges the unsupported testing environment. +endif::[] + +| Patch the cluster-level networking configuration by changing the `networkType` from `OVNKubernetes` to `OpenShiftSDN`. +| +Cluster Network Operator (CNO):: Performs the following actions: ++ +-- +* Sets migration related fields in the `network.operator` custom resource (CR) and waits for routable MTUs to be applied to all nodes by the Machine Config Operator (MCO). +* Patches the `network.operator` CR to set the migration mode to `Live` for OpenShiftSDN and deploys the OpenShiftSDN network plugin in migration mode. +* Deploys OVN-Kubernetes with hybrid overlay enabled. +* Waits for both CNI plugins to be deployed and updates the conditions in the status of the `network.config` CR. +* Triggers the MCO to apply the new machine config to each machine config pool, which includes node cordoning, draining, and rebooting. +* Removes migration-related fields from the `network.operator` CR and performs cleanup actions, such as deleting OpenShift SDN resources and redeploying OVN-Kubernetes in normal mode with the necessary configurations. +* Waits for the OpenShiftSDN redeployment and updates the status conditions in the `network.config` CR to indicate migration completion. +-- + +|=== + +.Prerequisites + +* The {oc-first} is installed. +* Access to the cluster as a user with the cluster-admin role is available. +* The cluster is installed on infrastructure configured with the OVN-Kubernetes network plugin. +* A recent backup of the etcd database is available. +* A manual reboot can be triggered for each node. +* The cluster is in a known good state, without any errors. + +.Procedure + +. To initiate the rollback to OpenShift SDN, enter the following command: ++ +[source,terminal] +---- +$ oc patch Network.config.openshift.io cluster --type='merge' --patch '{"metadata":{"annotations":{"network.openshift.io/network-type-migration":""}},"spec":{"networkType":"OpenShiftSDN"}}' +---- + +. To watch the progress of your migration, enter the following command: ++ +[source,terminal] +---- +$ watch -n1 'oc get network.config/cluster -o json | jq ".status.conditions[]|\"\\(.type) \\(.status) \\(.reason) \\(.message)\"" -r | column --table --table-columns NAME,STATUS,REASON,MESSAGE --table-columns-limit 4; echo; oc get mcp -o wide; echo; oc get node -o "custom-columns=NAME:metadata.name,STATE:metadata.annotations.machineconfiguration\\.openshift\\.io/state,DESIRED:metadata.annotations.machineconfiguration\\.openshift\\.io/desiredConfig,CURRENT:metadata.annotations.machineconfiguration\\.openshift\\.io/currentConfig,REASON:metadata.annotations.machineconfiguration\\.openshift\\.io/reason"' +---- ++ +The command prints the following information every second: ++ +* The conditions on the status of the `network.config.openshift.io/cluster` object, reporting the progress of the migration. +* The status of different nodes with respect to the `machine-config-operator` resource, including whether they are upgrading or have been upgraded, as well as their current and desired configurations. + +. Complete the following steps only if the migration succeeds and your cluster is in a good state: + +.. Remove the `network.openshift.io/network-type-migration=` annotation from the `network.config` custom resource by entering the following command: ++ +[source,terminal] +---- +$ oc annotate network.config cluster network.openshift.io/network-type-migration- +---- + +.. Remove the OVN-Kubernetes network provider namespace by entering the following command: ++ +[source,terminal] +---- +$ oc delete namespace openshift-ovn-kubernetes +---- \ No newline at end of file diff --git a/modules/patching-ovnk-address-ranges.adoc b/modules/patching-ovnk-address-ranges.adoc new file mode 100644 index 000000000000..6af76d33029c --- /dev/null +++ b/modules/patching-ovnk-address-ranges.adoc @@ -0,0 +1,43 @@ +// Module included in the following assemblies: +// +// * networking/ovn_kubernetes_network_provider/migrate-from-openshift-sdn.adoc + +:_mod-docs-content-type: PROCEDURE +[id="patching-ovnk-address-ranges_{context}"] += Patching OVN-Kubernetes address ranges + +OVN-Kubernetes reserves the following IP address ranges: + +* `100.64.0.0/16`. This IP address range is used for the `internalJoinSubnet` parameter of OVN-Kubernetes by default. +* `100.88.0.0/16`. This IP address range is used for the `internalTransSwitchSubnet` parameter of OVN-Kubernetes by default. + +If these IP addresses have been used by OpenShift SDN or any external networks that might communicate with this cluster, you must patch them to use a different IP address range before initiating the limited live migration. + +The following procedure can be used to patch CIDR ranges that are in use by OpenShift SDN if the migration was initially blocked. + +[NOTE] +==== +This is an optional procedure and must only be used if the migration was blocked after using the `oc patch Network.config.openshift.io cluster --type='merge' --patch '{"metadata":{"annotations":{"network.openshift.io/network-type-migration":""}},"spec":{"networkType":"OVNKubernetes"}}'` command "Initiating the limited live migration process". +==== + +.Prerequisites + +* You have access to the cluster as a user with the `cluster-admin` role. + +.Procedure + +. If the `100.64.0.0/16` IP address range is already in use, enter the following command to patch it to a different range. The following example uses `100.63.0.0/16`. ++ +[source,terminal] +---- +$ oc patch network.operator.openshift.io cluster --type='merge' -p='{"spec":{"defaultNetwork":{"ovnKubernetesConfig":{"ipv4":{"internalJoinSubnet": "100.63.0.0/16"}}}}}' +---- + +. If the `100.88.0.0/16` IP address range is already in use, enter the following command to patch it to a different range. The following example uses `100.99.0.0/16`. ++ +[source,terminal] +---- +$ oc patch network.operator.openshift.io cluster --type='merge' -p='{"spec":{"defaultNetwork":{"ovnKubernetesConfig":{"ipv4":{"internalTransitSwitchSubnet": "100.99.0.0/16"}}}}}' +---- + +After patching the `100.64.0.0/16` and `100.88.0.0/16` IP address ranges, you can initiate the limited live migration. \ No newline at end of file diff --git a/modules/removing-egress-router-pods-before-initiating-limited-live-migration.adoc b/modules/removing-egress-router-pods-before-initiating-limited-live-migration.adoc new file mode 100644 index 000000000000..b8303fa33fab --- /dev/null +++ b/modules/removing-egress-router-pods-before-initiating-limited-live-migration.adoc @@ -0,0 +1,52 @@ +// Module included in the following assemblies: +// +// * networking/ovn_kubernetes_network_provider/migrate-from-openshift-sdn.adoc + +:_mod-docs-content-type: PROCEDURE +[id="removing-egress-router-pods-before-initiating-limited-live-migration_{context}"] += Removing egress router pods before initiating the limited live migration + +Before initiating the limited live migration, you must check for, and remove, any egress router pods. If there is an egress router pod on the cluster when performing a limited live migration, the Network Operator blocks the migration and returns the following error: + +[source,text] +---- +The cluster configuration is invalid (network type limited live migration is not supported for pods with `pod.network.openshift.io/assign-macvlan` annotation. + +Please remove all egress router pods). Use `oc edit network.config.openshift.io cluster` to fix. +---- + +.Prerequisites + +* You have access to the cluster as a user with the `cluster-admin` role. + +.Procedure + +. To locate egress router pods on your cluster, enter the following command: ++ +[source,terminal] +---- +$ oc get pods --all-namespaces -o json | jq '.items[] | select(.metadata.annotations."pod.network.openshift.io/assign-macvlan" == "true") | {name: .metadata.name, namespace: .metadata.namespace}' +---- ++ +.Example output ++ +[source,terminal] +---- +{ + "name": "egress-multi", + "namespace": "egress-router-project" +} +---- + +. Alternatively, you can query metrics on the {product-title} web console. + +.. On the {product-title} web console, click *Observe* -> *Metrics*. + +.. In the *Expression* box, enter `network_attachment_definition_instances{networks="egress-router"}`. Then, click *Add*. + +. To remove an egress router pod, enter the following command: ++ +[source,terminal] +---- +$ oc delete pod -n +---- \ No newline at end of file diff --git a/networking/ovn_kubernetes_network_provider/migrate-from-openshift-sdn.adoc b/networking/ovn_kubernetes_network_provider/migrate-from-openshift-sdn.adoc index 0e66650ddf29..5de4ba05b2ac 100644 --- a/networking/ovn_kubernetes_network_provider/migrate-from-openshift-sdn.adoc +++ b/networking/ovn_kubernetes_network_provider/migrate-from-openshift-sdn.adoc @@ -6,7 +6,12 @@ include::_attributes/common-attributes.adoc[] toc::[] -As a cluster administrator, you can migrate to the OVN-Kubernetes network plugin from the OpenShift SDN network plugin. +As a cluster administrator, you can migrate to the OVN-Kubernetes network plugin from the OpenShift SDN network plugin using the _offline_ migration method or the limited live migration method. + +[WARNING] +==== +It is not possible to upgrade a cluster to {product-title} 4.17 if it is using the OpenShift SDN network plugin. You must migrate to the OVN-Kubernetes plugin before upgrading to {product-title} 4.17. +==== To learn more about OVN-Kubernetes, read xref:../../networking/ovn_kubernetes_network_provider/about-ovn-kubernetes#about-ovn-kubernetes[About the OVN-Kubernetes network plugin]. @@ -15,7 +20,8 @@ To learn more about OVN-Kubernetes, read xref:../../networking/ovn_kubernetes_ne Do not automate the migration from OpenShift SDN to OVN-Kubernetes with a script or another tool like Red Hat Ansible Automation Platform. This might cause outages or crash your {product-title} cluster. ==== -// Migration to the OVN-Kubernetes network plugin +// How the migration process works +// Offline migration to the OVN-Kubernetes network plugin overview include::modules/nw-ovn-kubernetes-migration-about.adoc[leveloffset=+1] [role="_additional-resources"] @@ -24,11 +30,41 @@ include::modules/nw-ovn-kubernetes-migration-about.adoc[leveloffset=+1] * xref:../../updating/understanding_updates/understanding-update-channels-release.adoc#understanding-update-channels-release[Understanding update channels and releases] * xref:../../release_notes/ocp-4-15-release-notes.adoc#ocp-4-15-asynchronous-errata-updates_release-notes[Asynchronous errata updates] -// How the migration process works +// How the offline migration process works include::modules/nw-network-plugin-migration-process.adoc[leveloffset=+2] -// Migrating to the OVN-Kubernetes network plugin -include::modules/nw-ovn-kubernetes-migration.adoc[leveloffset=+1] +// Migrating to the OVN-Kubernetes network plugin by using the offline migration method +include::modules/nw-ovn-kubernetes-migration.adoc[leveloffset=+2] + +// Limited live migration to the OVN-Kubernetes network plugin overview +include::modules/nw-ovn-kubernetes-live-migration-about.adoc[leveloffset=+1] + +// How the live migration process works +include::modules/how-the-live-migration-process-works.adoc[leveloffset=+2] + +// Migrating to the OVN-Kubernetes network plugin by using the limited live migration method +include::modules/nw-ovn-kubernetes-live-migration.adoc[leveloffset=+2] + +// Checking cluster resources before initiating the limited live migration +include::modules/checking-cluster-resources-before-initiating-limited-live-migration.adoc[leveloffset=+3] + +// Removing egress router pods before initiating the limited live migration +include::modules/removing-egress-router-pods-before-initiating-limited-live-migration.adoc[leveloffset=+3] + +// Initiating the limited live migration process +include::modules/initiating-limited-live-migration.adoc[leveloffset=+3] + +// Patching OVN-Kubernetes address ranges +include::modules/patching-ovnk-address-ranges.adoc[leveloffset=+3] + +// Checking cluster resources after initiating the limited live migration +include::modules/checking-cluster-resources-after-initiating-limited-live-migration.adoc[leveloffset=+3] + +// Checking limited live migration metrics +include::modules/nw-ovn-kubernetes-checking-live-migration-metrics.adoc[leveloffset=+2] + +// Information about limited live migration metrics +include::modules/live-migration-metrics-information.adoc[leveloffset=+3] [role="_additional-resources"] [id="migrate-from-openshift-sdn-additional-resources"] @@ -37,16 +73,18 @@ include::modules/nw-ovn-kubernetes-migration.adoc[leveloffset=+1] * link:https://access.redhat.com/labs/ocpnc/[Red Hat OpenShift Network Calculator] * xref:../../networking/cluster-network-operator.adoc#nw-operator-configuration-parameters-for-ovn-sdn_cluster-network-operator[Configuration parameters for the OVN-Kubernetes network plugin] * xref:../../backup_and_restore/control_plane_backup_and_restore/backing-up-etcd.adoc#backup-etcd[Backing up etcd] -* xref:../../networking/network_policy/about-network-policy.adoc#about-network-policy[About network policy] +* xref:../../networking/network_security/network_policy/about-network-policy.adoc#about-network-policy[About network policy] * xref:../../networking/changing-cluster-network-mtu.adoc#nw-cluster-mtu-change_changing-cluster-network-mtu[Changing the cluster MTU] * xref:../../networking/changing-cluster-network-mtu.adoc#mtu-value-selection_changing-cluster-network-mtu[MTU value selection] +* xref:../../networking/network_security/network_policy/about-network-policy.adoc#nw-networkpolicy-optimize-ovn_about-network-policy[About network policy] * OVN-Kubernetes capabilities - xref:../../networking/ovn_kubernetes_network_provider/configuring-egress-ips-ovn.adoc#configuring-egress-ips-ovn[Configuring an egress IP address] -- xref:../../networking/ovn_kubernetes_network_provider/configuring-egress-firewall-ovn.adoc#configuring-egress-firewall-ovn[Configuring an egress firewall for a project] +- xref:../../networking/network_security/egress_firewall/configuring-egress-firewall-ovn.adoc#configuring-egress-firewall-ovn[Configuring an egress firewall for a project] - link:https://access.redhat.com/solutions/7078619[OVN-Kubernetes egress firewall blocks process to deploy application as DeploymentConfig] - xref:../../networking/ovn_kubernetes_network_provider/enabling-multicast.adoc#nw-ovn-kubernetes-enabling-multicast[Enabling multicast for a project] * OpenShift SDN capabilities - xref:../../networking/openshift_sdn/assigning-egress-ips.adoc#assigning-egress-ips[Configuring egress IPs for a project] - xref:../../networking/openshift_sdn/configuring-egress-firewall.adoc#configuring-egress-firewall[Configuring an egress firewall for a project] - xref:../../networking/openshift_sdn/enabling-multicast.adoc#enabling-multicast[Enabling multicast for a project] -* xref:../../rest_api/operator_apis/network-operator-openshift-io-v1.adoc#network-operator-openshift-io-v1[Network [operator.openshift.io/v1]] +- xref:../../networking/openshift_sdn/deploying-egress-router-layer3-redirection.adoc#deploying-egress-router-layer3-redirection[Deploying an egress router pod in redirect mode] +* xref:../../rest_api/operator_apis/network-operator-openshift-io-v1.adoc#network-operator-openshift-io-v1[Network [operator.openshift.io/v1]] \ No newline at end of file diff --git a/networking/ovn_kubernetes_network_provider/rollback-to-openshift-sdn.adoc b/networking/ovn_kubernetes_network_provider/rollback-to-openshift-sdn.adoc index 033c309d7673..6460cb629b87 100644 --- a/networking/ovn_kubernetes_network_provider/rollback-to-openshift-sdn.adoc +++ b/networking/ovn_kubernetes_network_provider/rollback-to-openshift-sdn.adoc @@ -6,8 +6,15 @@ include::_attributes/common-attributes.adoc[] toc::[] -As a cluster administrator, you can rollback to the OpenShift SDN from the OVN-Kubernetes network plugin only after the migration to the OVN-Kubernetes network plugin is completed and successful. +As a cluster administrator, you can roll back to the OpenShift SDN network plugin from the OVN-Kubernetes network plugin using either the _offline_ migration method, or the _limited live_ migration method. This can only be done after the migration to the OVN-Kubernetes network plugin has successfully completed. + +[NOTE] +==== +* If you used the offline migration method to migrate to the OpenShift SDN network plugin from the OVN-Kubernetes network plugin, you should use the offline migration rollback method. +* If you used the limited live migration method to migrate to the OpenShift SDN network plugin from the OVN-Kubernetes network plugin, you should use the limited live migration rollback method. +==== include::snippets/sdn-deprecation-statement.adoc[] -include::modules/nw-ovn-kubernetes-rollback.adoc[leveloffset=+1] \ No newline at end of file +include::modules/nw-ovn-kubernetes-rollback.adoc[leveloffset=+1] +include::modules/nw-ovn-kubernetes-rollback-live.adoc[leveloffset=+1] \ No newline at end of file