From 3b9f10975d509fc9799b10d535a1c61d99a986a9 Mon Sep 17 00:00:00 2001 From: dfitzmau Date: Tue, 28 Oct 2025 17:14:37 +0000 Subject: [PATCH] OCPBUGS-61836: Updated the br-ex NNCP example's ip values --- .../upi/installing-bare-metal.adoc | 14 +- ...manifest-file-customized-br-ex-bridge.adoc | 147 +++++++++--------- ...-additional-install-config-parameters.adoc | 4 +- ...-nmstate-updating-node-network-config.adoc | 2 +- 4 files changed, 91 insertions(+), 76 deletions(-) diff --git a/installing/installing_bare_metal/upi/installing-bare-metal.adoc b/installing/installing_bare_metal/upi/installing-bare-metal.adoc index 0dfce75be354..d3781282a3bd 100644 --- a/installing/installing_bare_metal/upi/installing-bare-metal.adoc +++ b/installing/installing_bare_metal/upi/installing-bare-metal.adoc @@ -10,7 +10,8 @@ In {product-title} {product-version}, you can install a cluster on bare-metal in [IMPORTANT] ==== -While you might be able to follow this procedure to deploy a cluster on virtualized or cloud environments, you must be aware of additional considerations for non-bare-metal platforms. Review the information in the link:https://access.redhat.com/articles/4207611[guidelines for deploying {product-title} on non-tested platforms] before you attempt to install an {product-title} cluster in such an environment. +While you might be able to follow this procedure to deploy a cluster on virtualized or cloud environments, you must be aware of additional +considerations for non-bare-metal platforms. Review the information in the link:https://access.redhat.com/articles/4207611[guidelines for deploying {product-title} on non-tested platforms] before you attempt to install an {product-title} cluster in such an environment. ==== == Prerequisites @@ -24,6 +25,7 @@ While you might be able to follow this procedure to deploy a cluster on virtuali Be sure to also review this site list if you are configuring a proxy. ==== +// Internet access for OpenShift Container Platform include::modules/cluster-entitlements.adoc[leveloffset=+1] [role="_additional-resources"] @@ -39,8 +41,10 @@ of the required machines. This section describes the requirements for deploying {product-title} on user-provisioned infrastructure. +// Required machines for cluster installation include::modules/installation-machine-requirements.adoc[leveloffset=+2] +// Minimum resource requirements for cluster installation include::modules/installation-minimum-resource-requirements.adoc[leveloffset=+2] [role="_additional-resources"] @@ -81,6 +85,7 @@ include::modules/installation-dns-user-infra.adoc[leveloffset=+2] * xref:../../../installing/installing_bare_metal/upi/installing-bare-metal.adoc#installation-user-provisioned-validating-dns_installing-bare-metal[Validating DNS resolution for user-provisioned infrastructure] +// Load balancing requirements for user-provisioned infrastructure include::modules/installation-load-balancing-user-infra.adoc[leveloffset=+2] // Creating a manifest object that includes a customized `br-ex` bridge @@ -99,6 +104,7 @@ include::modules/creating-scaling-machine-sets-compute-nodes-networking.adoc[lev // Enabling OVS balance-slb mode for your cluster include::modules/enabling-OVS-balance-slb-mode.adoc[leveloffset=+1] +// Preparing the user-provisioned infrastructure include::modules/installation-infrastructure-user-infra.adoc[leveloffset=+1] [role="_additional-resources"] @@ -113,6 +119,7 @@ include::modules/installation-infrastructure-user-infra.adoc[leveloffset=+1] * xref:../../../installing/installing_bare_metal/upi/installing-bare-metal.adoc#installation-user-provisioned-validating-dns_installing-bare-metal[Validating DNS resolution for user-provisioned infrastructure] * xref:../../../installing/installing_bare_metal/upi/installing-bare-metal.adoc#installation-load-balancing-user-infra_installing-bare-metal[Load balancing requirements for user-provisioned infrastructure] +// Validating DNS resolution for user-provisioned infrastructure include::modules/installation-user-provisioned-validating-dns.adoc[leveloffset=+1] [role="_additional-resources"] @@ -121,6 +128,7 @@ include::modules/installation-user-provisioned-validating-dns.adoc[leveloffset=+ * xref:../../../installing/installing_bare_metal/upi/installing-bare-metal.adoc#installation-dns-user-infra_installing-bare-metal[User-provisioned DNS requirements] * xref:../../../installing/installing_bare_metal/upi/installing-bare-metal.adoc#installation-load-balancing-user-infra_installing-bare-metal[Load balancing requirements for user-provisioned infrastructure] +// Generating a key pair for cluster node SSH access include::modules/ssh-agent-using.adoc[leveloffset=+1] [role="_additional-resources"] @@ -128,16 +136,20 @@ include::modules/ssh-agent-using.adoc[leveloffset=+1] * xref:../../../support/troubleshooting/verifying-node-health.adoc#verifying-node-health[Verifying node health] +// Obtaining the installation program include::modules/installation-obtaining-installer.adoc[leveloffset=+1] +// Installing the OpenShift CLI include::modules/cli-installing-cli.adoc[leveloffset=+1] +// Manually creating the installation configuration file include::modules/installation-initializing-manual.adoc[leveloffset=+1] [role="_additional-resources"] .Additional resources * xref:../../../installing/installing_bare_metal/upi/installation-config-parameters-bare-metal.adoc#installation-config-parameters-bare-metal[Installation configuration parameters for bare metal] +// Sample install-config.yaml file for bare metal include::modules/installation-bare-metal-config-yaml.adoc[leveloffset=+2] [role="_additional-resources"] diff --git a/modules/creating-manifest-file-customized-br-ex-bridge.adoc b/modules/creating-manifest-file-customized-br-ex-bridge.adoc index a9845da54558..b51046fa216a 100644 --- a/modules/creating-manifest-file-customized-br-ex-bridge.adoc +++ b/modules/creating-manifest-file-customized-br-ex-bridge.adoc @@ -1,34 +1,31 @@ // Module included in the following assemblies: // -// IPI // * installing/installing_bare_metal/ipi/ipi-install-installation-workflow.adoc // * installing/installing_bare_metal/bare-metal-postinstallation-configuration.adoc -// UPI // * installing/installing_bare_metal/upi/installing-bare-metal-network-customizations.adoc // * installing/installing_bare_metal/upi/installing-restricted-networks-bare-metal.adoc // * installing/installing_bare_metal/upi/installing-bare-metal.adoc -ifeval::["{context}" == "ipi-install-post-installation-configuration"] -:postinstall-bare-metal-ipi: -endif::[] -ifeval::["{context}" == "post-install-bare-metal-configuration"] -:postinstall-bare-metal-upi: +ifeval::["{context}" == "bare-metal-postinstallation-configuration"] +:postinstall-bare-metal: endif::[] :_mod-docs-content-type: PROCEDURE [id="creating-manifest-file-customized-br-ex-bridge_{context}"] = Creating a manifest object that includes a customized `br-ex` bridge -ifndef::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[] +ifndef::postinstall-bare-metal[] As an alternative to using the `configure-ovs.sh` shell script to set a `br-ex` bridge on a bare-metal platform, you can create a `MachineConfig` object that includes an NMState configuration file. The host `nmstate-configuration.service` and `nmstate.service` apply the NMState configuration file to each node that runs in your cluster. -endif::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[] +endif::postinstall-bare-metal[] + +ifdef::postinstall-bare-metal[] +As an alternative to using the `configure-ovs.sh` shell script to set a `br-ex` bridge on a bare-metal platform, you can create a `NodeNetworkConfigurationPolicy` (NNCP) custom resource (CR) that includes an NMState configuration file. -ifdef::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[] -As an alternative to using the `configure-ovs.sh` shell script to set a `br-ex` bridge on a bare-metal platform, you can create a `NodeNetworkConfigurationPolicy` (NNCP) custom resource (CR) that includes an NMState configuration file. The Kubernetes NMState Operator uses the NMState configuration file to create a customized `br-ex` bridge network configuration on each node in your cluster. +The Kubernetes NMState Operator uses the NMState configuration file to create a customized `br-ex` bridge network configuration on each node in your cluster. [IMPORTANT] ==== -After creating the `NodeNetworkConfigurationPolicy` CR, copy content from the NMState configuration file that was created during cluster installation into the NNCP CR. An incomplete NNCP CR file means that the the network policy described in the file cannot get applied to nodes in the cluster. +After creating the `NodeNetworkConfigurationPolicy` CR, copy content from the NMState configuration file that was created during cluster installation into the NNCP CR. An incomplete NNCP CR file means that the network policy described in the file cannot get applied to nodes in the cluster. ==== This feature supports the following tasks: @@ -36,7 +33,7 @@ This feature supports the following tasks: * Modifying the maximum transmission unit (MTU) for your cluster. * Modifying attributes of a different bond interface, such as MIImon (Media Independent Interface Monitor), bonding mode, or Quality of Service (QoS). * Updating DNS values. -endif::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[] +endif::postinstall-bare-metal[] Consider the following use cases for creating a manifest object that includes a customized `br-ex` bridge: @@ -44,15 +41,14 @@ Consider the following use cases for creating a manifest object that includes a * You want to deploy the bridge on a different interface than the interface available on a host or server IP address. * You want to make advanced configurations to the bridge that are not possible with the `configure-ovs.sh` shell script. Using the script for these configurations might result in the bridge failing to connect multiple network interfaces and facilitating data forwarding between the interfaces. -ifndef::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[] - +ifndef::postinstall-bare-metal[] [NOTE] ==== If you require an environment with a single network interface controller (NIC) and default network settings, use the `configure-ovs.sh` shell script. ==== After you install {op-system-first} and the system reboots, the Machine Config Operator injects Ignition configuration files into each node in your cluster, so that each node received the `br-ex` bridge network configuration. To prevent configuration conflicts, the `configure-ovs.sh` shell script receives a signal to not configure the `br-ex` bridge. -endif::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[] +endif::postinstall-bare-metal[] [WARNING] ==== @@ -75,29 +71,29 @@ The following list of interface names are reserved and you cannot use the names ==== .Prerequisites -ifndef::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[] +ifndef::postinstall-bare-metal[] * Optional: You have installed the link:https://nmstate.io/[`nmstate`] API so that you can validate the NMState configuration. -endif::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[] +endif::postinstall-bare-metal[] -ifdef::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[] +ifdef::postinstall-bare-metal[] * You set a customized `br-ex` by using the alternative method to `configure-ovs`. * You installed the Kubernetes NMState Operator. -endif::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[] +endif::postinstall-bare-metal[] .Procedure -ifndef::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[] +ifndef::postinstall-bare-metal[] . Create a NMState configuration file that has decoded base64 information for your customized `br-ex` bridge network: + .Example of an NMState configuration for a customized `br-ex` bridge network [source,yaml] ---- interfaces: -- name: enp2s0 <1> - type: ethernet <2> - state: up <3> +- name: enp2s0 + type: ethernet + state: up ipv4: - enabled: false <4> + enabled: false ipv6: enabled: false - name: br-ex @@ -113,7 +109,7 @@ interfaces: options: mcast-snooping-enable: true port: - - name: enp2s0 <5> + - name: enp2s0 - name: br-ex - name: br-ex type: ovs-interface @@ -122,27 +118,33 @@ interfaces: ipv4: enabled: true dhcp: true - auto-route-metric: 48 <6> + auto-route-metric: 48 ipv6: enabled: true dhcp: true auto-route-metric: 48 # ... ---- -<1> Name of the interface. -<2> The type of ethernet. -<3> The requested state for the interface after creation. -<4> Disables IPv4 and IPv6 in this example. -<5> The node NIC to which the bridge attaches. -<6> Set the parameter to `48` to ensure the `br-ex` default route always has the highest precedence (lowest metric). This configuration prevents routing conflicts with any other interfaces that are automatically configured by the `NetworkManager` service. ++ +where: ++ +`interfaces.name`:: Name of the interface. +`interfaces.type`:: The type of ethernet. +`interfaces.state`:: The requested state for the interface after creation. +`ipv4.enabled`:: Disables IPv4 and IPv6 in this example. +`port.name`:: The node NIC to which the bridge attaches. +`auto-route-metric`:: Set the parameter to `48` to ensure the `br-ex` default route always has the highest precedence (lowest metric). This configuration prevents routing conflicts with any other interfaces that are automatically configured by the `NetworkManager` service. . Use the `cat` command to base64-encode the contents of the NMState configuration: + [source,terminal] ---- -$ cat .yaml | base64 <1> +$ cat .yml | base64 ---- -<1> Replace `` with the name of your NMState resource YAML file. ++ +where: ++ +``:: Replace `` with the name of your NMState resource YAML file. . Create a `MachineConfig` manifest file and define a customized `br-ex` bridge network configuration analogous to the following example: + @@ -153,7 +155,7 @@ kind: MachineConfig metadata: labels: machineconfiguration.openshift.io/role: worker - name: 10-br-ex-worker <1> + name: 10-br-ex-worker spec: config: ignition: @@ -161,20 +163,23 @@ spec: storage: files: - contents: - source: data:text/plain;charset=utf-8;base64, <2> + source: data:text/plain;charset=utf-8;base64, mode: 0644 overwrite: true - path: /etc/nmstate/openshift/worker-0.yml <3> + path: /etc/nmstate/openshift/worker-0.yml - contents: source: data:text/plain;charset=utf-8;base64, mode: 0644 overwrite: true - path: /etc/nmstate/openshift/worker-1.yml <3> + path: /etc/nmstate/openshift/worker-1.yml # ... ---- -<1> The name of the policy. -<2> Writes the encoded base64 information to the specified path. -<3> For each node in your cluster, specify the hostname path to your node and the base-64 encoded Ignition configuration file data for the machine type. The `worker` role is the default role for nodes in your cluster. The `.yaml` extension does not work when specifying the short hostname, `hostname -s`, path for each node or all nodes in the `MachineConfig` manifest file. ++ +where: ++ +`metadata.name`:: The name of the policy. +`contents.source`:: Writes the encoded base64 information to the specified path. +`path`:: For each node in your cluster, specify the hostname path to your node and the base-64 encoded Ignition configuration file data for the machine type. The `worker` role is the default role for nodes in your cluster. You must use the `.yml` extension for configuration files, such as `$(hostname -s).yml` when specifying the short hostname path for each node or all nodes in the `MachineConfig` manifest file. + If you have a single global configuration specified in an `/etc/nmstate/openshift/cluster.yml` configuration file that you want to apply to all nodes in your cluster, you do not need to specify the short hostname path for each node, such as `/etc/nmstate/openshift/.yml`. For example: + @@ -188,35 +193,32 @@ If you have a single global configuration specified in an `/etc/nmstate/openshif path: /etc/nmstate/openshift/cluster.yml # ... ---- -endif::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[] +endif::postinstall-bare-metal[] -ifdef::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[] -* Create a `NodeNetworkConfigurationPolicy` (NNCP) CR and define a customized `br-ex` bridge network configuration. Depending on your needs, ensure that you set a masquerade IP for either the `ipv4.address.ip`, `ipv6.address.ip`, or both parameters. Always include a masquerade IP address in the NNCP CR and this address must match an in-use IP address block. +ifdef::postinstall-bare-metal[] +* Create a `NodeNetworkConfigurationPolicy` (NNCP) CR and define a customized `br-ex` bridge network configuration. The `br-ex` NNCP CR must include the OVN-Kubernetes masquerade IP address and subnet of your network. The example NNCP CR includes default values in the `ipv4.address.ip` and `ipv6.address.ip` parameters. You can set the masquerade IP address in the `ipv4.address.ip`, `ipv6.address.ip`, or both parameters. + [IMPORTANT] ==== -As a post-installation task, you can configure most parameters for a customized `br-ex` bridge that you defined in an existing NNCP CR, except for the primary IP address of the customized `br-ex` bridge. - -If you want to convert your single-stack cluster network to a dual-stack cluster network, you can add or change a secondary IPv6 address in the NNCP CR, but the existing primary IP address cannot be changed. +As a post-installation task, you cannot change the primary IP address of the customized `br-ex` bridge. If you want to convert your single-stack cluster network to a dual-stack cluster network, you can add or change a secondary IPv6 address in the NNCP CR, but the existing primary IP address cannot be changed. ==== + -.Example of an NNCP CR that sets IPv6 and IPv4 masquerade IP addresses [source,yaml] ---- apiVersion: nmstate.io/v1 kind: NodeNetworkConfigurationPolicy metadata: - name: worker-0-br-ex <1> + name: worker-0-br-ex spec: nodeSelector: kubernetes.io/hostname: worker-0 desiredState: interfaces: - - name: enp2s0 <2> - type: ethernet <3> - state: up <4> + - name: enp2s0 + type: ethernet + state: up ipv4: - enabled: false <5> + enabled: false ipv6: enabled: false - name: br-ex @@ -232,7 +234,7 @@ spec: options: mcast-snooping-enable: true port: - - name: enp2s0 <6> + - name: enp2s0 - name: br-ex - name: br-ex type: ovs-interface @@ -241,35 +243,36 @@ spec: ipv4: enabled: true dhcp: true - auto-route-metric: 48 <7> + auto-route-metric: 48 address: - - ip: "169.254.169.2" - prefix-length: 29 + - ip: "169.254.0.2" + prefix-length: 17 ipv6: enabled: true dhcp: true auto-route-metric: 48 address: - ip: "fd69::2" - prefix-length: 125 + prefix-length: 112 # ... ---- -<1> Name of the policy. -<2> Name of the interface. -<3> The type of ethernet. -<4> The requested state for the interface after creation. -<5> Disables IPv4 and IPv6 in this example. -<6> The node NIC to which the bridge is attached. -<7> Set the parameter to `48` to ensure the `br-ex` default route always has the highest precedence (lowest metric). This configuration prevents routing conflicts with any other interfaces that are automatically configured by the `NetworkManager` service. -endif::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[] ++ +where: ++ +`metadata.name`:: Name of the policy. +`interfaces.name`:: Name of the interface. +`interfaces.type`:: The type of ethernet. +`interfaces.state`:: The requested state for the interface after creation. +`ipv4.enabled`:: Disables IPv4 and IPv6 in this example. +`port.name`:: The node NIC to which the bridge is attached. +`address.ip`:: Shows the default IPv4 and IPv6 IP addresses. Ensure that you set the masquerade IPv4 and IPv6 IP addresses of your network. +`auto-route-metric`:: Set the parameter to `48` to ensure the `br-ex` default route always has the highest precedence (lowest metric). This configuration prevents routing conflicts with any other interfaces that are automatically configured by the `NetworkManager` service. +endif::postinstall-bare-metal[] .Next steps * Scaling compute nodes to apply the manifest object that includes a customized `br-ex` bridge to each compute node that exists in your cluster. For more information, see "Expanding the cluster" in the _Additional resources_ section. -ifeval::["{context}" == "ipi-install-post-installation-configuration"] -:!postinstall-bare-metal-ipi: -endif::[] -ifeval::["{context}" == "bare-metal-configuration"] -:!postinstall-bare-metal-upi: +ifeval::["{context}" == "bare-metal-postinstallation-configuration"] +:!postinstall-bare-metal: endif::[] diff --git a/modules/ipi-install-additional-install-config-parameters.adoc b/modules/ipi-install-additional-install-config-parameters.adoc index 506b1c2c16fa..a5934067d54a 100644 --- a/modules/ipi-install-additional-install-config-parameters.adoc +++ b/modules/ipi-install-additional-install-config-parameters.adoc @@ -119,7 +119,7 @@ a| `provisioningNetworkInterface` | | The name of the network interface on node | `apiVIPs` | a| (Optional) The virtual IP address for Kubernetes API communication. -You must either provide this setting in the `install-config.yaml` file as a reserved IP from the `MachineNetwork` parameter or preconfigured in the DNS so that the default name resolves correctly. Use the virtual IP address and not the FQDN when adding a value to the `apiVIPs` configuration setting in the `install-config.yaml` file. The primary IP address must be from the IPv4 network when using dual stack networking. If not set, the installation program uses `api..` to derive the IP address from the DNS. +You must either provide this setting in the `install-config.yaml` file as a reserved IP from the `MachineNetwork` parameter or preconfigured in the DNS so that the default name resolves correctly. Use the virtual IP address and not the FQDN when adding a value to the `apiVIPs` configuration setting in the `install-config.yaml` file. For dual-stack networking, the primary IP address can be either an IPv4 network or an IPv6 network. If not set, the installation program uses `api..` to derive the IP address from the DNS. [NOTE] ==== @@ -131,7 +131,7 @@ Before {product-title} 4.12, the cluster installation program only accepted an I | `ingressVIPs` | a| (Optional) The virtual IP address for ingress traffic. -You must either provide this setting in the `install-config.yaml` file as a reserved IP from the `MachineNetwork` parameter or preconfigured in the DNS so that the default name resolves correctly. Use the virtual IP address and not the FQDN when adding a value to the `ingressVIPs` configuration setting in the `install-config.yaml` file. The primary IP address must be from the IPv4 network when using dual stack networking. If not set, the installation program uses `test.apps..` to derive the IP address from the DNS. +You must either provide this setting in the `install-config.yaml` file as a reserved IP from the `MachineNetwork` parameter or preconfigured in the DNS so that the default name resolves correctly. Use the virtual IP address and not the FQDN when adding a value to the `ingressVIPs` configuration setting in the `install-config.yaml` file. For dual-stack networking, the primary IP address can be either an IPv4 network or an IPv6 network. If not set, the installation program uses `test.apps..` to derive the IP address from the DNS. [NOTE] ==== diff --git a/networking/k8s_nmstate/k8s-nmstate-updating-node-network-config.adoc b/networking/k8s_nmstate/k8s-nmstate-updating-node-network-config.adoc index b94c6372f156..104d21962972 100644 --- a/networking/k8s_nmstate/k8s-nmstate-updating-node-network-config.adoc +++ b/networking/k8s_nmstate/k8s-nmstate-updating-node-network-config.adoc @@ -13,7 +13,7 @@ For more information about how to install the NMState Operator, see xref:../../n [IMPORTANT] ==== -You cannot provide any configuration that modifies the br-ex bridge, an OVN-Kubernetes-managed Open vSwitch bridge. However, you can configure a customized br-ex bridge. +You cannot modify an existing `br-ex` bridge, an OVN-Kubernetes-managed Open vSwitch bridge, or any interfaces, bonds, VLANs, and so on that associate with the `br-ex` bridge. However, you can configure a customized br-ex bridge. For more information, see "Creating a manifest object that includes a customized br-ex bridge" in the _Deploying installer-provisioned clusters on bare metal_ document or the _Installing a user-provisioned cluster on bare metal_ document. ====