From fbcfce2295e1253f7b912f23651defd2481d8d07 Mon Sep 17 00:00:00 2001 From: dfitzmau Date: Tue, 28 Oct 2025 17:14:37 +0000 Subject: [PATCH] OCPBUGS-61836: Updated the br-ex NNCP example's ip values --- .../upi/installing-bare-metal.adoc | 21 ++-- ...manifest-file-customized-br-ex-bridge.adoc | 100 ++++++++++-------- ...-additional-install-config-parameters.adoc | 4 +- ...-nmstate-updating-node-network-config.adoc | 2 +- 4 files changed, 73 insertions(+), 54 deletions(-) diff --git a/installing/installing_bare_metal/upi/installing-bare-metal.adoc b/installing/installing_bare_metal/upi/installing-bare-metal.adoc index c191111405d4..f949f6522f5b 100644 --- a/installing/installing_bare_metal/upi/installing-bare-metal.adoc +++ b/installing/installing_bare_metal/upi/installing-bare-metal.adoc @@ -6,16 +6,12 @@ include::_attributes/common-attributes.adoc[] toc::[] -In {product-title} {product-version}, you can install a cluster on -bare metal infrastructure that you provision. +In {product-title} {product-version}, you can install a cluster on bare-metal infrastructure that you provision. [IMPORTANT] ==== -While you might be able to follow this procedure to deploy a cluster on -virtualized or cloud environments, you must be aware of additional -considerations for non-bare metal platforms. Review the information in the -link:https://access.redhat.com/articles/4207611[guidelines for deploying {product-title} on non-tested platforms] -before you attempt to install an {product-title} cluster in such an environment. +While you might be able to follow this procedure to deploy a cluster on virtualized or cloud environments, you must be aware of additional +considerations for non-bare-metal platforms. Review the information in the link:https://access.redhat.com/articles/4207611[guidelines for deploying {product-title} on non-tested platforms] before you attempt to install an {product-title} cluster in such an environment. ==== == Prerequisites @@ -29,6 +25,7 @@ before you attempt to install an {product-title} cluster in such an environment. Be sure to also review this site list if you are configuring a proxy. ==== +// Internet access for OpenShift Container Platform include::modules/cluster-entitlements.adoc[leveloffset=+1] [role="_additional-resources"] @@ -44,8 +41,10 @@ of the required machines. This section describes the requirements for deploying {product-title} on user-provisioned infrastructure. +// Required machines for cluster installation include::modules/installation-machine-requirements.adoc[leveloffset=+2] +// Minimum resource requirements for cluster installation include::modules/installation-minimum-resource-requirements.adoc[leveloffset=+2] [role="_additional-resources"] @@ -86,6 +85,7 @@ include::modules/installation-dns-user-infra.adoc[leveloffset=+2] * xref:../../../installing/installing_bare_metal/upi/installing-bare-metal.adoc#installation-user-provisioned-validating-dns_installing-bare-metal[Validating DNS resolution for user-provisioned infrastructure] +// Load balancing requirements for user-provisioned infrastructure include::modules/installation-load-balancing-user-infra.adoc[leveloffset=+2] // Creating a manifest object that includes a customized `br-ex` bridge @@ -104,6 +104,7 @@ include::modules/creating-scaling-machine-sets-compute-nodes-networking.adoc[lev // Enabling OVS balance-slb mode for your cluster include::modules/enabling-OVS-balance-slb-mode.adoc[leveloffset=+1] +// Preparing the user-provisioned infrastructure include::modules/installation-infrastructure-user-infra.adoc[leveloffset=+1] [role="_additional-resources"] @@ -118,6 +119,7 @@ include::modules/installation-infrastructure-user-infra.adoc[leveloffset=+1] * xref:../../../installing/installing_bare_metal/upi/installing-bare-metal.adoc#installation-user-provisioned-validating-dns_installing-bare-metal[Validating DNS resolution for user-provisioned infrastructure] * xref:../../../installing/installing_bare_metal/upi/installing-bare-metal.adoc#installation-load-balancing-user-infra_installing-bare-metal[Load balancing requirements for user-provisioned infrastructure] +// Validating DNS resolution for user-provisioned infrastructure include::modules/installation-user-provisioned-validating-dns.adoc[leveloffset=+1] [role="_additional-resources"] @@ -126,6 +128,7 @@ include::modules/installation-user-provisioned-validating-dns.adoc[leveloffset=+ * xref:../../../installing/installing_bare_metal/upi/installing-bare-metal.adoc#installation-dns-user-infra_installing-bare-metal[User-provisioned DNS requirements] * xref:../../../installing/installing_bare_metal/upi/installing-bare-metal.adoc#installation-load-balancing-user-infra_installing-bare-metal[Load balancing requirements for user-provisioned infrastructure] +// Generating a key pair for cluster node SSH access include::modules/ssh-agent-using.adoc[leveloffset=+1] [role="_additional-resources"] @@ -133,16 +136,20 @@ include::modules/ssh-agent-using.adoc[leveloffset=+1] * xref:../../../support/troubleshooting/verifying-node-health.adoc#verifying-node-health[Verifying node health] +// Obtaining the installation program include::modules/installation-obtaining-installer.adoc[leveloffset=+1] +// Installing the OpenShift CLI include::modules/cli-installing-cli.adoc[leveloffset=+1] +// Manually creating the installation configuration file include::modules/installation-initializing-manual.adoc[leveloffset=+1] [role="_additional-resources"] .Additional resources * xref:../../../installing/installing_bare_metal/upi/installation-config-parameters-bare-metal.adoc#installation-config-parameters-bare-metal[Installation configuration parameters for bare metal] +// Sample install-config.yaml file for bare metal include::modules/installation-bare-metal-config-yaml.adoc[leveloffset=+2] [role="_additional-resources"] diff --git a/modules/creating-manifest-file-customized-br-ex-bridge.adoc b/modules/creating-manifest-file-customized-br-ex-bridge.adoc index 961c707276e5..b51046fa216a 100644 --- a/modules/creating-manifest-file-customized-br-ex-bridge.adoc +++ b/modules/creating-manifest-file-customized-br-ex-bridge.adoc @@ -19,7 +19,9 @@ As an alternative to using the `configure-ovs.sh` shell script to set a `br-ex` endif::postinstall-bare-metal[] ifdef::postinstall-bare-metal[] -As an alternative to using the `configure-ovs.sh` shell script to set a `br-ex` bridge on a bare-metal platform, you can create a `NodeNetworkConfigurationPolicy` (NNCP) custom resource (CR) that includes an NMState configuration file. The Kubernetes NMState Operator uses the NMState configuration file to create a customized `br-ex` bridge network configuration on each node in your cluster. +As an alternative to using the `configure-ovs.sh` shell script to set a `br-ex` bridge on a bare-metal platform, you can create a `NodeNetworkConfigurationPolicy` (NNCP) custom resource (CR) that includes an NMState configuration file. + +The Kubernetes NMState Operator uses the NMState configuration file to create a customized `br-ex` bridge network configuration on each node in your cluster. [IMPORTANT] ==== @@ -87,11 +89,11 @@ ifndef::postinstall-bare-metal[] [source,yaml] ---- interfaces: -- name: enp2s0 <1> - type: ethernet <2> - state: up <3> +- name: enp2s0 + type: ethernet + state: up ipv4: - enabled: false <4> + enabled: false ipv6: enabled: false - name: br-ex @@ -107,7 +109,7 @@ interfaces: options: mcast-snooping-enable: true port: - - name: enp2s0 <5> + - name: enp2s0 - name: br-ex - name: br-ex type: ovs-interface @@ -116,27 +118,33 @@ interfaces: ipv4: enabled: true dhcp: true - auto-route-metric: 48 <6> + auto-route-metric: 48 ipv6: enabled: true dhcp: true auto-route-metric: 48 # ... ---- -<1> Name of the interface. -<2> The type of ethernet. -<3> The requested state for the interface after creation. -<4> Disables IPv4 and IPv6 in this example. -<5> The node NIC to which the bridge attaches. -<6> Set the parameter to `48` to ensure the `br-ex` default route always has the highest precedence (lowest metric). This configuration prevents routing conflicts with any other interfaces that are automatically configured by the `NetworkManager` service. ++ +where: ++ +`interfaces.name`:: Name of the interface. +`interfaces.type`:: The type of ethernet. +`interfaces.state`:: The requested state for the interface after creation. +`ipv4.enabled`:: Disables IPv4 and IPv6 in this example. +`port.name`:: The node NIC to which the bridge attaches. +`auto-route-metric`:: Set the parameter to `48` to ensure the `br-ex` default route always has the highest precedence (lowest metric). This configuration prevents routing conflicts with any other interfaces that are automatically configured by the `NetworkManager` service. . Use the `cat` command to base64-encode the contents of the NMState configuration: + [source,terminal] ---- -$ cat .yaml | base64 <1> +$ cat .yml | base64 ---- -<1> Replace `` with the name of your NMState resource YAML file. ++ +where: ++ +``:: Replace `` with the name of your NMState resource YAML file. . Create a `MachineConfig` manifest file and define a customized `br-ex` bridge network configuration analogous to the following example: + @@ -147,7 +155,7 @@ kind: MachineConfig metadata: labels: machineconfiguration.openshift.io/role: worker - name: 10-br-ex-worker <1> + name: 10-br-ex-worker spec: config: ignition: @@ -155,20 +163,23 @@ spec: storage: files: - contents: - source: data:text/plain;charset=utf-8;base64, <2> + source: data:text/plain;charset=utf-8;base64, mode: 0644 overwrite: true - path: /etc/nmstate/openshift/worker-0.yml <3> + path: /etc/nmstate/openshift/worker-0.yml - contents: source: data:text/plain;charset=utf-8;base64, mode: 0644 overwrite: true - path: /etc/nmstate/openshift/worker-1.yml <3> + path: /etc/nmstate/openshift/worker-1.yml # ... ---- -<1> The name of the policy. -<2> Writes the encoded base64 information to the specified path. -<3> For each node in your cluster, specify the hostname path to your node and the base-64 encoded Ignition configuration file data for the machine type. The `worker` role is the default role for nodes in your cluster. The `.yaml` extension does not work when specifying the short hostname, `hostname -s`, path for each node or all nodes in the `MachineConfig` manifest file. ++ +where: ++ +`metadata.name`:: The name of the policy. +`contents.source`:: Writes the encoded base64 information to the specified path. +`path`:: For each node in your cluster, specify the hostname path to your node and the base-64 encoded Ignition configuration file data for the machine type. The `worker` role is the default role for nodes in your cluster. You must use the `.yml` extension for configuration files, such as `$(hostname -s).yml` when specifying the short hostname path for each node or all nodes in the `MachineConfig` manifest file. + If you have a single global configuration specified in an `/etc/nmstate/openshift/cluster.yml` configuration file that you want to apply to all nodes in your cluster, you do not need to specify the short hostname path for each node, such as `/etc/nmstate/openshift/.yml`. For example: + @@ -185,32 +196,29 @@ If you have a single global configuration specified in an `/etc/nmstate/openshif endif::postinstall-bare-metal[] ifdef::postinstall-bare-metal[] -* Create a `NodeNetworkConfigurationPolicy` (NNCP) CR and define a customized `br-ex` bridge network configuration. Depending on your needs, ensure that you set a masquerade IP for either the `ipv4.address.ip`, `ipv6.address.ip`, or both parameters. Always include a masquerade IP address in the NNCP CR and this address must match an in-use IP address block. +* Create a `NodeNetworkConfigurationPolicy` (NNCP) CR and define a customized `br-ex` bridge network configuration. The `br-ex` NNCP CR must include the OVN-Kubernetes masquerade IP address and subnet of your network. The example NNCP CR includes default values in the `ipv4.address.ip` and `ipv6.address.ip` parameters. You can set the masquerade IP address in the `ipv4.address.ip`, `ipv6.address.ip`, or both parameters. + [IMPORTANT] ==== -As a post-installation task, you can configure most parameters for a customized `br-ex` bridge that you defined in an existing NNCP CR, except for the primary IP address of the customized `br-ex` bridge. - -If you want to convert your single-stack cluster network to a dual-stack cluster network, you can add or change a secondary IPv6 address in the NNCP CR, but the existing primary IP address cannot be changed. +As a post-installation task, you cannot change the primary IP address of the customized `br-ex` bridge. If you want to convert your single-stack cluster network to a dual-stack cluster network, you can add or change a secondary IPv6 address in the NNCP CR, but the existing primary IP address cannot be changed. ==== + -.Example of an NNCP CR that sets IPv6 and IPv4 masquerade IP addresses [source,yaml] ---- apiVersion: nmstate.io/v1 kind: NodeNetworkConfigurationPolicy metadata: - name: worker-0-br-ex <1> + name: worker-0-br-ex spec: nodeSelector: kubernetes.io/hostname: worker-0 desiredState: interfaces: - - name: enp2s0 <2> - type: ethernet <3> - state: up <4> + - name: enp2s0 + type: ethernet + state: up ipv4: - enabled: false <5> + enabled: false ipv6: enabled: false - name: br-ex @@ -226,7 +234,7 @@ spec: options: mcast-snooping-enable: true port: - - name: enp2s0 <6> + - name: enp2s0 - name: br-ex - name: br-ex type: ovs-interface @@ -235,26 +243,30 @@ spec: ipv4: enabled: true dhcp: true - auto-route-metric: 48 <7> + auto-route-metric: 48 address: - - ip: "169.254.169.2" - prefix-length: 29 + - ip: "169.254.0.2" + prefix-length: 17 ipv6: enabled: true dhcp: true auto-route-metric: 48 address: - ip: "fd69::2" - prefix-length: 125 + prefix-length: 112 # ... ---- -<1> Name of the policy. -<2> Name of the interface. -<3> The type of ethernet. -<4> The requested state for the interface after creation. -<5> Disables IPv4 and IPv6 in this example. -<6> The node NIC to which the bridge is attached. -<7> Set the parameter to `48` to ensure the `br-ex` default route always has the highest precedence (lowest metric). This configuration prevents routing conflicts with any other interfaces that are automatically configured by the `NetworkManager` service. ++ +where: ++ +`metadata.name`:: Name of the policy. +`interfaces.name`:: Name of the interface. +`interfaces.type`:: The type of ethernet. +`interfaces.state`:: The requested state for the interface after creation. +`ipv4.enabled`:: Disables IPv4 and IPv6 in this example. +`port.name`:: The node NIC to which the bridge is attached. +`address.ip`:: Shows the default IPv4 and IPv6 IP addresses. Ensure that you set the masquerade IPv4 and IPv6 IP addresses of your network. +`auto-route-metric`:: Set the parameter to `48` to ensure the `br-ex` default route always has the highest precedence (lowest metric). This configuration prevents routing conflicts with any other interfaces that are automatically configured by the `NetworkManager` service. endif::postinstall-bare-metal[] .Next steps diff --git a/modules/ipi-install-additional-install-config-parameters.adoc b/modules/ipi-install-additional-install-config-parameters.adoc index 4bd961e15f8e..88c5be1a1ee0 100644 --- a/modules/ipi-install-additional-install-config-parameters.adoc +++ b/modules/ipi-install-additional-install-config-parameters.adoc @@ -136,7 +136,7 @@ a| `provisioningNetworkInterface` | | The name of the network interface on node | `apiVIPs` | a| (Optional) The virtual IP address for Kubernetes API communication. -You must either provide this setting in the `install-config.yaml` file as a reserved IP from the `MachineNetwork` parameter or preconfigured in the DNS so that the default name resolves correctly. Use the virtual IP address and not the FQDN when adding a value to the `apiVIPs` configuration setting in the `install-config.yaml` file. The primary IP address must be from the IPv4 network when using dual stack networking. If not set, the installation program uses `api..` to derive the IP address from the DNS. +You must either provide this setting in the `install-config.yaml` file as a reserved IP from the `MachineNetwork` parameter or preconfigured in the DNS so that the default name resolves correctly. Use the virtual IP address and not the FQDN when adding a value to the `apiVIPs` configuration setting in the `install-config.yaml` file. For dual-stack networking, the primary IP address can be either an IPv4 network or an IPv6 network. If not set, the installation program uses `api..` to derive the IP address from the DNS. [NOTE] ==== @@ -148,7 +148,7 @@ Before {product-title} 4.12, the cluster installation program only accepted an I | `ingressVIPs` | a| (Optional) The virtual IP address for ingress traffic. -You must either provide this setting in the `install-config.yaml` file as a reserved IP from the `MachineNetwork` parameter or preconfigured in the DNS so that the default name resolves correctly. Use the virtual IP address and not the FQDN when adding a value to the `ingressVIPs` configuration setting in the `install-config.yaml` file. The primary IP address must be from the IPv4 network when using dual stack networking. If not set, the installation program uses `test.apps..` to derive the IP address from the DNS. +You must either provide this setting in the `install-config.yaml` file as a reserved IP from the `MachineNetwork` parameter or preconfigured in the DNS so that the default name resolves correctly. Use the virtual IP address and not the FQDN when adding a value to the `ingressVIPs` configuration setting in the `install-config.yaml` file. For dual-stack networking, the primary IP address can be either an IPv4 network or an IPv6 network. If not set, the installation program uses `test.apps..` to derive the IP address from the DNS. [NOTE] ==== diff --git a/networking/k8s_nmstate/k8s-nmstate-updating-node-network-config.adoc b/networking/k8s_nmstate/k8s-nmstate-updating-node-network-config.adoc index ebca76783e45..ddb750a041f4 100644 --- a/networking/k8s_nmstate/k8s-nmstate-updating-node-network-config.adoc +++ b/networking/k8s_nmstate/k8s-nmstate-updating-node-network-config.adoc @@ -13,7 +13,7 @@ For more information about how to install the NMState Operator, see xref:../../n [IMPORTANT] ==== -You cannot provide any configuration that modifies the br-ex bridge, an OVN-Kubernetes-managed Open vSwitch bridge. However, you can configure a customized br-ex bridge. +You cannot modify an existing `br-ex` bridge, an OVN-Kubernetes-managed Open vSwitch bridge, or any interfaces, bonds, VLANs, and so on that associate with the `br-ex` bridge. However, you can configure a customized br-ex bridge. For more information, see "Creating a manifest object that includes a customized br-ex bridge" in the _Deploying installer-provisioned clusters on bare metal_ document or the _Installing a user-provisioned cluster on bare metal_ document. ====