Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion installing/installing_bare_metal/upi/installing-bare-metal.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@ In {product-title} {product-version}, you can install a cluster on bare-metal in

[IMPORTANT]
====
While you might be able to follow this procedure to deploy a cluster on virtualized or cloud environments, you must be aware of additional considerations for non-bare-metal platforms. Review the information in the link:https://access.redhat.com/articles/4207611[guidelines for deploying {product-title} on non-tested platforms] before you attempt to install an {product-title} cluster in such an environment.
While you might be able to follow this procedure to deploy a cluster on virtualized or cloud environments, you must be aware of additional
considerations for non-bare-metal platforms. Review the information in the link:https://access.redhat.com/articles/4207611[guidelines for deploying {product-title} on non-tested platforms] before you attempt to install an {product-title} cluster in such an environment.
====

== Prerequisites
Expand All @@ -24,6 +25,7 @@ While you might be able to follow this procedure to deploy a cluster on virtuali
Be sure to also review this site list if you are configuring a proxy.
====

// Internet access for OpenShift Container Platform
include::modules/cluster-entitlements.adoc[leveloffset=+1]

[role="_additional-resources"]
Expand All @@ -39,8 +41,10 @@ of the required machines.

This section describes the requirements for deploying {product-title} on user-provisioned infrastructure.

// Required machines for cluster installation
include::modules/installation-machine-requirements.adoc[leveloffset=+2]

// Minimum resource requirements for cluster installation
include::modules/installation-minimum-resource-requirements.adoc[leveloffset=+2]

[role="_additional-resources"]
Expand Down Expand Up @@ -81,6 +85,7 @@ include::modules/installation-dns-user-infra.adoc[leveloffset=+2]

* xref:../../../installing/installing_bare_metal/upi/installing-bare-metal.adoc#installation-user-provisioned-validating-dns_installing-bare-metal[Validating DNS resolution for user-provisioned infrastructure]

// Load balancing requirements for user-provisioned infrastructure
include::modules/installation-load-balancing-user-infra.adoc[leveloffset=+2]

// Creating a manifest object that includes a customized `br-ex` bridge
Expand All @@ -99,6 +104,7 @@ include::modules/creating-scaling-machine-sets-compute-nodes-networking.adoc[lev
// Enabling OVS balance-slb mode for your cluster
include::modules/enabling-OVS-balance-slb-mode.adoc[leveloffset=+1]

// Preparing the user-provisioned infrastructure
include::modules/installation-infrastructure-user-infra.adoc[leveloffset=+1]

[role="_additional-resources"]
Expand All @@ -113,6 +119,7 @@ include::modules/installation-infrastructure-user-infra.adoc[leveloffset=+1]
* xref:../../../installing/installing_bare_metal/upi/installing-bare-metal.adoc#installation-user-provisioned-validating-dns_installing-bare-metal[Validating DNS resolution for user-provisioned infrastructure]
* xref:../../../installing/installing_bare_metal/upi/installing-bare-metal.adoc#installation-load-balancing-user-infra_installing-bare-metal[Load balancing requirements for user-provisioned infrastructure]

// Validating DNS resolution for user-provisioned infrastructure
include::modules/installation-user-provisioned-validating-dns.adoc[leveloffset=+1]

[role="_additional-resources"]
Expand All @@ -121,23 +128,28 @@ include::modules/installation-user-provisioned-validating-dns.adoc[leveloffset=+
* xref:../../../installing/installing_bare_metal/upi/installing-bare-metal.adoc#installation-dns-user-infra_installing-bare-metal[User-provisioned DNS requirements]
* xref:../../../installing/installing_bare_metal/upi/installing-bare-metal.adoc#installation-load-balancing-user-infra_installing-bare-metal[Load balancing requirements for user-provisioned infrastructure]

// Generating a key pair for cluster node SSH access
include::modules/ssh-agent-using.adoc[leveloffset=+1]

[role="_additional-resources"]
.Additional resources

* xref:../../../support/troubleshooting/verifying-node-health.adoc#verifying-node-health[Verifying node health]

// Obtaining the installation program
include::modules/installation-obtaining-installer.adoc[leveloffset=+1]

// Installing the OpenShift CLI
include::modules/cli-installing-cli.adoc[leveloffset=+1]

// Manually creating the installation configuration file
include::modules/installation-initializing-manual.adoc[leveloffset=+1]

[role="_additional-resources"]
.Additional resources
* xref:../../../installing/installing_bare_metal/upi/installation-config-parameters-bare-metal.adoc#installation-config-parameters-bare-metal[Installation configuration parameters for bare metal]

// Sample install-config.yaml file for bare metal
include::modules/installation-bare-metal-config-yaml.adoc[leveloffset=+2]

[role="_additional-resources"]
Expand Down
147 changes: 75 additions & 72 deletions modules/creating-manifest-file-customized-br-ex-bridge.adoc
Original file line number Diff line number Diff line change
@@ -1,58 +1,54 @@
// Module included in the following assemblies:
//
// IPI
// * installing/installing_bare_metal/ipi/ipi-install-installation-workflow.adoc
// * installing/installing_bare_metal/bare-metal-postinstallation-configuration.adoc
// UPI
// * installing/installing_bare_metal/upi/installing-bare-metal-network-customizations.adoc
// * installing/installing_bare_metal/upi/installing-restricted-networks-bare-metal.adoc
// * installing/installing_bare_metal/upi/installing-bare-metal.adoc

ifeval::["{context}" == "ipi-install-post-installation-configuration"]
:postinstall-bare-metal-ipi:
endif::[]
ifeval::["{context}" == "post-install-bare-metal-configuration"]
:postinstall-bare-metal-upi:
ifeval::["{context}" == "bare-metal-postinstallation-configuration"]
:postinstall-bare-metal:
endif::[]

:_mod-docs-content-type: PROCEDURE
[id="creating-manifest-file-customized-br-ex-bridge_{context}"]
= Creating a manifest object that includes a customized `br-ex` bridge

ifndef::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[]
ifndef::postinstall-bare-metal[]
As an alternative to using the `configure-ovs.sh` shell script to set a `br-ex` bridge on a bare-metal platform, you can create a `MachineConfig` object that includes an NMState configuration file. The host `nmstate-configuration.service` and `nmstate.service` apply the NMState configuration file to each node that runs in your cluster.
endif::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[]
endif::postinstall-bare-metal[]

ifdef::postinstall-bare-metal[]
As an alternative to using the `configure-ovs.sh` shell script to set a `br-ex` bridge on a bare-metal platform, you can create a `NodeNetworkConfigurationPolicy` (NNCP) custom resource (CR) that includes an NMState configuration file.

ifdef::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[]
As an alternative to using the `configure-ovs.sh` shell script to set a `br-ex` bridge on a bare-metal platform, you can create a `NodeNetworkConfigurationPolicy` (NNCP) custom resource (CR) that includes an NMState configuration file. The Kubernetes NMState Operator uses the NMState configuration file to create a customized `br-ex` bridge network configuration on each node in your cluster.
The Kubernetes NMState Operator uses the NMState configuration file to create a customized `br-ex` bridge network configuration on each node in your cluster.

[IMPORTANT]
====
After creating the `NodeNetworkConfigurationPolicy` CR, copy content from the NMState configuration file that was created during cluster installation into the NNCP CR. An incomplete NNCP CR file means that the the network policy described in the file cannot get applied to nodes in the cluster.
After creating the `NodeNetworkConfigurationPolicy` CR, copy content from the NMState configuration file that was created during cluster installation into the NNCP CR. An incomplete NNCP CR file means that the network policy described in the file cannot get applied to nodes in the cluster.
====

This feature supports the following tasks:

* Modifying the maximum transmission unit (MTU) for your cluster.
* Modifying attributes of a different bond interface, such as MIImon (Media Independent Interface Monitor), bonding mode, or Quality of Service (QoS).
* Updating DNS values.
endif::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[]
endif::postinstall-bare-metal[]

Consider the following use cases for creating a manifest object that includes a customized `br-ex` bridge:

* You want to make postinstallation changes to the bridge, such as changing the Open vSwitch (OVS) or OVN-Kubernetes `br-ex` bridge network. The `configure-ovs.sh` shell script does not support making postinstallation changes to the bridge.
* You want to deploy the bridge on a different interface than the interface available on a host or server IP address.
* You want to make advanced configurations to the bridge that are not possible with the `configure-ovs.sh` shell script. Using the script for these configurations might result in the bridge failing to connect multiple network interfaces and facilitating data forwarding between the interfaces.

ifndef::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[]

ifndef::postinstall-bare-metal[]
[NOTE]
====
If you require an environment with a single network interface controller (NIC) and default network settings, use the `configure-ovs.sh` shell script.
====

After you install {op-system-first} and the system reboots, the Machine Config Operator injects Ignition configuration files into each node in your cluster, so that each node received the `br-ex` bridge network configuration. To prevent configuration conflicts, the `configure-ovs.sh` shell script receives a signal to not configure the `br-ex` bridge.
endif::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[]
endif::postinstall-bare-metal[]

[WARNING]
====
Expand All @@ -75,29 +71,29 @@ The following list of interface names are reserved and you cannot use the names
====

.Prerequisites
ifndef::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[]
ifndef::postinstall-bare-metal[]
* Optional: You have installed the link:https://nmstate.io/[`nmstate`] API so that you can validate the NMState configuration.
endif::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[]
endif::postinstall-bare-metal[]

ifdef::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[]
ifdef::postinstall-bare-metal[]
* You set a customized `br-ex` by using the alternative method to `configure-ovs`.
* You installed the Kubernetes NMState Operator.
endif::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[]
endif::postinstall-bare-metal[]

.Procedure

ifndef::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[]
ifndef::postinstall-bare-metal[]
. Create a NMState configuration file that has decoded base64 information for your customized `br-ex` bridge network:
+
.Example of an NMState configuration for a customized `br-ex` bridge network
[source,yaml]
----
interfaces:
- name: enp2s0 <1>
type: ethernet <2>
state: up <3>
- name: enp2s0
type: ethernet
state: up
ipv4:
enabled: false <4>
enabled: false
ipv6:
enabled: false
- name: br-ex
Expand All @@ -113,7 +109,7 @@ interfaces:
options:
mcast-snooping-enable: true
port:
- name: enp2s0 <5>
- name: enp2s0
- name: br-ex
- name: br-ex
type: ovs-interface
Expand All @@ -122,27 +118,33 @@ interfaces:
ipv4:
enabled: true
dhcp: true
auto-route-metric: 48 <6>
auto-route-metric: 48
ipv6:
enabled: true
dhcp: true
auto-route-metric: 48
# ...
----
<1> Name of the interface.
<2> The type of ethernet.
<3> The requested state for the interface after creation.
<4> Disables IPv4 and IPv6 in this example.
<5> The node NIC to which the bridge attaches.
<6> Set the parameter to `48` to ensure the `br-ex` default route always has the highest precedence (lowest metric). This configuration prevents routing conflicts with any other interfaces that are automatically configured by the `NetworkManager` service.
+
where:
+
`interfaces.name`:: Name of the interface.
`interfaces.type`:: The type of ethernet.
`interfaces.state`:: The requested state for the interface after creation.
`ipv4.enabled`:: Disables IPv4 and IPv6 in this example.
`port.name`:: The node NIC to which the bridge attaches.
`auto-route-metric`:: Set the parameter to `48` to ensure the `br-ex` default route always has the highest precedence (lowest metric). This configuration prevents routing conflicts with any other interfaces that are automatically configured by the `NetworkManager` service.

. Use the `cat` command to base64-encode the contents of the NMState configuration:
+
[source,terminal]
----
$ cat <nmstate_configuration>.yaml | base64 <1>
$ cat <nmstate_configuration>.yml | base64
----
<1> Replace `<nmstate_configuration>` with the name of your NMState resource YAML file.
+
where:
+
`<nmstate_configuration>`:: Replace `<nmstate_configuration>` with the name of your NMState resource YAML file.

. Create a `MachineConfig` manifest file and define a customized `br-ex` bridge network configuration analogous to the following example:
+
Expand All @@ -153,28 +155,31 @@ kind: MachineConfig
metadata:
labels:
machineconfiguration.openshift.io/role: worker
name: 10-br-ex-worker <1>
name: 10-br-ex-worker
spec:
config:
ignition:
version: 3.2.0
storage:
files:
- contents:
source: data:text/plain;charset=utf-8;base64,<base64_encoded_nmstate_configuration> <2>
source: data:text/plain;charset=utf-8;base64,<base64_encoded_nmstate_configuration>
mode: 0644
overwrite: true
path: /etc/nmstate/openshift/worker-0.yml <3>
path: /etc/nmstate/openshift/worker-0.yml
- contents:
source: data:text/plain;charset=utf-8;base64,<base64_encoded_nmstate_configuration>
mode: 0644
overwrite: true
path: /etc/nmstate/openshift/worker-1.yml <3>
path: /etc/nmstate/openshift/worker-1.yml
# ...
----
<1> The name of the policy.
<2> Writes the encoded base64 information to the specified path.
<3> For each node in your cluster, specify the hostname path to your node and the base-64 encoded Ignition configuration file data for the machine type. The `worker` role is the default role for nodes in your cluster. The `.yaml` extension does not work when specifying the short hostname, `hostname -s`, path for each node or all nodes in the `MachineConfig` manifest file.
+
where:
+
`metadata.name`:: The name of the policy.
`contents.source`:: Writes the encoded base64 information to the specified path.
`path`:: For each node in your cluster, specify the hostname path to your node and the base-64 encoded Ignition configuration file data for the machine type. The `worker` role is the default role for nodes in your cluster. You must use the `.yml` extension for configuration files, such as `$(hostname -s).yml` when specifying the short hostname path for each node or all nodes in the `MachineConfig` manifest file.
+
If you have a single global configuration specified in an `/etc/nmstate/openshift/cluster.yml` configuration file that you want to apply to all nodes in your cluster, you do not need to specify the short hostname path for each node, such as `/etc/nmstate/openshift/<node_hostname>.yml`. For example:
+
Expand All @@ -188,35 +193,32 @@ If you have a single global configuration specified in an `/etc/nmstate/openshif
path: /etc/nmstate/openshift/cluster.yml
# ...
----
endif::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[]
endif::postinstall-bare-metal[]

ifdef::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[]
* Create a `NodeNetworkConfigurationPolicy` (NNCP) CR and define a customized `br-ex` bridge network configuration. Depending on your needs, ensure that you set a masquerade IP for either the `ipv4.address.ip`, `ipv6.address.ip`, or both parameters. Always include a masquerade IP address in the NNCP CR and this address must match an in-use IP address block.
ifdef::postinstall-bare-metal[]
* Create a `NodeNetworkConfigurationPolicy` (NNCP) CR and define a customized `br-ex` bridge network configuration. The `br-ex` NNCP CR must include the OVN-Kubernetes masquerade IP address and subnet of your network. The example NNCP CR includes default values in the `ipv4.address.ip` and `ipv6.address.ip` parameters. You can set the masquerade IP address in the `ipv4.address.ip`, `ipv6.address.ip`, or both parameters.
+
[IMPORTANT]
====
As a post-installation task, you can configure most parameters for a customized `br-ex` bridge that you defined in an existing NNCP CR, except for the primary IP address of the customized `br-ex` bridge.

If you want to convert your single-stack cluster network to a dual-stack cluster network, you can add or change a secondary IPv6 address in the NNCP CR, but the existing primary IP address cannot be changed.
As a post-installation task, you cannot change the primary IP address of the customized `br-ex` bridge. If you want to convert your single-stack cluster network to a dual-stack cluster network, you can add or change a secondary IPv6 address in the NNCP CR, but the existing primary IP address cannot be changed.
====
+
.Example of an NNCP CR that sets IPv6 and IPv4 masquerade IP addresses
[source,yaml]
----
apiVersion: nmstate.io/v1
kind: NodeNetworkConfigurationPolicy
metadata:
name: worker-0-br-ex <1>
name: worker-0-br-ex
spec:
nodeSelector:
kubernetes.io/hostname: worker-0
desiredState:
interfaces:
- name: enp2s0 <2>
type: ethernet <3>
state: up <4>
- name: enp2s0
type: ethernet
state: up
ipv4:
enabled: false <5>
enabled: false
ipv6:
enabled: false
- name: br-ex
Expand All @@ -232,7 +234,7 @@ spec:
options:
mcast-snooping-enable: true
port:
- name: enp2s0 <6>
- name: enp2s0
- name: br-ex
- name: br-ex
type: ovs-interface
Expand All @@ -241,35 +243,36 @@ spec:
ipv4:
enabled: true
dhcp: true
auto-route-metric: 48 <7>
auto-route-metric: 48
address:
- ip: "169.254.169.2"
prefix-length: 29
- ip: "169.254.0.2"
prefix-length: 17
ipv6:
enabled: true
dhcp: true
auto-route-metric: 48
address:
- ip: "fd69::2"
prefix-length: 125
prefix-length: 112
# ...
----
<1> Name of the policy.
<2> Name of the interface.
<3> The type of ethernet.
<4> The requested state for the interface after creation.
<5> Disables IPv4 and IPv6 in this example.
<6> The node NIC to which the bridge is attached.
<7> Set the parameter to `48` to ensure the `br-ex` default route always has the highest precedence (lowest metric). This configuration prevents routing conflicts with any other interfaces that are automatically configured by the `NetworkManager` service.
endif::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[]
+
where:
+
`metadata.name`:: Name of the policy.
`interfaces.name`:: Name of the interface.
`interfaces.type`:: The type of ethernet.
`interfaces.state`:: The requested state for the interface after creation.
`ipv4.enabled`:: Disables IPv4 and IPv6 in this example.
`port.name`:: The node NIC to which the bridge is attached.
`address.ip`:: Shows the default IPv4 and IPv6 IP addresses. Ensure that you set the masquerade IPv4 and IPv6 IP addresses of your network.
`auto-route-metric`:: Set the parameter to `48` to ensure the `br-ex` default route always has the highest precedence (lowest metric). This configuration prevents routing conflicts with any other interfaces that are automatically configured by the `NetworkManager` service.
endif::postinstall-bare-metal[]

.Next steps

* Scaling compute nodes to apply the manifest object that includes a customized `br-ex` bridge to each compute node that exists in your cluster. For more information, see "Expanding the cluster" in the _Additional resources_ section.

ifeval::["{context}" == "ipi-install-post-installation-configuration"]
:!postinstall-bare-metal-ipi:
endif::[]
ifeval::["{context}" == "bare-metal-configuration"]
:!postinstall-bare-metal-upi:
ifeval::["{context}" == "bare-metal-postinstallation-configuration"]
:!postinstall-bare-metal:
endif::[]
Loading