Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 14 additions & 7 deletions installing/installing_bare_metal/upi/installing-bare-metal.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,12 @@ include::_attributes/common-attributes.adoc[]

toc::[]

In {product-title} {product-version}, you can install a cluster on
bare metal infrastructure that you provision.
In {product-title} {product-version}, you can install a cluster on bare-metal infrastructure that you provision.

[IMPORTANT]
====
While you might be able to follow this procedure to deploy a cluster on
virtualized or cloud environments, you must be aware of additional
considerations for non-bare metal platforms. Review the information in the
link:https://access.redhat.com/articles/4207611[guidelines for deploying {product-title} on non-tested platforms]
before you attempt to install an {product-title} cluster in such an environment.
While you might be able to follow this procedure to deploy a cluster on virtualized or cloud environments, you must be aware of additional
considerations for non-bare-metal platforms. Review the information in the link:https://access.redhat.com/articles/4207611[guidelines for deploying {product-title} on non-tested platforms] before you attempt to install an {product-title} cluster in such an environment.
====

== Prerequisites
Expand All @@ -29,6 +25,7 @@ before you attempt to install an {product-title} cluster in such an environment.
Be sure to also review this site list if you are configuring a proxy.
====

// Internet access for OpenShift Container Platform
include::modules/cluster-entitlements.adoc[leveloffset=+1]

[role="_additional-resources"]
Expand All @@ -44,8 +41,10 @@ of the required machines.

This section describes the requirements for deploying {product-title} on user-provisioned infrastructure.

// Required machines for cluster installation
include::modules/installation-machine-requirements.adoc[leveloffset=+2]

// Minimum resource requirements for cluster installation
include::modules/installation-minimum-resource-requirements.adoc[leveloffset=+2]

[role="_additional-resources"]
Expand Down Expand Up @@ -86,6 +85,7 @@ include::modules/installation-dns-user-infra.adoc[leveloffset=+2]

* xref:../../../installing/installing_bare_metal/upi/installing-bare-metal.adoc#installation-user-provisioned-validating-dns_installing-bare-metal[Validating DNS resolution for user-provisioned infrastructure]

// Load balancing requirements for user-provisioned infrastructure
include::modules/installation-load-balancing-user-infra.adoc[leveloffset=+2]

// Creating a manifest object that includes a customized `br-ex` bridge
Expand All @@ -104,6 +104,7 @@ include::modules/creating-scaling-machine-sets-compute-nodes-networking.adoc[lev
// Enabling OVS balance-slb mode for your cluster
include::modules/enabling-OVS-balance-slb-mode.adoc[leveloffset=+1]

// Preparing the user-provisioned infrastructure
include::modules/installation-infrastructure-user-infra.adoc[leveloffset=+1]

[role="_additional-resources"]
Expand All @@ -118,6 +119,7 @@ include::modules/installation-infrastructure-user-infra.adoc[leveloffset=+1]
* xref:../../../installing/installing_bare_metal/upi/installing-bare-metal.adoc#installation-user-provisioned-validating-dns_installing-bare-metal[Validating DNS resolution for user-provisioned infrastructure]
* xref:../../../installing/installing_bare_metal/upi/installing-bare-metal.adoc#installation-load-balancing-user-infra_installing-bare-metal[Load balancing requirements for user-provisioned infrastructure]

// Validating DNS resolution for user-provisioned infrastructure
include::modules/installation-user-provisioned-validating-dns.adoc[leveloffset=+1]

[role="_additional-resources"]
Expand All @@ -126,23 +128,28 @@ include::modules/installation-user-provisioned-validating-dns.adoc[leveloffset=+
* xref:../../../installing/installing_bare_metal/upi/installing-bare-metal.adoc#installation-dns-user-infra_installing-bare-metal[User-provisioned DNS requirements]
* xref:../../../installing/installing_bare_metal/upi/installing-bare-metal.adoc#installation-load-balancing-user-infra_installing-bare-metal[Load balancing requirements for user-provisioned infrastructure]

// Generating a key pair for cluster node SSH access
include::modules/ssh-agent-using.adoc[leveloffset=+1]

[role="_additional-resources"]
.Additional resources

* xref:../../../support/troubleshooting/verifying-node-health.adoc#verifying-node-health[Verifying node health]

// Obtaining the installation program
include::modules/installation-obtaining-installer.adoc[leveloffset=+1]

// Installing the OpenShift CLI
include::modules/cli-installing-cli.adoc[leveloffset=+1]

// Manually creating the installation configuration file
include::modules/installation-initializing-manual.adoc[leveloffset=+1]

[role="_additional-resources"]
.Additional resources
* xref:../../../installing/installing_bare_metal/upi/installation-config-parameters-bare-metal.adoc#installation-config-parameters-bare-metal[Installation configuration parameters for bare metal]

// Sample install-config.yaml file for bare metal
include::modules/installation-bare-metal-config-yaml.adoc[leveloffset=+2]

[role="_additional-resources"]
Expand Down
100 changes: 56 additions & 44 deletions modules/creating-manifest-file-customized-br-ex-bridge.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,9 @@ As an alternative to using the `configure-ovs.sh` shell script to set a `br-ex`
endif::postinstall-bare-metal[]

ifdef::postinstall-bare-metal[]
As an alternative to using the `configure-ovs.sh` shell script to set a `br-ex` bridge on a bare-metal platform, you can create a `NodeNetworkConfigurationPolicy` (NNCP) custom resource (CR) that includes an NMState configuration file. The Kubernetes NMState Operator uses the NMState configuration file to create a customized `br-ex` bridge network configuration on each node in your cluster.
As an alternative to using the `configure-ovs.sh` shell script to set a `br-ex` bridge on a bare-metal platform, you can create a `NodeNetworkConfigurationPolicy` (NNCP) custom resource (CR) that includes an NMState configuration file.

The Kubernetes NMState Operator uses the NMState configuration file to create a customized `br-ex` bridge network configuration on each node in your cluster.

[IMPORTANT]
====
Expand Down Expand Up @@ -87,11 +89,11 @@ ifndef::postinstall-bare-metal[]
[source,yaml]
----
interfaces:
- name: enp2s0 <1>
type: ethernet <2>
state: up <3>
- name: enp2s0
type: ethernet
state: up
ipv4:
enabled: false <4>
enabled: false
ipv6:
enabled: false
- name: br-ex
Expand All @@ -107,7 +109,7 @@ interfaces:
options:
mcast-snooping-enable: true
port:
- name: enp2s0 <5>
- name: enp2s0
- name: br-ex
- name: br-ex
type: ovs-interface
Expand All @@ -116,27 +118,33 @@ interfaces:
ipv4:
enabled: true
dhcp: true
auto-route-metric: 48 <6>
auto-route-metric: 48
ipv6:
enabled: true
dhcp: true
auto-route-metric: 48
# ...
----
<1> Name of the interface.
<2> The type of ethernet.
<3> The requested state for the interface after creation.
<4> Disables IPv4 and IPv6 in this example.
<5> The node NIC to which the bridge attaches.
<6> Set the parameter to `48` to ensure the `br-ex` default route always has the highest precedence (lowest metric). This configuration prevents routing conflicts with any other interfaces that are automatically configured by the `NetworkManager` service.
+
where:
+
`interfaces.name`:: Name of the interface.
`interfaces.type`:: The type of ethernet.
`interfaces.state`:: The requested state for the interface after creation.
`ipv4.enabled`:: Disables IPv4 and IPv6 in this example.
`port.name`:: The node NIC to which the bridge attaches.
`auto-route-metric`:: Set the parameter to `48` to ensure the `br-ex` default route always has the highest precedence (lowest metric). This configuration prevents routing conflicts with any other interfaces that are automatically configured by the `NetworkManager` service.

. Use the `cat` command to base64-encode the contents of the NMState configuration:
+
[source,terminal]
----
$ cat <nmstate_configuration>.yaml | base64 <1>
$ cat <nmstate_configuration>.yml | base64
----
<1> Replace `<nmstate_configuration>` with the name of your NMState resource YAML file.
+
where:
+
`<nmstate_configuration>`:: Replace `<nmstate_configuration>` with the name of your NMState resource YAML file.

. Create a `MachineConfig` manifest file and define a customized `br-ex` bridge network configuration analogous to the following example:
+
Expand All @@ -147,28 +155,31 @@ kind: MachineConfig
metadata:
labels:
machineconfiguration.openshift.io/role: worker
name: 10-br-ex-worker <1>
name: 10-br-ex-worker
spec:
config:
ignition:
version: 3.2.0
storage:
files:
- contents:
source: data:text/plain;charset=utf-8;base64,<base64_encoded_nmstate_configuration> <2>
source: data:text/plain;charset=utf-8;base64,<base64_encoded_nmstate_configuration>
mode: 0644
overwrite: true
path: /etc/nmstate/openshift/worker-0.yml <3>
path: /etc/nmstate/openshift/worker-0.yml
- contents:
source: data:text/plain;charset=utf-8;base64,<base64_encoded_nmstate_configuration>
mode: 0644
overwrite: true
path: /etc/nmstate/openshift/worker-1.yml <3>
path: /etc/nmstate/openshift/worker-1.yml
# ...
----
<1> The name of the policy.
<2> Writes the encoded base64 information to the specified path.
<3> For each node in your cluster, specify the hostname path to your node and the base-64 encoded Ignition configuration file data for the machine type. The `worker` role is the default role for nodes in your cluster. The `.yaml` extension does not work when specifying the short hostname, `hostname -s`, path for each node or all nodes in the `MachineConfig` manifest file.
+
where:
+
`metadata.name`:: The name of the policy.
`contents.source`:: Writes the encoded base64 information to the specified path.
`path`:: For each node in your cluster, specify the hostname path to your node and the base-64 encoded Ignition configuration file data for the machine type. The `worker` role is the default role for nodes in your cluster. You must use the `.yml` extension for configuration files, such as `$(hostname -s).yml` when specifying the short hostname path for each node or all nodes in the `MachineConfig` manifest file.
+
If you have a single global configuration specified in an `/etc/nmstate/openshift/cluster.yml` configuration file that you want to apply to all nodes in your cluster, you do not need to specify the short hostname path for each node, such as `/etc/nmstate/openshift/<node_hostname>.yml`. For example:
+
Expand All @@ -192,32 +203,29 @@ $ oc apply -f <machine_config>.yml
endif::postinstall-bare-metal[]

ifdef::postinstall-bare-metal[]
* Create a `NodeNetworkConfigurationPolicy` (NNCP) CR and define a customized `br-ex` bridge network configuration. Depending on your needs, ensure that you set a masquerade IP for either the `ipv4.address.ip`, `ipv6.address.ip`, or both parameters. Always include a masquerade IP address in the NNCP CR and this address must match an in-use IP address block.
* Create a `NodeNetworkConfigurationPolicy` (NNCP) CR and define a customized `br-ex` bridge network configuration. The `br-ex` NNCP CR must include the OVN-Kubernetes masquerade IP address and subnet of your network. The example NNCP CR includes default values in the `ipv4.address.ip` and `ipv6.address.ip` parameters. You can set the masquerade IP address in the `ipv4.address.ip`, `ipv6.address.ip`, or both parameters.
+
[IMPORTANT]
====
As a post-installation task, you can configure most parameters for a customized `br-ex` bridge that you defined in an existing NNCP CR, except for the primary IP address of the customized `br-ex` bridge.

If you want to convert your single-stack cluster network to a dual-stack cluster network, you can add or change a secondary IPv6 address in the NNCP CR, but the existing primary IP address cannot be changed.
As a post-installation task, you cannot change the primary IP address of the customized `br-ex` bridge. If you want to convert your single-stack cluster network to a dual-stack cluster network, you can add or change a secondary IPv6 address in the NNCP CR, but the existing primary IP address cannot be changed.
====
+
.Example of an NNCP CR that sets IPv6 and IPv4 masquerade IP addresses
[source,yaml]
----
apiVersion: nmstate.io/v1
kind: NodeNetworkConfigurationPolicy
metadata:
name: worker-0-br-ex <1>
name: worker-0-br-ex
spec:
nodeSelector:
kubernetes.io/hostname: worker-0
desiredState:
interfaces:
- name: enp2s0 <2>
type: ethernet <3>
state: up <4>
- name: enp2s0
type: ethernet
state: up
ipv4:
enabled: false <5>
enabled: false
ipv6:
enabled: false
- name: br-ex
Expand All @@ -233,7 +241,7 @@ spec:
options:
mcast-snooping-enable: true
port:
- name: enp2s0 <6>
- name: enp2s0
- name: br-ex
- name: br-ex
type: ovs-interface
Expand All @@ -242,26 +250,30 @@ spec:
ipv4:
enabled: true
dhcp: true
auto-route-metric: 48 <7>
auto-route-metric: 48
address:
- ip: "169.254.169.2"
prefix-length: 29
- ip: "169.254.0.2"
prefix-length: 17
ipv6:
enabled: true
dhcp: true
auto-route-metric: 48
address:
- ip: "fd69::2"
prefix-length: 125
prefix-length: 112
# ...
----
<1> Name of the policy.
<2> Name of the interface.
<3> The type of ethernet.
<4> The requested state for the interface after creation.
<5> Disables IPv4 and IPv6 in this example.
<6> The node NIC to which the bridge is attached.
<7> Set the parameter to `48` to ensure the `br-ex` default route always has the highest precedence (lowest metric). This configuration prevents routing conflicts with any other interfaces that are automatically configured by the `NetworkManager` service.
+
where:
+
`metadata.name`:: Name of the policy.
`interfaces.name`:: Name of the interface.
`interfaces.type`:: The type of ethernet.
`interfaces.state`:: The requested state for the interface after creation.
`ipv4.enabled`:: Disables IPv4 and IPv6 in this example.
`port.name`:: The node NIC to which the bridge is attached.
`address.ip`:: Shows the default IPv4 and IPv6 IP addresses. Ensure that you set the masquerade IPv4 and IPv6 IP addresses of your network.
`auto-route-metric`:: Set the parameter to `48` to ensure the `br-ex` default route always has the highest precedence (lowest metric). This configuration prevents routing conflicts with any other interfaces that are automatically configured by the `NetworkManager` service.
endif::postinstall-bare-metal[]

.Next steps
Expand Down
4 changes: 2 additions & 2 deletions modules/ipi-install-additional-install-config-parameters.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ a| `provisioningNetworkInterface` | | The name of the network interface on node

| `apiVIPs` | a| (Optional) The virtual IP address for Kubernetes API communication.

You must either provide this setting in the `install-config.yaml` file as a reserved IP from the `MachineNetwork` parameter or preconfigured in the DNS so that the default name resolves correctly. Use the virtual IP address and not the FQDN when adding a value to the `apiVIPs` configuration setting in the `install-config.yaml` file. The primary IP address must be from the IPv4 network when using dual stack networking. If not set, the installation program uses `api.<cluster_name>.<base_domain>` to derive the IP address from the DNS.
You must either provide this setting in the `install-config.yaml` file as a reserved IP from the `MachineNetwork` parameter or preconfigured in the DNS so that the default name resolves correctly. Use the virtual IP address and not the FQDN when adding a value to the `apiVIPs` configuration setting in the `install-config.yaml` file. For dual-stack networking, the primary IP address can be either an IPv4 network or an IPv6 network. If not set, the installation program uses `api.<cluster_name>.<base_domain>` to derive the IP address from the DNS.

[NOTE]
====
Expand All @@ -148,7 +148,7 @@ Before {product-title} 4.12, the cluster installation program only accepted an I

| `ingressVIPs` | a| (Optional) The virtual IP address for ingress traffic.

You must either provide this setting in the `install-config.yaml` file as a reserved IP from the `MachineNetwork` parameter or preconfigured in the DNS so that the default name resolves correctly. Use the virtual IP address and not the FQDN when adding a value to the `ingressVIPs` configuration setting in the `install-config.yaml` file. The primary IP address must be from the IPv4 network when using dual stack networking. If not set, the installation program uses `test.apps.<cluster_name>.<base_domain>` to derive the IP address from the DNS.
You must either provide this setting in the `install-config.yaml` file as a reserved IP from the `MachineNetwork` parameter or preconfigured in the DNS so that the default name resolves correctly. Use the virtual IP address and not the FQDN when adding a value to the `ingressVIPs` configuration setting in the `install-config.yaml` file. For dual-stack networking, the primary IP address can be either an IPv4 network or an IPv6 network. If not set, the installation program uses `test.apps.<cluster_name>.<base_domain>` to derive the IP address from the DNS.

[NOTE]
====
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ For more information about how to install the NMState Operator, see xref:../../n

[IMPORTANT]
====
You cannot provide any configuration that modifies the br-ex bridge, an OVN-Kubernetes-managed Open vSwitch bridge. However, you can configure a customized br-ex bridge.
You cannot modify an existing `br-ex` bridge, an OVN-Kubernetes-managed Open vSwitch bridge, or any interfaces, bonds, VLANs, and so on that associate with the `br-ex` bridge. However, you can configure a customized br-ex bridge.

For more information, see "Creating a manifest object that includes a customized br-ex bridge" in the _Deploying installer-provisioned clusters on bare metal_ document or the _Installing a user-provisioned cluster on bare metal_ document.
====
Expand Down