Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
107 changes: 63 additions & 44 deletions modules/creating-manifest-file-customized-br-ex-bridge.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,9 @@ As an alternative to using the `configure-ovs.sh` shell script to set a `br-ex`
endif::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[]

ifdef::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[]
As an alternative to using the `configure-ovs.sh` shell script to set a `br-ex` bridge on a bare-metal platform, you can create a `NodeNetworkConfigurationPolicy` (NNCP) custom resource (CR) that includes an NMState configuration file. The Kubernetes NMState Operator uses the NMState configuration file to create a customized `br-ex` bridge network configuration on each node in your cluster.
As an alternative to using the `configure-ovs.sh` shell script to set a `br-ex` bridge on a bare-metal platform, you can create a `NodeNetworkConfigurationPolicy` (NNCP) custom resource (CR) that includes an NMState configuration file.

The Kubernetes NMState Operator uses the NMState configuration file to create a customized `br-ex` bridge network configuration on each node in your cluster.

[IMPORTANT]
====
Expand Down Expand Up @@ -92,11 +94,11 @@ ifndef::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[]
[source,yaml]
----
interfaces:
- name: enp2s0 <1>
type: ethernet <2>
state: up <3>
- name: enp2s0
type: ethernet
state: up
ipv4:
enabled: false <4>
enabled: false
ipv6:
enabled: false
- name: br-ex
Expand All @@ -112,7 +114,7 @@ interfaces:
options:
mcast-snooping-enable: true
port:
- name: enp2s0 <5>
- name: enp2s0
- name: br-ex
- name: br-ex
type: ovs-interface
Expand All @@ -121,27 +123,33 @@ interfaces:
ipv4:
enabled: true
dhcp: true
auto-route-metric: 48 <6>
auto-route-metric: 48
ipv6:
enabled: true
dhcp: true
auto-route-metric: 48
# ...
----
<1> Name of the interface.
<2> The type of ethernet.
<3> The requested state for the interface after creation.
<4> Disables IPv4 and IPv6 in this example.
<5> The node NIC to which the bridge attaches.
<6> Set the parameter to `48` to ensure the `br-ex` default route always has the highest precedence (lowest metric). This configuration prevents routing conflicts with any other interfaces that are automatically configured by the `NetworkManager` service.
+
where:
+
`interfaces.name`:: Name of the interface.
`interfaces.type`:: The type of ethernet.
`interfaces.state`:: The requested state for the interface after creation.
`ipv4.enabled`:: Disables IPv4 and IPv6 in this example.
`port.name`:: The node NIC to which the bridge attaches.
`auto-route-metric`:: Set the parameter to `48` to ensure the `br-ex` default route always has the highest precedence (lowest metric). This configuration prevents routing conflicts with any other interfaces that are automatically configured by the `NetworkManager` service.

. Use the `cat` command to base64-encode the contents of the NMState configuration:
+
[source,terminal]
----
$ cat <nmstate_configuration>.yaml | base64 <1>
$ cat <nmstate_configuration>.yml | base64
----
<1> Replace `<nmstate_configuration>` with the name of your NMState resource YAML file.
+
where:
+
`<nmstate_configuration>`:: Replace `<nmstate_configuration>` with the name of your NMState resource YAML file.

. Create a `MachineConfig` manifest file and define a customized `br-ex` bridge network configuration analogous to the following example:
+
Expand All @@ -152,28 +160,31 @@ kind: MachineConfig
metadata:
labels:
machineconfiguration.openshift.io/role: worker
name: 10-br-ex-worker <1>
name: 10-br-ex-worker
spec:
config:
ignition:
version: 3.2.0
storage:
files:
- contents:
source: data:text/plain;charset=utf-8;base64,<base64_encoded_nmstate_configuration> <2>
source: data:text/plain;charset=utf-8;base64,<base64_encoded_nmstate_configuration>
mode: 0644
overwrite: true
path: /etc/nmstate/openshift/worker-0.yml <3>
path: /etc/nmstate/openshift/worker-0.yml
- contents:
source: data:text/plain;charset=utf-8;base64,<base64_encoded_nmstate_configuration>
mode: 0644
overwrite: true
path: /etc/nmstate/openshift/worker-1.yml <3>
path: /etc/nmstate/openshift/worker-1.yml
# ...
----
<1> The name of the policy.
<2> Writes the encoded base64 information to the specified path.
<3> For each node in your cluster, specify the hostname path to your node and the base-64 encoded Ignition configuration file data for the machine type. The `worker` role is the default role for nodes in your cluster. The `.yaml` extension does not work when specifying the short hostname, `hostname -s`, path for each node or all nodes in the `MachineConfig` manifest file.
+
where:
+
`metadata.name`:: The name of the policy.
`contents.source`:: Writes the encoded base64 information to the specified path.
`path`:: For each node in your cluster, specify the hostname path to your node and the base-64 encoded Ignition configuration file data for the machine type. The `worker` role is the default role for nodes in your cluster. You must use the `.yml` extension for configuration files, such as `$(hostname -s).yml` when specifying the short hostname path for each node or all nodes in the `MachineConfig` manifest file.
+
If you have a single global configuration specified in an `/etc/nmstate/openshift/cluster.yml` configuration file that you want to apply to all nodes in your cluster, you do not need to specify the short hostname path for each node, such as `/etc/nmstate/openshift/<node_hostname>.yml`. For example:
+
Expand All @@ -187,35 +198,39 @@ If you have a single global configuration specified in an `/etc/nmstate/openshif
path: /etc/nmstate/openshift/cluster.yml
# ...
----

. Apply the updates from the `MachineConfig` object to your cluster by entering the following command:
+
[source,terminal]
----
$ oc apply -f <machine_config>.yml
----
endif::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[]

ifdef::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[]
* Create a `NodeNetworkConfigurationPolicy` (NNCP) CR and define a customized `br-ex` bridge network configuration. Depending on your needs, ensure that you set a masquerade IP for either the `ipv4.address.ip`, `ipv6.address.ip`, or both parameters. Always include a masquerade IP address in the NNCP CR and this address must match an in-use IP address block.
* Create a `NodeNetworkConfigurationPolicy` (NNCP) CR and define a customized `br-ex` bridge network configuration. The `br-ex` NNCP CR must include the OVN-Kubernetes masquerade IP address and subnet of your network. The example NNCP CR includes default values in the `ipv4.address.ip` and `ipv6.address.ip` parameters. You can set the masquerade IP address in the `ipv4.address.ip`, `ipv6.address.ip`, or both parameters.
+
[IMPORTANT]
====
As a post-installation task, you can configure most parameters for a customized `br-ex` bridge that you defined in an existing NNCP CR, except for the primary IP address of the customized `br-ex` bridge.

If you want to convert your single-stack cluster network to a dual-stack cluster network, you can add or change a secondary IPv6 address in the NNCP CR, but the existing primary IP address cannot be changed.
As a post-installation task, you cannot change the primary IP address of the customized `br-ex` bridge. If you want to convert your single-stack cluster network to a dual-stack cluster network, you can add or change a secondary IPv6 address in the NNCP CR, but the existing primary IP address cannot be changed.
====
+
.Example of an NNCP CR that sets IPv6 and IPv4 masquerade IP addresses
[source,yaml]
----
apiVersion: nmstate.io/v1
kind: NodeNetworkConfigurationPolicy
metadata:
name: worker-0-br-ex <1>
name: worker-0-br-ex
spec:
nodeSelector:
kubernetes.io/hostname: worker-0
desiredState:
interfaces:
- name: enp2s0 <2>
type: ethernet <3>
state: up <4>
- name: enp2s0
type: ethernet
state: up
ipv4:
enabled: false <5>
enabled: false
ipv6:
enabled: false
- name: br-ex
Expand All @@ -231,7 +246,7 @@ spec:
options:
mcast-snooping-enable: true
port:
- name: enp2s0 <6>
- name: enp2s0
- name: br-ex
- name: br-ex
type: ovs-interface
Expand All @@ -240,26 +255,30 @@ spec:
ipv4:
enabled: true
dhcp: true
auto-route-metric: 48 <7>
auto-route-metric: 48
address:
- ip: "169.254.169.2"
prefix-length: 29
- ip: "169.254.0.2"
prefix-length: 17
ipv6:
enabled: true
dhcp: true
auto-route-metric: 48
address:
- ip: "fd69::2"
prefix-length: 125
prefix-length: 112
# ...
----
<1> Name of the policy.
<2> Name of the interface.
<3> The type of ethernet.
<4> The requested state for the interface after creation.
<5> Disables IPv4 and IPv6 in this example.
<6> The node NIC to which the bridge is attached.
<7> Set the parameter to `48` to ensure the `br-ex` default route always has the highest precedence (lowest metric). This configuration prevents routing conflicts with any other interfaces that are automatically configured by the `NetworkManager` service.
+
where:
+
`metadata.name`:: Name of the policy.
`interfaces.name`:: Name of the interface.
`interfaces.type`:: The type of ethernet.
`interfaces.state`:: The requested state for the interface after creation.
`ipv4.enabled`:: Disables IPv4 and IPv6 in this example.
`port.name`:: The node NIC to which the bridge is attached.
`address.ip`:: Shows the default IPv4 and IPv6 IP addresses. Ensure that you set the masquerade IPv4 and IPv6 IP addresses of your network.
`auto-route-metric`:: Set the parameter to `48` to ensure the `br-ex` default route always has the highest precedence (lowest metric). This configuration prevents routing conflicts with any other interfaces that are automatically configured by the `NetworkManager` service.
endif::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[]

.Next steps
Expand Down
4 changes: 2 additions & 2 deletions modules/ipi-install-additional-install-config-parameters.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ a| `provisioningNetworkInterface` | | The name of the network interface on node

| `apiVIPs` | a| (Optional) The virtual IP address for Kubernetes API communication.

This setting must either be provided in the `install-config.yaml` file as a reserved IP from the MachineNetwork or preconfigured in the DNS so that the default name resolves correctly. Use the virtual IP address and not the FQDN when adding a value to the `apiVIPs` configuration setting in the `install-config.yaml` file. The primary IP address must be from the IPv4 network when using dual stack networking. If not set, the installation program uses `api.<cluster_name>.<base_domain>` to derive the IP address from the DNS.
You must either provide this setting in the `install-config.yaml` file as a reserved IP from the `MachineNetwork` parameter or preconfigured in the DNS so that the default name resolves correctly. Use the virtual IP address and not the FQDN when adding a value to the `apiVIPs` configuration setting in the `install-config.yaml` file. For dual-stack networking, the primary IP address can be either an IPv4 network or an IPv6 network. If not set, the installation program uses `api.<cluster_name>.<base_domain>` to derive the IP address from the DNS.

[NOTE]
====
Expand All @@ -117,7 +117,7 @@ Before {product-title} 4.12, the cluster installation program only accepted an I

| `ingressVIPs` | a| (Optional) The virtual IP address for ingress traffic.

This setting must either be provided in the `install-config.yaml` file as a reserved IP from the MachineNetwork or preconfigured in the DNS so that the default name resolves correctly. Use the virtual IP address and not the FQDN when adding a value to the `ingressVIPs` configuration setting in the `install-config.yaml` file. The primary IP address must be from the IPv4 network when using dual stack networking. If not set, the installation program uses `test.apps.<cluster_name>.<base_domain>` to derive the IP address from the DNS.
You must either provide this setting in the `install-config.yaml` file as a reserved IP from the `MachineNetwork` parameter or preconfigured in the DNS so that the default name resolves correctly. Use the virtual IP address and not the FQDN when adding a value to the `ingressVIPs` configuration setting in the `install-config.yaml` file. For dual-stack networking, the primary IP address can be either an IPv4 network or an IPv6 network. If not set, the installation program uses `test.apps.<cluster_name>.<base_domain>` to derive the IP address from the DNS.

[NOTE]
====
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ For more information about how to install the NMState Operator, see xref:../../n

[IMPORTANT]
====
You cannot provide any configuration that modifies the br-ex bridge, an OVN-Kubernetes-managed Open vSwitch bridge. However, you can configure a customized br-ex bridge.
You cannot modify an existing `br-ex` bridge, an OVN-Kubernetes-managed Open vSwitch bridge, or any interfaces, bonds, VLANs, and so on that associate with the `br-ex` bridge. However, you can configure a customized br-ex bridge.

For more information, see "Creating a manifest object that includes a customized br-ex bridge" in the _Deploying installer-provisioned clusters on bare metal_ document or the _Installing a user-provisioned cluster on bare metal_ document.
====
Expand Down