Skip to content

Commit

Permalink
Bug 1886229 - Make multipath docs platform-agnostic
Browse files Browse the repository at this point in the history
  • Loading branch information
Bob committed Apr 23, 2021
1 parent 1ca1f0e commit fce597c
Show file tree
Hide file tree
Showing 4 changed files with 152 additions and 46 deletions.
45 changes: 8 additions & 37 deletions modules/installation-complete-user-infra.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -147,40 +147,11 @@ command.
+
If the pod logs display, the Kubernetes API server can communicate with the
cluster machines.
ifdef::ibm-z,ibm-power[]
. For an installation with FCP, additional steps are required to enable multipathing.
.. To enable multipathing on master nodes, apply the following machine config file:
+
[source,yaml]
--
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
labels:
machineconfiguration.openshift.io/role: "master"
name: 99-master-kargs-mpath
spec:
kernelArguments:
- 'rd.multipath=default'
- 'root=/dev/disk/by-label/dm-mpath-root'
--

.. To enable multipathing on worker nodes, apply the following machine config file:
+
[source,yaml]
--
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
labels:
machineconfiguration.openshift.io/role: "worker"
name: 99-worker-kargs-mpath
spec:
kernelArguments:
- 'rd.multipath=default'
- 'root=/dev/disk/by-label/dm-mpath-root'
--
endif::ibm-z,ibm-power[]
. For an installation with Fibre Channel Protocol (FCP), additional steps are required to enable multipathing. Do not enable multipathing during installation.
+
See "Enabling multipathing with kernel arguments on RHCOS" in the _Post-installation configuration_ documentation for more information.

ifdef::ibm-power[]
.. To display a boot list and specify the possible boot devices if the system is booted in normal mode, enter the following command:
+
Expand All @@ -199,10 +170,10 @@ sdd
sde
----
+
If the original boot disk path is down, the node reboots from the alternate device registered in the normal boot device list.
If the original boot disk path is down, the node reboots from the alternate device registered in the normal boot device list.
endif::ibm-power[]
ifdef::ibm-z,ibm-power[]
.. All the worker nodes are restarted. To monitor the process, enter the following command:
.. All the worker nodes are restarted. To monitor the process, enter the following command:
+
[source,terminal]
----
Expand All @@ -211,7 +182,7 @@ $ oc get nodes -w
+
[NOTE]
====
If you have additional machine types such as infrastructure nodes, repeat the process for these types.
If you have additional machine types such as infrastructure nodes, repeat the process for these types.
====
endif::ibm-z,ibm-power[]

Expand Down Expand Up @@ -244,4 +215,4 @@ endif::[]
ifeval::["{context}" == "installing-restricted-networks-ibm-power"]
:!ibm-power:
:restricted:
endif::[]
endif::[]
18 changes: 9 additions & 9 deletions modules/installation-ibm-z-user-infra-machines-iso.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -48,9 +48,9 @@ The rootfs image is the same for FCP and DASD.
** For `coreos.live.rootfs_url=`, specify the matching rootfs artifact for the kernel and initramfs you are booting. Only HTTP and HTTPS protocols are supported.

** For installations on DASD-type disks, complete the following tasks:
... For `coreos.inst.install_dev=`, specify `dasda`.
... For `coreos.inst.install_dev=`, specify `dasda`.
... Use `rd.dasd=` to specify the DASD where {op-system} is to be installed.
... Leave all other parameters unchanged.
... Leave all other parameters unchanged.
+
Example parameter file, `bootstrap-0.parm`, for the bootstrap machine:
+
Expand All @@ -66,36 +66,36 @@ rd.znet=qeth,0.0.bdf0,0.0.bdf1,0.0.bdf2,layer2=1,portno=0 zfcp.allow_lun_scan=0
+
[NOTE]
====
`dfltcc=off` is required for IBM z15 and LinuxONE III.
`dfltcc=off` is required for IBM z15 and LinuxONE III.
====

** For installations on FCP-type disks, complete the following tasks:
... Use `rd.zfcp=<adapter>,<wwpn>,<lun>` to specify the FCP disk where {op-system} is to be installed. For multipathing repeat this step for each additional path.
... Use `rd.zfcp=<adapter>,<wwpn>,<lun>` to specify the FCP disk where {op-system} is to be installed. For multipathing repeat this step for each additional path.
... For multipathing, set the following parameter: `rd.multipath=default`.
... For multipathing, set the install device as: `coreos.inst.install_dev=/dev/mapper/mpatha`.
... For single-path installation, set the install device as: `coreos.inst.install_dev=sda`.
+
[NOTE]
====
If additional LUNs are configured with NPIV, FCP requires `zfcp.allow_lun_scan=0`.
If additional LUNs are configured with NPIV, FCP requires `zfcp.allow_lun_scan=0`.
====
... Leave all other parameters unchanged.
+
[NOTE]
====
Additional post-installation steps are required to fully enable multipathing. See “Completing installation on user-provisioned infrastructure” for more information.
Additional post-installation steps are required to fully enable multipathing. See “Completing installation on user-provisioned infrastructure” for more information.
====
+
The following is an example parameter file `bootstrap-0.parm` for the bootstrap machine with multipathing:
+
[source,terminal]
----
rd.neednet=1 dfltcc=off rd.multipath=default console=ttysclp0 coreos.inst.install_dev=/dev/mapper/mpatha
rd.neednet=1 dfltcc=off rd.multipath=default console=ttysclp0 coreos.inst.install_dev=/dev/mapper/mpatha
coreos.live.rootfs_url=http://cl1.provide.example.com:8080/assets/rhcos-live-rootfs.s390x.img
coreos.inst.ignition_url=http://cl1.provide.example.com:8080/ignition/bootstrap.ign
ip=172.18.78.2::172.18.78.1:255.255.255.0:::none nameserver=172.18.78.1
rd.znet=qeth,0.0.bdf0,0.0.bdf1,0.0.bdf2,layer2=1,portno=0 zfcp.allow_lun_scan=0 cio_ignore=all,
!condev
!condev
rd.zfcp=0.0.1987,0x50050763070bc5e3,0x4008400B00000000
rd.zfcp=0.0.19C7,0x50050763070bc5e3,0x4008400B00000000
rd.zfcp=0.0.1987,0x50050763071bc5e3,0x4008400B00000000
Expand All @@ -104,7 +104,7 @@ rd.zfcp=0.0.19C7,0x50050763071bc5e3,0x4008400B00000000
+
[NOTE]
====
`dfltcc=off` is required for IBM z15 and LinuxONE III.
`dfltcc=off` is required for IBM z15 and LinuxONE III.
====

. Transfer the initramfs, kernel, parameter files, and {op-system} images to z/VM, for example with FTP. For details about how to transfer the files with FTP and boot from the virtual reader, see link:https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/installation_guide/sect-installing-zvm-s390[Installing under Z/VM].
Expand Down
134 changes: 134 additions & 0 deletions modules/rhcos-enabling-multipath.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
// Module included in the following assemblies:
//
// * post_installation_configuration/machine-configuration-tasks.adoc

[id="rhcos-enabling-multipath_{context}"]
= Enabling multipathing with kernel arguments on {op-system}

{op-system} supports multipathing on the primary disk, allowing stronger resilience to hardware failure to achieve higher host availability.

[IMPORTANT]
====
Multipathing is only supported when it is activated using the machine config as documented in the following procedure. It must be enabled after {op-system} installation.
====

.Prerequisites
* You have a running {product-title} cluster that uses version 4.7 or later.
* You are logged in to the cluster as a user with administrative privileges.

.Procedure

. To enable multipathing on master nodes:

* Create a machine config file, such as `99-master-kargs-mpath.yaml`, that instructs the cluster to add the `master` label and that identifies the multipath kernel argument, for example:

+
[source,yaml]
----
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
labels:
machineconfiguration.openshift.io/role: "master"
name: 99-master-kargs-mpath
spec:
kernelArguments:
- 'rd.multipath=default'
- 'root=/dev/disk/by-label/dm-mpath-root'
----

. To enable multipathing on worker nodes:

* Create a machine config file, such as `99-worker-kargs-mpath.yaml`, that instructs the cluster to add the `worker` label and that identifies the multipath kernel argument, for example:
+
[source,yaml]
----
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
labels:
machineconfiguration.openshift.io/role: "worker"
name: 99-worker-kargs-mpath
spec:
kernelArguments:
- 'rd.multipath=default'
- 'root=/dev/disk/by-label/dm-mpath-root'
----

. Create the new machine config by using either the master or worker YAML file you previously created:
+
[source,terminal]
----
$ oc create -f ./99-master-kargs-mpath.yaml
----

. Check the machine configs to see that the new one was added:
+
[source,terminal]
----
$ oc get MachineConfig
----
+
.Example output
[source,terminal]
----
NAME GENERATEDBYCONTROLLER IGNITIONVERSION AGE
00-master 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
00-worker 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
01-master-container-runtime 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
01-master-kubelet 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
01-worker-container-runtime 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
01-worker-kubelet 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
99-master-kargs-mpath 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 105s
99-master-generated-registries 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
99-master-ssh 3.2.0 40m
99-worker-generated-registries 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
99-worker-ssh 3.2.0 40m
rendered-master-23e785de7587df95a4b517e0647e5ab7 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
rendered-worker-5d596d9293ca3ea80c896a1191735bb1 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
----

. Check the nodes:
+
[source,terminal]
----
$ oc get nodes
----
+
.Example output
[source,terminal]
----
NAME STATUS ROLES AGE VERSION
ip-10-0-136-161.ec2.internal Ready worker 28m v1.20.0
ip-10-0-136-243.ec2.internal Ready master 34m v1.20.0
ip-10-0-141-105.ec2.internal Ready,SchedulingDisabled worker 28m v1.20.0
ip-10-0-142-249.ec2.internal Ready master 34m v1.20.0
ip-10-0-153-11.ec2.internal Ready worker 28m v1.20.0
ip-10-0-153-150.ec2.internal Ready master 34m v1.20.0
----
+
You can see that scheduling on each worker node is disabled as the change is being applied.

. Check that the kernel argument worked by going to one of the worker nodes and listing
the kernel command line arguments (in `/proc/cmdline` on the host):
+
[source,terminal]
----
$ oc debug node/ip-10-0-141-105.ec2.internal
----
+
.Example output
[source,terminal]
----
Starting pod/ip-10-0-141-105ec2internal-debug ...
To use host binaries, run `chroot /host`
sh-4.2# cat /host/proc/cmdline
...
rd.multipath=default root=/dev/disk/by-label/dm-mpath-root
...
sh-4.2# exit
----
+
You should see the added kernel arguments.
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ link:https://access.redhat.com/solutions/4510281[updating] SSH authorized keys,

include::modules/installation-special-config-crony.adoc[leveloffset=+2]
include::modules/nodes-nodes-kernel-arguments.adoc[leveloffset=+2]
include::modules/rhcos-enabling-multipath.adoc[leveloffset=+2]
include::modules/nodes-nodes-rtkernel-arguments.adoc[leveloffset=+2]
include::modules/machineconfig-modify-journald.adoc[leveloffset=+2]
include::modules/rhcos-add-extensions.adoc[leveloffset=+2]
Expand Down

0 comments on commit fce597c

Please sign in to comment.