From fce597c0ed714593d709f230a28ac2b7b880e919 Mon Sep 17 00:00:00 2001 From: Bob Date: Tue, 30 Mar 2021 19:48:06 -0400 Subject: [PATCH] Bug 1886229 - Make multipath docs platform-agnostic --- modules/installation-complete-user-infra.adoc | 45 ++---- ...llation-ibm-z-user-infra-machines-iso.adoc | 18 +-- modules/rhcos-enabling-multipath.adoc | 134 ++++++++++++++++++ .../machine-configuration-tasks.adoc | 1 + 4 files changed, 152 insertions(+), 46 deletions(-) create mode 100644 modules/rhcos-enabling-multipath.adoc diff --git a/modules/installation-complete-user-infra.adoc b/modules/installation-complete-user-infra.adoc index 8f96bd08f6fa..9e64c0e6d3e1 100644 --- a/modules/installation-complete-user-infra.adoc +++ b/modules/installation-complete-user-infra.adoc @@ -147,40 +147,11 @@ command. + If the pod logs display, the Kubernetes API server can communicate with the cluster machines. -ifdef::ibm-z,ibm-power[] -. For an installation with FCP, additional steps are required to enable multipathing. -.. To enable multipathing on master nodes, apply the following machine config file: -+ -[source,yaml] --- -apiVersion: machineconfiguration.openshift.io/v1 -kind: MachineConfig -metadata: - labels: - machineconfiguration.openshift.io/role: "master" - name: 99-master-kargs-mpath -spec: - kernelArguments: - - 'rd.multipath=default' - - 'root=/dev/disk/by-label/dm-mpath-root' --- -.. To enable multipathing on worker nodes, apply the following machine config file: -+ -[source,yaml] --- -apiVersion: machineconfiguration.openshift.io/v1 -kind: MachineConfig -metadata: - labels: - machineconfiguration.openshift.io/role: "worker" - name: 99-worker-kargs-mpath -spec: - kernelArguments: - - 'rd.multipath=default' - - 'root=/dev/disk/by-label/dm-mpath-root' --- -endif::ibm-z,ibm-power[] +. For an installation with Fibre Channel Protocol (FCP), additional steps are required to enable multipathing. Do not enable multipathing during installation. ++ +See "Enabling multipathing with kernel arguments on RHCOS" in the _Post-installation configuration_ documentation for more information. + ifdef::ibm-power[] .. To display a boot list and specify the possible boot devices if the system is booted in normal mode, enter the following command: + @@ -199,10 +170,10 @@ sdd sde ---- + -If the original boot disk path is down, the node reboots from the alternate device registered in the normal boot device list. +If the original boot disk path is down, the node reboots from the alternate device registered in the normal boot device list. endif::ibm-power[] ifdef::ibm-z,ibm-power[] -.. All the worker nodes are restarted. To monitor the process, enter the following command: +.. All the worker nodes are restarted. To monitor the process, enter the following command: + [source,terminal] ---- @@ -211,7 +182,7 @@ $ oc get nodes -w + [NOTE] ==== -If you have additional machine types such as infrastructure nodes, repeat the process for these types. +If you have additional machine types such as infrastructure nodes, repeat the process for these types. ==== endif::ibm-z,ibm-power[] @@ -244,4 +215,4 @@ endif::[] ifeval::["{context}" == "installing-restricted-networks-ibm-power"] :!ibm-power: :restricted: -endif::[] \ No newline at end of file +endif::[] diff --git a/modules/installation-ibm-z-user-infra-machines-iso.adoc b/modules/installation-ibm-z-user-infra-machines-iso.adoc index 8cdcb61189ed..7d89e6b0696a 100644 --- a/modules/installation-ibm-z-user-infra-machines-iso.adoc +++ b/modules/installation-ibm-z-user-infra-machines-iso.adoc @@ -48,9 +48,9 @@ The rootfs image is the same for FCP and DASD. ** For `coreos.live.rootfs_url=`, specify the matching rootfs artifact for the kernel and initramfs you are booting. Only HTTP and HTTPS protocols are supported. ** For installations on DASD-type disks, complete the following tasks: -... For `coreos.inst.install_dev=`, specify `dasda`. +... For `coreos.inst.install_dev=`, specify `dasda`. ... Use `rd.dasd=` to specify the DASD where {op-system} is to be installed. -... Leave all other parameters unchanged. +... Leave all other parameters unchanged. + Example parameter file, `bootstrap-0.parm`, for the bootstrap machine: + @@ -66,36 +66,36 @@ rd.znet=qeth,0.0.bdf0,0.0.bdf1,0.0.bdf2,layer2=1,portno=0 zfcp.allow_lun_scan=0 + [NOTE] ==== -`dfltcc=off` is required for IBM z15 and LinuxONE III. +`dfltcc=off` is required for IBM z15 and LinuxONE III. ==== ** For installations on FCP-type disks, complete the following tasks: -... Use `rd.zfcp=,,` to specify the FCP disk where {op-system} is to be installed. For multipathing repeat this step for each additional path. +... Use `rd.zfcp=,,` to specify the FCP disk where {op-system} is to be installed. For multipathing repeat this step for each additional path. ... For multipathing, set the following parameter: `rd.multipath=default`. ... For multipathing, set the install device as: `coreos.inst.install_dev=/dev/mapper/mpatha`. ... For single-path installation, set the install device as: `coreos.inst.install_dev=sda`. + [NOTE] ==== -If additional LUNs are configured with NPIV, FCP requires `zfcp.allow_lun_scan=0`. +If additional LUNs are configured with NPIV, FCP requires `zfcp.allow_lun_scan=0`. ==== ... Leave all other parameters unchanged. + [NOTE] ==== -Additional post-installation steps are required to fully enable multipathing. See “Completing installation on user-provisioned infrastructure” for more information. +Additional post-installation steps are required to fully enable multipathing. See “Completing installation on user-provisioned infrastructure” for more information. ==== + The following is an example parameter file `bootstrap-0.parm` for the bootstrap machine with multipathing: + [source,terminal] ---- -rd.neednet=1 dfltcc=off rd.multipath=default console=ttysclp0 coreos.inst.install_dev=/dev/mapper/mpatha +rd.neednet=1 dfltcc=off rd.multipath=default console=ttysclp0 coreos.inst.install_dev=/dev/mapper/mpatha coreos.live.rootfs_url=http://cl1.provide.example.com:8080/assets/rhcos-live-rootfs.s390x.img coreos.inst.ignition_url=http://cl1.provide.example.com:8080/ignition/bootstrap.ign ip=172.18.78.2::172.18.78.1:255.255.255.0:::none nameserver=172.18.78.1 rd.znet=qeth,0.0.bdf0,0.0.bdf1,0.0.bdf2,layer2=1,portno=0 zfcp.allow_lun_scan=0 cio_ignore=all, -!condev +!condev rd.zfcp=0.0.1987,0x50050763070bc5e3,0x4008400B00000000 rd.zfcp=0.0.19C7,0x50050763070bc5e3,0x4008400B00000000 rd.zfcp=0.0.1987,0x50050763071bc5e3,0x4008400B00000000 @@ -104,7 +104,7 @@ rd.zfcp=0.0.19C7,0x50050763071bc5e3,0x4008400B00000000 + [NOTE] ==== -`dfltcc=off` is required for IBM z15 and LinuxONE III. +`dfltcc=off` is required for IBM z15 and LinuxONE III. ==== . Transfer the initramfs, kernel, parameter files, and {op-system} images to z/VM, for example with FTP. For details about how to transfer the files with FTP and boot from the virtual reader, see link:https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/installation_guide/sect-installing-zvm-s390[Installing under Z/VM]. diff --git a/modules/rhcos-enabling-multipath.adoc b/modules/rhcos-enabling-multipath.adoc new file mode 100644 index 000000000000..5da860e28711 --- /dev/null +++ b/modules/rhcos-enabling-multipath.adoc @@ -0,0 +1,134 @@ +// Module included in the following assemblies: +// +// * post_installation_configuration/machine-configuration-tasks.adoc + +[id="rhcos-enabling-multipath_{context}"] += Enabling multipathing with kernel arguments on {op-system} + +{op-system} supports multipathing on the primary disk, allowing stronger resilience to hardware failure to achieve higher host availability. + +[IMPORTANT] +==== +Multipathing is only supported when it is activated using the machine config as documented in the following procedure. It must be enabled after {op-system} installation. +==== + +.Prerequisites +* You have a running {product-title} cluster that uses version 4.7 or later. +* You are logged in to the cluster as a user with administrative privileges. + +.Procedure + +. To enable multipathing on master nodes: + +* Create a machine config file, such as `99-master-kargs-mpath.yaml`, that instructs the cluster to add the `master` label and that identifies the multipath kernel argument, for example: + ++ +[source,yaml] +---- +apiVersion: machineconfiguration.openshift.io/v1 +kind: MachineConfig +metadata: + labels: + machineconfiguration.openshift.io/role: "master" + name: 99-master-kargs-mpath +spec: + kernelArguments: + - 'rd.multipath=default' + - 'root=/dev/disk/by-label/dm-mpath-root' +---- + +. To enable multipathing on worker nodes: + +* Create a machine config file, such as `99-worker-kargs-mpath.yaml`, that instructs the cluster to add the `worker` label and that identifies the multipath kernel argument, for example: ++ +[source,yaml] +---- +apiVersion: machineconfiguration.openshift.io/v1 +kind: MachineConfig +metadata: + labels: + machineconfiguration.openshift.io/role: "worker" + name: 99-worker-kargs-mpath +spec: + kernelArguments: + - 'rd.multipath=default' + - 'root=/dev/disk/by-label/dm-mpath-root' +---- + +. Create the new machine config by using either the master or worker YAML file you previously created: ++ +[source,terminal] +---- +$ oc create -f ./99-master-kargs-mpath.yaml +---- + +. Check the machine configs to see that the new one was added: ++ +[source,terminal] +---- +$ oc get MachineConfig +---- ++ +.Example output +[source,terminal] +---- +NAME GENERATEDBYCONTROLLER IGNITIONVERSION AGE +00-master 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m +00-worker 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m +01-master-container-runtime 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m +01-master-kubelet 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m +01-worker-container-runtime 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m +01-worker-kubelet 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m +99-master-kargs-mpath 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 105s +99-master-generated-registries 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m +99-master-ssh 3.2.0 40m +99-worker-generated-registries 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m +99-worker-ssh 3.2.0 40m +rendered-master-23e785de7587df95a4b517e0647e5ab7 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m +rendered-worker-5d596d9293ca3ea80c896a1191735bb1 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m +---- + +. Check the nodes: ++ +[source,terminal] +---- +$ oc get nodes +---- ++ +.Example output +[source,terminal] +---- +NAME STATUS ROLES AGE VERSION +ip-10-0-136-161.ec2.internal Ready worker 28m v1.20.0 +ip-10-0-136-243.ec2.internal Ready master 34m v1.20.0 +ip-10-0-141-105.ec2.internal Ready,SchedulingDisabled worker 28m v1.20.0 +ip-10-0-142-249.ec2.internal Ready master 34m v1.20.0 +ip-10-0-153-11.ec2.internal Ready worker 28m v1.20.0 +ip-10-0-153-150.ec2.internal Ready master 34m v1.20.0 +---- ++ +You can see that scheduling on each worker node is disabled as the change is being applied. + +. Check that the kernel argument worked by going to one of the worker nodes and listing +the kernel command line arguments (in `/proc/cmdline` on the host): ++ +[source,terminal] +---- +$ oc debug node/ip-10-0-141-105.ec2.internal +---- ++ +.Example output +[source,terminal] +---- +Starting pod/ip-10-0-141-105ec2internal-debug ... +To use host binaries, run `chroot /host` + +sh-4.2# cat /host/proc/cmdline +... +rd.multipath=default root=/dev/disk/by-label/dm-mpath-root +... + +sh-4.2# exit +---- ++ +You should see the added kernel arguments. diff --git a/post_installation_configuration/machine-configuration-tasks.adoc b/post_installation_configuration/machine-configuration-tasks.adoc index 4bdb10752953..99b57d8aa6ff 100644 --- a/post_installation_configuration/machine-configuration-tasks.adoc +++ b/post_installation_configuration/machine-configuration-tasks.adoc @@ -28,6 +28,7 @@ link:https://access.redhat.com/solutions/4510281[updating] SSH authorized keys, include::modules/installation-special-config-crony.adoc[leveloffset=+2] include::modules/nodes-nodes-kernel-arguments.adoc[leveloffset=+2] +include::modules/rhcos-enabling-multipath.adoc[leveloffset=+2] include::modules/nodes-nodes-rtkernel-arguments.adoc[leveloffset=+2] include::modules/machineconfig-modify-journald.adoc[leveloffset=+2] include::modules/rhcos-add-extensions.adoc[leveloffset=+2]