Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bug 1886229 - Make multipath docs platform-agnostic #31140

Merged
merged 1 commit into from
Apr 23, 2021
Merged

Bug 1886229 - Make multipath docs platform-agnostic #31140

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 8 additions & 37 deletions modules/installation-complete-user-infra.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -147,40 +147,11 @@ command.
+
If the pod logs display, the Kubernetes API server can communicate with the
cluster machines.
ifdef::ibm-z,ibm-power[]
. For an installation with FCP, additional steps are required to enable multipathing.
.. To enable multipathing on master nodes, apply the following machine config file:
+
[source,yaml]
--
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
labels:
machineconfiguration.openshift.io/role: "master"
name: 99-master-kargs-mpath
spec:
kernelArguments:
- 'rd.multipath=default'
- 'root=/dev/disk/by-label/dm-mpath-root'
--

.. To enable multipathing on worker nodes, apply the following machine config file:
+
[source,yaml]
--
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
labels:
machineconfiguration.openshift.io/role: "worker"
name: 99-worker-kargs-mpath
spec:
kernelArguments:
- 'rd.multipath=default'
- 'root=/dev/disk/by-label/dm-mpath-root'
--
endif::ibm-z,ibm-power[]
. For an installation with Fibre Channel Protocol (FCP), additional steps are required to enable multipathing. Do not enable multipathing during installation.
+
See "Enabling multipathing with kernel arguments on RHCOS" in the _Post-installation configuration_ documentation for more information.

ifdef::ibm-power[]
.. To display a boot list and specify the possible boot devices if the system is booted in normal mode, enter the following command:
+
Expand All @@ -199,10 +170,10 @@ sdd
sde
----
+
If the original boot disk path is down, the node reboots from the alternate device registered in the normal boot device list.
If the original boot disk path is down, the node reboots from the alternate device registered in the normal boot device list.
endif::ibm-power[]
ifdef::ibm-z,ibm-power[]
.. All the worker nodes are restarted. To monitor the process, enter the following command:
.. All the worker nodes are restarted. To monitor the process, enter the following command:
+
[source,terminal]
----
Expand All @@ -211,7 +182,7 @@ $ oc get nodes -w
+
[NOTE]
====
If you have additional machine types such as infrastructure nodes, repeat the process for these types.
If you have additional machine types such as infrastructure nodes, repeat the process for these types.
====
endif::ibm-z,ibm-power[]

Expand Down Expand Up @@ -244,4 +215,4 @@ endif::[]
ifeval::["{context}" == "installing-restricted-networks-ibm-power"]
:!ibm-power:
:restricted:
endif::[]
endif::[]
18 changes: 9 additions & 9 deletions modules/installation-ibm-z-user-infra-machines-iso.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,9 +48,9 @@ The rootfs image is the same for FCP and DASD.
** For `coreos.live.rootfs_url=`, specify the matching rootfs artifact for the kernel and initramfs you are booting. Only HTTP and HTTPS protocols are supported.

** For installations on DASD-type disks, complete the following tasks:
... For `coreos.inst.install_dev=`, specify `dasda`.
... For `coreos.inst.install_dev=`, specify `dasda`.
... Use `rd.dasd=` to specify the DASD where {op-system} is to be installed.
... Leave all other parameters unchanged.
... Leave all other parameters unchanged.
+
Example parameter file, `bootstrap-0.parm`, for the bootstrap machine:
+
Expand All @@ -66,36 +66,36 @@ rd.znet=qeth,0.0.bdf0,0.0.bdf1,0.0.bdf2,layer2=1,portno=0 zfcp.allow_lun_scan=0
+
[NOTE]
====
`dfltcc=off` is required for IBM z15 and LinuxONE III.
`dfltcc=off` is required for IBM z15 and LinuxONE III.
====

** For installations on FCP-type disks, complete the following tasks:
... Use `rd.zfcp=<adapter>,<wwpn>,<lun>` to specify the FCP disk where {op-system} is to be installed. For multipathing repeat this step for each additional path.
... Use `rd.zfcp=<adapter>,<wwpn>,<lun>` to specify the FCP disk where {op-system} is to be installed. For multipathing repeat this step for each additional path.
... For multipathing, set the following parameter: `rd.multipath=default`.
... For multipathing, set the install device as: `coreos.inst.install_dev=/dev/mapper/mpatha`.
... For single-path installation, set the install device as: `coreos.inst.install_dev=sda`.
+
[NOTE]
====
If additional LUNs are configured with NPIV, FCP requires `zfcp.allow_lun_scan=0`.
If additional LUNs are configured with NPIV, FCP requires `zfcp.allow_lun_scan=0`.
====
... Leave all other parameters unchanged.
+
[NOTE]
====
Additional post-installation steps are required to fully enable multipathing. See “Completing installation on user-provisioned infrastructure” for more information.
Additional post-installation steps are required to fully enable multipathing. See “Completing installation on user-provisioned infrastructure” for more information.
====
+
The following is an example parameter file `bootstrap-0.parm` for the bootstrap machine with multipathing:
+
[source,terminal]
----
rd.neednet=1 dfltcc=off rd.multipath=default console=ttysclp0 coreos.inst.install_dev=/dev/mapper/mpatha
rd.neednet=1 dfltcc=off rd.multipath=default console=ttysclp0 coreos.inst.install_dev=/dev/mapper/mpatha
coreos.live.rootfs_url=http://cl1.provide.example.com:8080/assets/rhcos-live-rootfs.s390x.img
coreos.inst.ignition_url=http://cl1.provide.example.com:8080/ignition/bootstrap.ign
ip=172.18.78.2::172.18.78.1:255.255.255.0:::none nameserver=172.18.78.1
rd.znet=qeth,0.0.bdf0,0.0.bdf1,0.0.bdf2,layer2=1,portno=0 zfcp.allow_lun_scan=0 cio_ignore=all,
!condev
!condev
rd.zfcp=0.0.1987,0x50050763070bc5e3,0x4008400B00000000
rd.zfcp=0.0.19C7,0x50050763070bc5e3,0x4008400B00000000
rd.zfcp=0.0.1987,0x50050763071bc5e3,0x4008400B00000000
Expand All @@ -104,7 +104,7 @@ rd.zfcp=0.0.19C7,0x50050763071bc5e3,0x4008400B00000000
+
[NOTE]
====
`dfltcc=off` is required for IBM z15 and LinuxONE III.
`dfltcc=off` is required for IBM z15 and LinuxONE III.
====

. Transfer the initramfs, kernel, parameter files, and {op-system} images to z/VM, for example with FTP. For details about how to transfer the files with FTP and boot from the virtual reader, see link:https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/installation_guide/sect-installing-zvm-s390[Installing under Z/VM].
Expand Down
134 changes: 134 additions & 0 deletions modules/rhcos-enabling-multipath.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
// Module included in the following assemblies:
//
// * post_installation_configuration/machine-configuration-tasks.adoc

[id="rhcos-enabling-multipath_{context}"]
= Enabling multipathing with kernel arguments on {op-system}

{op-system} supports multipathing on the primary disk, allowing stronger resilience to hardware failure to achieve higher host availability.

[IMPORTANT]
====
Multipathing is only supported when it is activated using the machine config as documented in the following procedure. It must be enabled after {op-system} installation.
====

.Prerequisites
* You have a running {product-title} cluster that uses version 4.7 or later.
* You are logged in to the cluster as a user with administrative privileges.

.Procedure

. To enable multipathing on master nodes:

* Create a machine config file, such as `99-master-kargs-mpath.yaml`, that instructs the cluster to add the `master` label and that identifies the multipath kernel argument, for example:

+
[source,yaml]
----
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
labels:
machineconfiguration.openshift.io/role: "master"
name: 99-master-kargs-mpath
spec:
kernelArguments:
- 'rd.multipath=default'
- 'root=/dev/disk/by-label/dm-mpath-root'
----

. To enable multipathing on worker nodes:

* Create a machine config file, such as `99-worker-kargs-mpath.yaml`, that instructs the cluster to add the `worker` label and that identifies the multipath kernel argument, for example:
+
[source,yaml]
----
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
labels:
machineconfiguration.openshift.io/role: "worker"
name: 99-worker-kargs-mpath
spec:
kernelArguments:
- 'rd.multipath=default'
- 'root=/dev/disk/by-label/dm-mpath-root'
----

. Create the new machine config by using either the master or worker YAML file you previously created:
+
[source,terminal]
----
$ oc create -f ./99-master-kargs-mpath.yaml
----

. Check the machine configs to see that the new one was added:
+
[source,terminal]
----
$ oc get MachineConfig
----
+
.Example output
[source,terminal]
----
NAME GENERATEDBYCONTROLLER IGNITIONVERSION AGE
00-master 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
00-worker 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
01-master-container-runtime 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
01-master-kubelet 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
01-worker-container-runtime 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
01-worker-kubelet 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
99-master-kargs-mpath 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 105s
99-master-generated-registries 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
99-master-ssh 3.2.0 40m
99-worker-generated-registries 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
99-worker-ssh 3.2.0 40m
rendered-master-23e785de7587df95a4b517e0647e5ab7 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
rendered-worker-5d596d9293ca3ea80c896a1191735bb1 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
----

. Check the nodes:
+
[source,terminal]
----
$ oc get nodes
----
+
.Example output
[source,terminal]
----
NAME STATUS ROLES AGE VERSION
ip-10-0-136-161.ec2.internal Ready worker 28m v1.20.0
ip-10-0-136-243.ec2.internal Ready master 34m v1.20.0
ip-10-0-141-105.ec2.internal Ready,SchedulingDisabled worker 28m v1.20.0
ip-10-0-142-249.ec2.internal Ready master 34m v1.20.0
ip-10-0-153-11.ec2.internal Ready worker 28m v1.20.0
ip-10-0-153-150.ec2.internal Ready master 34m v1.20.0
----
+
You can see that scheduling on each worker node is disabled as the change is being applied.

. Check that the kernel argument worked by going to one of the worker nodes and listing
the kernel command line arguments (in `/proc/cmdline` on the host):
+
[source,terminal]
----
$ oc debug node/ip-10-0-141-105.ec2.internal
----
+
.Example output
[source,terminal]
----
Starting pod/ip-10-0-141-105ec2internal-debug ...
To use host binaries, run `chroot /host`

sh-4.2# cat /host/proc/cmdline
...
rd.multipath=default root=/dev/disk/by-label/dm-mpath-root
bobfuru marked this conversation as resolved.
Show resolved Hide resolved
...

sh-4.2# exit
----
+
You should see the added kernel arguments.
1 change: 1 addition & 0 deletions post_installation_configuration/machine-configuration-tasks.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ link:https://access.redhat.com/solutions/4510281[updating] SSH authorized keys,

include::modules/installation-special-config-crony.adoc[leveloffset=+2]
include::modules/nodes-nodes-kernel-arguments.adoc[leveloffset=+2]
include::modules/rhcos-enabling-multipath.adoc[leveloffset=+2]
include::modules/nodes-nodes-rtkernel-arguments.adoc[leveloffset=+2]
include::modules/machineconfig-modify-journald.adoc[leveloffset=+2]
include::modules/rhcos-add-extensions.adoc[leveloffset=+2]
Expand Down