Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't boot from SAN #2556

Open
danielwieberneit opened this Issue Feb 22, 2019 · 6 comments

Comments

Projects
None yet
3 participants
@danielwieberneit
Copy link

danielwieberneit commented Feb 22, 2019

Issue Report

MPATH LUN ist not recognized as full Disk by coreos-install Skript.
Error: Target block device /dev/dm-0 is not a full disk."

Environment

HP ProLiant DL 380 Gen9
2 x HP Store Fabric 8 Gb Single Port PCI-e FC HBA
IBM SAN Volume Controller (SVC)

We load dm-multipath Modul:

  • path: /etc/modules-load.d/multipath.conf
    filesystem: root
    mode: 0644
    contents:
    inline: |
    dm-multipath

Desired Feature

A FC LUN / Multipath Device (/dev/dm-0) should be choosable as install-disk
mpath type should be recognized as full disk.

Other Information

Line 446 in coreos-install:
if ! [[ $(lsblk -n -d -o TYPE "${DEVICE}") =~ ^(disk|loop|lvm)$ ]]; then
echo "$0: Target block device (${DEVICE}) is not a full disk." >&2
exit 1
fi
lsblk -n -d -o TYPE "${DEVICE}" returns mpath

@bgilbert

This comment has been minimized.

Copy link
Member

bgilbert commented Feb 22, 2019

I'm not sure whether Container Linux would successfully boot, even if the install completed normally. Could you try modifying the check to accept mpath and see whether you get a bootable system?

@danielwieberneit

This comment has been minimized.

Copy link
Author

danielwieberneit commented Feb 22, 2019

we copied coreos-install to a tmp Folder, added mpath. System booted after Execution.

@lucab

This comment has been minimized.

Copy link
Member

lucab commented Feb 25, 2019

@danielwieberneit can you please also double-check that it boots fine on both first-boot and on subsequent reboots?

Additionally, is the /etc/modules-load.d/multipath.conf above configured for the installer or for the final OS?

@danielwieberneit

This comment has been minimized.

Copy link
Author

danielwieberneit commented Feb 25, 2019

I tried to reproduce. I run in BLKRRPART: Device or resource busy. The partitions are already in use by the kernel and I get problems restarting the changed coreos-install Skript

@danielwieberneit

This comment has been minimized.

Copy link
Author

danielwieberneit commented Feb 26, 2019

There seam to be a lot of problems to make CoreOS boot from a multi-path device. I found 2 further problems in coreos-install Skript. It is probably not all.

line 508
blockdev --rereadpt "${DEVICE}" && unset try && break ||

blockdev --rereadpt "/dev/dm-0"
blockdev: ioctl error on BLKRRPART: Invalid argument
-> dm devices seam to be incompatible to ioctl

Line 636:
local OEM_DEV=$(blkid -t "LABEL=OEM" -o device "${DEVICE}"*)
blkid returns not /dev/dm-0, but /dev/mapper/3600507680c80840e1800000000000032

After working around I can boot. But our configuration from ignition.json still will not work effective.
In consequence I just boot from /dev/sda without multipathing. Thy is why multiple restarts seemed to work. I could not login...

Now I see blkid is used in line 611 and 614 too...

@danielwieberneit

This comment has been minimized.

Copy link
Author

danielwieberneit commented Feb 27, 2019

After some workarounds I was able to install to /dev/dm0.
I have the following messages in journal when booting the installed system:
kernel: device-mapper: multipath service-time: version 0.3.0 loaded
kernel: device-mapper: table: 254:1: multipath: error getting device
kernel: device-mapper: ioctl: error adding target to table
multipathd[4207]: 3600507680c80840e1800000000000032: ignoring map

HBA and disks are discovered 10s before.

Looks like it makes no difference installing to /dev/dm-0
I had the same results when installing to /dev/sda
Disks are already in use. Even though Multipathing is loaded the partitions are spread over the path:
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 20G 0 disk
|-sda1 8:1 0 128M 0 part /boot
|-sda2 8:2 0 2M 0 part
|-sda3 8:3 0 1G 0 part
|-sda4 8:4 0 1G 0 part
|-sda6 8:6 0 128M 0 part /usr/share/oem
|-sda7 8:7 0 64M 0 part
-sda9 8:9 0 17.7G 0 part / sdb 8:16 0 20G 0 disk |-sdb1 8:17 0 128M 0 part |-sdb2 8:18 0 2M 0 part |-sdb3 8:19 0 1G 0 part | -usr 254:0 0 1016M 1 crypt /usr
|-sdb4 8:20 0 1G 0 part
|-sdb6 8:22 0 128M 0 part
|-sdb7 8:23 0 64M 0 part
-sdb9 8:25 0 17.7G 0 part sdc 8:32 0 20G 0 disk |-sdc1 8:33 0 128M 0 part |-sdc2 8:34 0 2M 0 part |-sdc3 8:35 0 1G 0 part |-sdc4 8:36 0 1G 0 part |-sdc6 8:38 0 128M 0 part |-sdc7 8:39 0 64M 0 part -sdc9 8:41 0 17.7G 0 part
sdd 8:48 0 20G 0 disk
|-sdd1 8:49 0 128M 0 part
|-sdd2 8:50 0 2M 0 part
|-sdd3 8:51 0 1G 0 part
|-sdd4 8:52 0 1G 0 part
|-sdd6 8:54 0 128M 0 part
|-sdd7 8:55 0 64M 0 part
`-sdd9 8:57 0 17.7G 0 part

Is there something like a kernel boot option for multipathing?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.