CentOS 7 fails to boot after install

[RobertDeRose]

Background:

Using FreeNAS 9.10-Stable which is running FreeBSD 10.3-Stable and comes preinstalled with iohyve v0.7.7 2016/11/10 I Think I'll Go for a Walk Edition I've done the following things based on the README.

iohyve set centosguest loader=grub-bhyve
iohyve set centosguest os=centos7
iohyve install centosguest CentOS-7-x86_64-Everything-1511.iso
iohyve console centosguest

I set the following properties:

bargs          -A_-H_-P
boot           0
con            nmdm0
cpu            2
description    QuickBooks DB Server
install        no
loader         grub-bhyve
name           qb-db-server
os             centos7
persist        1
ram            2G
size           20G
tap            tap0
template       NO
vnc            NO
vnc_h          600
vnc_ip         127.0.0.1
vnc_tablet     NO
vnc_w          800
vnc_wait       NO

Installation goes fine, but when I go to do: iohyve start qb-db-server and attach to the console with iohyve console qb-db-server all I get is the grub console.

Not sure what, if anything, I should be changing during install, I just do a minimal install with the default partitioning, which I believe uses LVM.

This is the my zfs list output:

NAME                                                     USED  AVAIL  REFER  MOUNTPOINT
Stone                                                   29.8G  1.73T    88K  /mnt/Stone
Stone/.system                                           4.70M  1.73T    96K  legacy
Stone/.system/configs-786244a42181458097835dd50a77a6a0   264K  1.73T   264K  legacy
Stone/.system/configs-eab18b758b91471d95803a91d80bfcda   176K  1.73T   176K  legacy
Stone/.system/cores                                     1.27M  1.73T  1.27M  legacy
Stone/.system/rrd-786244a42181458097835dd50a77a6a0        88K  1.73T    88K  legacy
Stone/.system/rrd-eab18b758b91471d95803a91d80bfcda        88K  1.73T    88K  legacy
Stone/.system/samba4                                     540K  1.73T   540K  legacy
Stone/.system/syslog-786244a42181458097835dd50a77a6a0    700K  1.73T   700K  legacy
Stone/.system/syslog-eab18b758b91471d95803a91d80bfcda   1.53M  1.73T  1.53M  legacy
Stone/Excalibur                                         20.6M  1.73T  20.6M  /mnt/Stone/Excalibur
Stone/Merlin                                            4.41M  1.73T  4.41M  /mnt/Stone/Merlin
Stone/iohyve                                            28.2G  1.73T    88K  /mnt/iohyve
Stone/iohyve/Firmware                                     88K  1.73T    88K  /mnt/iohyve/Firmware
Stone/iohyve/ISO                                        7.54G  1.73T    88K  /mnt/iohyve/ISO
Stone/iohyve/ISO/CentOS7.iso                            7.54G  1.73T  7.54G  /mnt/iohyve/ISO/CentOS7.iso
Stone/iohyve/qb-db-server                               20.6G  1.73T    92K  /mnt/iohyve/qb-db-server
Stone/iohyve/qb-db-server/disk0                         20.6G  1.75T  1.21G  -
Stone/jails                                             1.60G  1.73T    88K  /mnt/Stone/jails
Stone/jails/.warden-template-standard                   1.60G  1.73T  1.60G  /mnt/Stone/jails/.warden-template-standard
freenas-boot                                            1.04G  13.4G   176K  none
freenas-boot/ROOT                                       1.03G  13.4G   136K  none
freenas-boot/ROOT/Initial-Install                          8K  13.4G   848M  legacy
freenas-boot/ROOT/configured_working                       8K  13.4G   849M  legacy
freenas-boot/ROOT/default                               1.03G  13.4G  1.02G  legacy
freenas-boot/grub                                       7.28M  13.4G  7.28M  legacy

The device.map for that is created when I do iohyve start qb-db-server is:

(hd0) /dev/zvol/Stone/iohyve/qb-db-server/disk0
(cd0) /dev/zvol/Stone/iohyve/qb-db-server/disk0

Not sure why it points both devices to the disk.

And here is the grub.cfg

linux (cd0)/isolinux/vmlinuz
initrd (cd0)/isolinux/initrd.img
boot

Which is what I think is the problem, it doesn't seem to get updated to point to the hd0 device.

Any insight you can provide would be great. Thanks!

[pr1ntf]

In the current case, that grub.cfg is only used during the install.

When using iohyve start, it looks for the installed grub.cfg in the guest's /grub2 folder.

I'll get ahold of a current CentOS7 ISO and try to reproduce.

[RobertDeRose]

Yeah from what I can tell, it's failing to be able to load the Partition claiming that it's an invalid XFS partition node, or something along those lines, I kind of gave up because even once I figured out how to manaully load the grub.cfg by hand in the console. Bhyve would coredump every time iohyve would call the next command, which was to launch Bhyve itself.

[Firemad03]

I can confirm that I saw this problem too with Centos 7. I used Debian instead for now. I thought it was me that did a mistake.

[pr1ntf]

I am seeing issues with grub2-bhyve and the XFS partitions that CentOS 7 uses. This is for both the default LVM installation and the "Standard" installation.

I'll continue to mess about and see if we can get CentOS 7 working again. Or at the very least detail how to configure the CentOS 7 installation to not use XFS for the boot partition.

[RobertDeRose]

@pr1ntf that second option sounds like the easier of the two for now as I would image the inability to handle XFS partitions probably needs to be addressed in grub2-bhyve. I can open an issue with them unless you think you might. However, I didn't bother to keep any of my logs :(

[pr1ntf]

The developer of grub2-bhyve is aware of the XFS issues.

Unless you manually partition your CentOS7 install to be ext4 by hand, there is no current way to get around this. I hope to have a recipe for this soon. (A bit out of scope but an important thing to have).

Also note that I have not tried to boot CentOS7 using the UEFI bootrom. Last I tried something like this, however, the VM did not survive a host reboot.

[jadzdotcom]

I was having similar issues trying a CentOS 6.8 install (post install grub menu). I believe 6.8 would be ext4. Do you think the issue is specific to XFS?

[RobertDeRose]

@pr1ntf thank you very much for the confirmation that these issues were not mine alone! And thanks for checking in with grub2-bhyve. Maybe there is a way to pass in a kickstart file like this one based on these RHEL docs.

Probably way out of scope. Maybe it you be possible to host prebuild OS images on BinTray or something like that and remove the need for recipes all together?

[RobertDeRose]

@jadzdotcom I did attempt 6.8 too, had segfault when Bhyve loaded on FreeBSD 10.3.

[pr1ntf]

@jadzdotcom
In this case, I have loaded CentOS7 "by hand" (not using iohyve), so I see the XFS error (an actual XFS error in grub2-bhyve), even on my machine.

There is another issue open right now with users of FreeNAS and booting GRUB guests in general with iohyve over in issue #228.

@RobertDeRose
That kickstart file looks simple enough to make a Wiki Entry for. I'll give that a go, thanks!

[pr1ntf]

I have added this to the roadmap. Closing the issue out. Thank you both again!