installation/guides/zfs.md: new root-on-ZFS guide

[ahesford]

ZFSBootMenu strives to provide an excellent root-on-ZFS experience with Void Linux, so let's refer users to it (and its step-by-step guides) when possible. In the interest of balance, I've also included some necessary setup to prepare a ZFS hierarchy as a chroot target following the official instructions.

We probably need to iterate on this.

cc: @ericonr @zdykstra @Vaelatern @paper42

[ahesford]

The latest push made a couple of minor edits and now notes that hrmpf is maintained by a Void team member.

[Vaelatern]

Notes for iteration:

• Would include wake-from-hibernate instructions
• Would break up wall of text paragraph to shorter groupings

[HadetTheUndying]

So I'm having a few issues. I'm using refind on top of zfsbootmenu and my system will boot fine on the first boot then refind gets stuck if i reboot after booting into the fresh install. The menu entries are definitely there. I also can't boot into the ZFS menu after that. I know that's outside of the scope of the initial guide, but it's relevant to the linked zbm documentation for UEFI and encrypted installs. I've done this 3 times now in a VM and all 3 times have encountered this same issue. I'm not sure if using grub would solve it and at least in my vm i was unable to use zfsbootmenu directly, I get dropped into emergency UEFI shell when i do that. If these bootloader specific issues are going to be a problem there probably needs to specific documentation in the handbook for this, as the zbm repo's guide for installation is leading me to this issue every time.

There's also a couple typos here and there that i can try to highlight in a bit.

Also I just noticed the bit about having a separate /boot to avoid some issues I will try that in a bit and see if it solves the specific problem I'm having.

[zdykstra]

So I'm having a few issues. I'm using refind on top of zfsbootmenu and my system will boot fine on the first boot then refind gets stuck if i reboot after booting into the fresh install. The menu entries are definitely there. I also can't boot into the ZFS menu after that. I know that's outside of the scope of the initial guide, but it's relevant to the linked zbm documentation for UEFI and encrypted installs. I've done this 3 times now in a VM and all 3 times have encountered this same issue. I'm not sure if using grub would solve it and at least in my vm i was unable to use zfsbootmenu directly, I get dropped into emergency UEFI shell when i do that. If these bootloader specific issues are going to be a problem there probably needs to specific documentation in the handbook for this, as the zbm repo's guide for installation is leading me to this issue every time.

There's also a couple typos here and there that i can try to highlight in a bit.

Also I just noticed the bit about having a separate /boot to avoid some issues I will try that in a bit and see if it solves the specific problem I'm having.

There isn't enough detail in this to help narrow down what might be happening here. I'd lean strongly towards a weird interaction with rEFInd and the UEFI firmware used in your VM, however. I'd recommend cutting rEFInd out entirely and directly booting the ZFSBootMenu EFI to reduce complicated interactions.

I haven't heard of anybody on bare metal having this issue.

[HadetTheUndying]

So I'm having a few issues. I'm using refind on top of zfsbootmenu and my system will boot fine on the first boot then refind gets stuck if i reboot after booting into the fresh install. The menu entries are definitely there. I also can't boot into the ZFS menu after that. I know that's outside of the scope of the initial guide, but it's relevant to the linked zbm documentation for UEFI and encrypted installs. I've done this 3 times now in a VM and all 3 times have encountered this same issue. I'm not sure if using grub would solve it and at least in my vm i was unable to use zfsbootmenu directly, I get dropped into emergency UEFI shell when i do that. If these bootloader specific issues are going to be a problem there probably needs to specific documentation in the handbook for this, as the zbm repo's guide for installation is leading me to this issue every time.
There's also a couple typos here and there that i can try to highlight in a bit.
Also I just noticed the bit about having a separate /boot to avoid some issues I will try that in a bit and see if it solves the specific problem I'm having.

There isn't enough detail in this to help narrow down what might be happening here. I'd lean strongly towards a weird interaction with rEFInd and the UEFI firmware used in your VM, however. I'd recommend cutting rEFInd out entirely and directly booting the ZFSBootMenu EFI to reduce complicated interactions.

I haven't heard of anybody on bare metal having this issue.

hrmpf is having issues booting into uefi mode on qemu with omvf for example, the regular void image however doesn't have this issue, just requires a little more initial setup. Still having a ton of issues because of this and the refind issue. Once i get a fully reproducible setup going, I'll make some suggestions for changes. I know that's somewhat out of scope of this discussion i guess but here's what happens when using the hrmpf image

(chroot) # efibootmgr -c -d /dev/vda -p 1 -L "ZFSBootMenu" -l \\EFI\\VOID\\VMLINUZ.EFI
EFI variables are not supported on this system.
(chroot) # ls /sys/firmware/
acpi  dmi  efi	memmap	qemu_fw_cfg
(chroot) # ls /sys/firmware/efi/
config_table  efivars  fw_platform_size  fw_vendor  runtime  runtime-map  systab

This does not happen when using the latest Void image i don't have the EFI issue and can boot into the new environment ONCE with refind, but afterwards refind gets stuck or confused or something. I'm following the documentation line for line, only changing UUIDs and drive labels where needed. I will try direct ZFS boot doing the install from the base Void image later tonight or early tomorrow.

[HadetTheUndying]

Okay so coming back to this. The EFI booting issues were definitely to do with qemu +omvf for sure. I have a properly working encrypted ZFS on root on bare metal. I was able to follow the guide and zbm guide to setup UEFI on one system and BIOS on another. I do think that a little more emphasis needs to be made on bootloader setup, and I would certainly recommend using refind over GRUB for UEFI as it only takes changing one file to configure correctly.

[ahesford]

I want to de-emphasize bootloader setup because, if you have a separate /boot, the configuration isn't any different from any other system. The point of the guide is to highlight where things need to be different, so that people can follow the standard guide and deviate only where necessary.

[ahesford]

I have broken up the installation instructions with some headers. Under the partitioning section, I added a brief discussion about putting swap on zvols to emphasize the need for a dedicated swap partition when using hibernation. Otherwise, resuming from suspended images is no different than on other systems.