Test request: Hetzner ipv6 only instance

[bretton]

It's not known how the script works with ipv6-only instances. ipv4 addresses are an addon now.

There is a corresponding issue here
depenguin-me/depenguin-builder#2

[grembo]

@bretton Step 1: Make depenguin.me available over IPv6 :)

[bretton]

@bretton Step 1: Make depenguin.me available over IPv6 :)

doh! should be fixed

[grembo]

@bretton With the latest changes, I managed to do a full IPv6 only install now.

Server type: AX51-NVMe

This is how the procedure worked (there were a few bumps in the road):

Setup Procedure

• Log into Robot, reboot into Linux rescue mode using key based auth
• Run the standard depenguin.me procedure:

  wget https://depenguin.me/run.sh && \
    chmod 755 run.sh && \
    ./run.sh ~/.ssh/authorized_keys

• ssh into mfsbsd@<ipaddr> port 1022 as shown by run.sh
• Run:

  sudo bash
  /root/enable_ipv6.sh

• Pre-fetch bsd distribution files:

  cd /tmp
  mkdir download
  cd download
  fetch ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/13.1-RELEASE/MANIFEST
  fetch ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/13.1-RELEASE/base.txz
  fetch ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/13.1-RELEASE/kernel.txz
  fetch ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/13.1-RELEASE/kernel-dbg.txz
  fetch ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/13.1-RELEASE/lib32.txz
  ln -s /tmp/download /usr/freebsd-dist
  sha256 *txz

• Compare checksums with https://download.freebsd.org/ftp/releases/amd64/13.1-RELEASE/MANIFEST
• Run bsdinstall, skip over all network settings
• Auto ZFS setup with the following settings:
  • Pool Type/Disks: mirror
  • Pool Name: zroot
  • Force 4K Sectors?: YES
  • Encrypt Disks: NO
  • Partition Scheme: GPT (UEFI+BIOS)
  • Swap Size: 64GB
  • Mirror Swap: YES
  • Encrypt Swap: YES
• Finish setup, add unprivileged user and invited it to group wheel
• At the end of bsdinstall, select YES to open a shell in the new system
• Modify /etc/resolv.conf to contain Hetzner resolvers:

  cat >/etc/resolv.conf <<EOF
  nameserver 2a01:4ff:ff00::add:1
  nameserver 2a01:4ff:ff00::add:2 
  EOF

• Install a couple of useful packages:

  pkg install bash sudo joe openntpd

• Configure networking (replace 2001:dead:beef:12::2 with your server's IP):

  sysrc ifconfig_igb0_name="untrusted"
  sysrc ifconfig_untrusted_ipv6="up"
  sysrc ifconfig_untrusted_aliases="inet6 2001:dead:beef:12::2/64"
  sysrc ipv6_defaultrouter="fe80::1%untrusted"

• Configure sudo:

  cat > /usr/local/etc/sudoers.d/wheel<<EOF
  %wheel ALL=(ALL) NOPASSWD: ALL
  EOF

• Enable openntpd

   service openntpd enable

• Alter /etc/fstab to contain the correct partition name for the EFI partition (and comment it out):

  # Device                Mountpoint      FStype  Options         Dump    Pass#
  #/dev/nvd0p1            /boot/efi       msdosfs rw              2       2
  /dev/mirror/swap.eli            none    swap    sw              0       0
  

Gotchas

• For this server, the correct network interface name was igb0, not em0 what is used while configuring it within QEMU.
• The NVMe SSDs will be called /dev/nvd0 and /dev/nvd1 once booted, within QEMU they are called /dev/ada0 and /dev/ada1. When installing a UEFI enabled boot loader, this creates the wrong entry in /etc/fstab, which will stop the server from booting correctly. It's best to comment out that line if it isn't needed (or correct the device name).
• You can switch your server to boot from UEFI at Hetzner (quick support ticket is enough, they are very responsive). Be aware that the boot order might be off by accident and therefore the rescue system might not work anymore. If this is the case, request a KVM remote console and enter the BIOS (F2) on boot to change the boot order to be:
  1. PXE IP4
  2. UEFI OS (SSD)
  3. EFI Menu
  4. PXE IP6

Screenshots

bsdinstall ZFS settings

For most applications it might make more sense to select GPT (BIOS) - i.e., no UEFI - as the partition scheme in bsdinstall.

Correct UEFI BIOS Boot Order

This will allow to keep using the rescue system, which relies on PXE-boot:

[bretton]

thanks @grembo

is the scripted bsdinstall outlined at https://depenguin.me/bsdinstall.html not sufficient?

adapts https://github.com/depenguin-me/depenguin-builder/blob/main/customfiles/INSTALLERCONFIG.sample to variables

necessary updates can be added there

[grembo]

thanks @grembo

is the scripted bsdinstall outlined at https://depenguin.me/bsdinstall.html not sufficient?

adapts https://github.com/depenguin-me/depenguin-builder/blob/main/customfiles/INSTALLERCONFIG.sample to variables

necessary updates can be added there

I didn't test the scripted one, as I was following "my" standard setup, sorry. But I would probably not include IP setup that way and the remarks regarding GPT+UEFI still apply :)

[bretton]

I'll close this as successful test

[grembo]

NOTE: Since FreeBSD 13.2, the correct gpt label is used in /etc/fstab, so commenting out is not mandatory anymore.