pvgrub boot: stuck at trying to make /dev/mapper/dmroot appear

[Rudd-O]

Latest F25 and R3.2 software.

With this playbook (which details the steps) running against both dom0 and a standalone AppVM called testply:

- hosts: dom0
  become: yes
  tasks:
  - include_role:
      name: install-packages
    vars:
      packages:
      - grub2-xen

- hosts: testply
  become: yes
  tasks:
  - ini_file:
      path: /etc/yum.repos.d/qubes-r3.repo
      section: qubes-vm-r3.2-current
      option: exclude
      value: kernel, kernel-devel
  - shell: rpm -q kernel-devel | grep qubes && { rpm -e --nodeps kernel-devel ; exit $? ; } || true
  - package: name=qubes-kernel-vm-support,kernel-modules,kernel-devel,grub2-tools,dkms state=present
  - shell: rpm -q kernel-devel --queryformat="%{version}-%{release}.%{arch}"
    register: kver
    check_mode: no
    changed_when: no
  - package: name=kernel-{{ kver.stdout }} state=present
  - shell: /usr/sbin/qubes-prepare-vm-kernel {{ kver.stdout }}
  - lineinfile: state=present create=true dest=/etc/default/grub regexp="^GRUB_TIMEOUT=" line="GRUB_TIMEOUT=0"
  - shell: grub2-mkconfig -o /boot/grub2/grub.cfg

- hosts: dom0
  tasks:
  - shell: qvm-prefs testply -s kernel pvgrub2

- hosts: testply
  become: yes
  tasks:
  - include: tasks/reboot.yml
  - action: setup

After the reboot, testply hangs at the initrd stage, reaching target Basic System and going no further. Eventually, boot gives up and says it got tired of waiting for /dev/mapper/dmroot.

[Rudd-O]

Cannot repro with TemplateVM-based AppVM. Can only repro with Standalone AppVM.

[Rudd-O]

Cannot repro with TemplateVM-based AppVM. Can only repro with Standalone AppVM.

[Rudd-O]

Updated playbook:

- hosts: dom0
  become: yes
  tasks:
  - include_role:
      name: install-packages
    vars:
      packages:
      - grub2-xen
  - shell: qvm-prefs testply -s kernel default
    become: False

- hosts: testply
  become: yes
  tasks:
  - shell: head -1 /etc/yum.repos.d/qubes-r3.repo
    register: reponame
    check_mode: no
    changed_when: no
  
  - ini_file:
      path: /etc/yum.repos.d/qubes-r3.repo
      section: '{{ reponame.stdout_lines[0][1:-1] }}'
      option: exclude
      value: kernel, kernel-devel

  - shell: rpm -q kernel-devel | grep qubes && { rpm -e --nodeps kernel-devel ; exit $? ; } || true

  - shell: rpm -q qubes-kernel-vm-support --queryformat="%{version}" || true
    register: u2mfnver
    check_mode: no
    changed_when: no

  - shell: rpm -e --nodeps qubes-kernel-vm-support
    when: u2mfnver.stdout.strip() != "" and u2mfnver.stdout|version_compare("3.2.3", "<=")

  - package:
      name: qubes-kernel-vm-support,kernel-modules,kernel-devel,grub2-tools,dkms
      state: present
      enablerepo: '*qubes-vm*testing*'

  - shell: rpm -q kernel-devel --queryformat="%{version}-%{release}.%{arch}"
    register: kver
    check_mode: no
    changed_when: no

  - package: name=kernel-{{ kver.stdout }} state=present

  - shell: /usr/sbin/qubes-prepare-vm-kernel {{ kver.stdout }}

  - lineinfile: state=present create=true dest=/etc/default/grub regexp="^GRUB_TIMEOUT=" line="GRUB_TIMEOUT=0"

  - shell: grub2-mkconfig -o /boot/grub2/grub.cfg

  - include: tasks/reboot.yml reboot_command=poweroff

- hosts: dom0
  tasks:
  - shell: qvm-prefs testply -s kernel pvgrub2

- hosts: testply
  become: yes
  tasks:
  - debug: msg={{ ansible_distribution }}
#  - include: tasks/reboot.yml reboot_command=poweroff

[Rudd-O]

Updated playbook:

- hosts: dom0
  become: yes
  tasks:
  - include_role:
      name: install-packages
    vars:
      packages:
      - grub2-xen
  - shell: qvm-prefs testply -s kernel default
    become: False

- hosts: testply
  become: yes
  tasks:
  - shell: head -1 /etc/yum.repos.d/qubes-r3.repo
    register: reponame
    check_mode: no
    changed_when: no
  
  - ini_file:
      path: /etc/yum.repos.d/qubes-r3.repo
      section: '{{ reponame.stdout_lines[0][1:-1] }}'
      option: exclude
      value: kernel, kernel-devel

  - shell: rpm -q kernel-devel | grep qubes && { rpm -e --nodeps kernel-devel ; exit $? ; } || true

  - shell: rpm -q qubes-kernel-vm-support --queryformat="%{version}" || true
    register: u2mfnver
    check_mode: no
    changed_when: no

  - shell: rpm -e --nodeps qubes-kernel-vm-support
    when: u2mfnver.stdout.strip() != "" and u2mfnver.stdout|version_compare("3.2.3", "<=")

  - package:
      name: qubes-kernel-vm-support,kernel-modules,kernel-devel,grub2-tools,dkms
      state: present
      enablerepo: '*qubes-vm*testing*'

  - shell: rpm -q kernel-devel --queryformat="%{version}-%{release}.%{arch}"
    register: kver
    check_mode: no
    changed_when: no

  - package: name=kernel-{{ kver.stdout }} state=present

  - shell: /usr/sbin/qubes-prepare-vm-kernel {{ kver.stdout }}

  - lineinfile: state=present create=true dest=/etc/default/grub regexp="^GRUB_TIMEOUT=" line="GRUB_TIMEOUT=0"

  - shell: grub2-mkconfig -o /boot/grub2/grub.cfg

  - include: tasks/reboot.yml reboot_command=poweroff

- hosts: dom0
  tasks:
  - shell: qvm-prefs testply -s kernel pvgrub2

- hosts: testply
  become: yes
  tasks:
  - debug: msg={{ ansible_distribution }}
#  - include: tasks/reboot.yml reboot_command=poweroff

[awokd]

@Rudd-O Just ran into this too. Try regenerating your initramfs before first reboot:

dracut -f /boot/initramfs-4.14.16-200.fc26.x86_64.img 4.14.16-200.fc26.x86_64

Also, first boot takes a long time (I think it's initializing swap space), so qrexec might time out. Maybe just hard reboot it again after 5 minutes, then it should work normally.

[awokd]

@Rudd-O Just ran into this too. Try regenerating your initramfs before first reboot:

dracut -f /boot/initramfs-4.14.16-200.fc26.x86_64.img 4.14.16-200.fc26.x86_64

Also, first boot takes a long time (I think it's initializing swap space), so qrexec might time out. Maybe just hard reboot it again after 5 minutes, then it should work normally.