OpenCore default boot selection doesn't work with OVMF.

[Pavo-IM]

Currently using Proxmox which uses Qemu+KVM and selecting Default boot disk in macOS does not set the default boot item in OpenCore boot menu. It always defaults to the first item in the list no matter the order.

[vit9696]

No log — no bug.

[Pavo-IM]

No log — no bug.

Not sure what kind of log you would need for this, but I will upload a Debug OpenCore log I guess.

[vit9696]

Yes, need a debug OpenCore log after the selection and EFI folder.

[Pavo-IM]

Here you go....
opencore-2020-07-18-211811.txt

[Pavo-IM]

Here is the EFI folder.
EFI.zip

[vit9696]

Not all the drivers are debug, so the log is incorrect.

[Pavo-IM]

Compiled it again and updated all the drivers except HFSPlus.efi to the Debug build ones because I can not find a Debug version of HFSPlus. Switch to Builtin boot picker menu.
opencore-2020-07-18-220743.txt

[vit9696]

macOS has:

PciRoot(0x1)/Pci(0x1C,0x1)/Pci(0x0,0x0)/SasEx(0x0100000000253852,0x71B0FB3E7074616C,0xAFAF,0xAFAF,0,0,0)/HD(2,GPT,1D094ACD-E7FC-494C-9E81-A3C5E5DC433C,0x64028,0x3A321FE0)/VenMedia(BE74FCF7-0B7C-49F3-9147-01F4042E6842,7B341CB285D4CE4788AA4C17C4C3F39E)/\321073E3-7B45-44E3-834A-C853F1C2C38E\System\Library\CoreServices\boot.efi

OVMF has:

PciRoot(0x0)/Pci(0x1C,0x1)/Pci(0x0,0x0)/NVMe(0x1,3E-FB-B0-71-52-38-25-00)/HD(2,GPT,1D094ACD-E7FC-494C-9E81-A3C5E5DC433C,0x64028,0x3A321FE0)/VenMedia(BE74FCF7-0B7C-49F3-9147-01F4042E6842,7B341CB285D4CE4788AA4C17C4C3F39E)/\321073E3-7B45-44E3-834A-C853F1C2C38E\System\Library\CoreServices\boot.efi

NVMe/SasEx we do fix, but PciRoot 1/0 are still different. Did you reconfigure your VM between choosing the operating system and saving the log? It looks like the order of PCI controllers changed to me.

[Pavo-IM]

I did not, I compiled the Debug version of OpenCorePkg and updated my EFI with the Debug drivers and went to System Preferences > Startup Disk and selected MacOS then when it asked to restart I clicked Restart. On reboot, I hit F2 for OVMF boot menu and selected the USB drive with the Debug OpenCore EFI on it and booted it, then once OpenCore boot menu loaded, It still defaulted to the first in the list which was my EFI on my actual drive and had to manually select MacOS from the boot menu to boot into macOS.

[vit9696]

Is this the latest version of OVMF? Below I attached the latest NOOPT version and the latest NOOPT version with serial output. Please check whether the issue is reproducible in the first version and provide the boot log with the second version.

Also, please provide the configuration you use for your virtual machine. Do you use multiple PCI bridges? What is the reason for this?

NOOPT.zip
NOOPT-SERIAL.zip

[Pavo-IM]

Checking the OVMF version you upload but here is the vm.conf file settings.

args: -device isa-applesmc,osk="ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc" -cpu host,vendor=GenuineIntel,+invtsc
balloon: 0
bios: ovmf
bootdisk: ide0
cores: 64
cpu: host
efidisk0: local-lvm:vm-100-disk-0,size=4M
hostpci0: 23:00,pcie=1,x-vga=1
hostpci1: 01:00.0,pcie=1
hostpci10: 25:00.3,pcie=1
hostpci11: 46:00.0,pcie=1
hostpci2: 02:00.0,pcie=1
hostpci3: 43:00.0,pcie=1
hostpci4: 44:00.0,pcie=1
hostpci5: 47:00.0,pcie=1
hostpci6: 48:00.1,pcie=1
hostpci7: 4b:00.0,pcie=1
hostpci8: 04:00.3,pcie=1
hostpci9: 48:00.3,pcie=1
localtime: 1
machine: q35
memory: 122880
name: BigSur
numa: 1
ostype: win10
scsihw: virtio-scsi-pci
smbios1: uuid=6623598a-7a98-4b99-8229-e44ed0d3568c
sockets: 1
tablet: 0
vga: none
vmgenid: d3eb685a-544e-4936-b26a-89f3e0fec696

[vit9696]

Could you attach not the vm.conf, but the resulting arguments passed to QEMU? Also, please make SysReport with ACPI tables.

[Pavo-IM]

Could you attach not the vm.conf, but the resulting arguments passed to QEMU?

I think this is what you are talking about....

/usr/bin/kvm -id 100 -name BigSur -chardev 'socket,id=qmp,path=/var/run/qemu-server/100.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' -mon 'chardev=qmp-event,mode=control' -pidfile /var/run/qemu-server/100.pid -daemonize -smbios 'type=1,uuid=6623598a-7a98-4b99-8229-e44ed0d3568c' -drive 'if=pflash,unit=0,format=raw,readonly,file=/usr/share/pve-edk2-firmware//OVMF_CODE.fd' -drive 'if=pflash,unit=1,format=raw,id=drive-efidisk0,size=131072,file=/dev/pve/vm-100-disk-0' -smp '64,sockets=1,cores=64,maxcpus=64' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' -vga none -nographic -no-hpet -cpu 'host,hv_ipi,hv_relaxed,hv_reset,hv_runtime,hv_spinlocks=0x1fff,hv_stimer,hv_synic,hv_time,hv_vapic,hv_vendor_id=proxmox,hv_vpindex,kvm=off,+kvm_pv_eoi,+kvm_pv_unhalt' -m 122880 -object 'memory-backend-ram,id=ram-node0,size=122880M' -numa 'node,nodeid=0,cpus=0-63,memdev=ram-node0' -readconfig /usr/share/qemu-server/pve-q35-4.0.cfg -device 'vmgenid,guid=d3eb685a-544e-4936-b26a-89f3e0fec696' -device 'vfio-pci,host=0000:23:00.0,id=hostpci0.0,bus=ich9-pcie-port-1,addr=0x0.0,multifunction=on' -device 'vfio-pci,host=0000:23:00.1,id=hostpci0.1,bus=ich9-pcie-port-1,addr=0x0.1' -device 'vfio-pci,host=0000:01:00.0,id=hostpci1,bus=ich9-pcie-port-2,addr=0x0' -device 'vfio-pci,host=0000:02:00.0,id=hostpci2,bus=ich9-pcie-port-3,addr=0x0' -device 'vfio-pci,host=0000:43:00.0,id=hostpci3,bus=ich9-pcie-port-4,addr=0x0' -device 'pcie-root-port,id=ich9-pcie-port-5,addr=10.0,x-speed=16,x-width=32,multifunction=on,bus=pcie.0,port=5,chassis=5' -device 'vfio-pci,host=0000:44:00.0,id=hostpci4,bus=ich9-pcie-port-5,addr=0x0' -device 'pcie-root-port,id=ich9-pcie-port-6,addr=10.1,x-speed=16,x-width=32,multifunction=on,bus=pcie.0,port=6,chassis=6' -device 'vfio-pci,host=0000:47:00.0,id=hostpci5,bus=ich9-pcie-port-6,addr=0x0' -device 'pcie-root-port,id=ich9-pcie-port-7,addr=10.2,x-speed=16,x-width=32,multifunction=on,bus=pcie.0,port=7,chassis=7' -device 'vfio-pci,host=0000:48:00.1,id=hostpci6,bus=ich9-pcie-port-7,addr=0x0' -device 'pcie-root-port,id=ich9-pcie-port-8,addr=10.3,x-speed=16,x-width=32,multifunction=on,bus=pcie.0,port=8,chassis=8' -device 'vfio-pci,host=0000:4b:00.0,id=hostpci7,bus=ich9-pcie-port-8,addr=0x0' -device 'pcie-root-port,id=ich9-pcie-port-9,addr=10.4,x-speed=16,x-width=32,multifunction=on,bus=pcie.0,port=9,chassis=9' -device 'vfio-pci,host=0000:04:00.3,id=hostpci8,bus=ich9-pcie-port-9,addr=0x0' -device 'pcie-root-port,id=ich9-pcie-port-10,addr=10.5,x-speed=16,x-width=32,multifunction=on,bus=pcie.0,port=10,chassis=10' -device 'vfio-pci,host=0000:48:00.3,id=hostpci9,bus=ich9-pcie-port-10,addr=0x0' -device 'pcie-root-port,id=ich9-pcie-port-11,addr=10.6,x-speed=16,x-width=32,multifunction=on,bus=pcie.0,port=11,chassis=11' -device 'vfio-pci,host=0000:25:00.3,id=hostpci10,bus=ich9-pcie-port-11,addr=0x0' -device 'pcie-root-port,id=ich9-pcie-port-12,addr=10.7,x-speed=16,x-width=32,multifunction=on,bus=pcie.0,port=12,chassis=12' -device 'vfio-pci,host=0000:46:00.0,id=hostpci11,bus=ich9-pcie-port-12,addr=0x0' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:e9957dd2fb64' -rtc 'driftfix=slew,base=localtime' -machine 'type=q35+pve0' -global 'kvm-pit.lost_tick_policy=discard' -device 'isa-applesmc,osk=ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc' -cpu 'host,vendor=GenuineIntel,+invtsc'

[Pavo-IM]

Is this the latest version of OVMF? Below I attached the latest NOOPT version and the latest NOOPT version with serial output. Please check whether the issue is reproducible in the first version and provide the boot log with the second version.

Also, please provide the configuration you use for your virtual machine. Do you use multiple PCI bridges? What is the reason for this?

NOOPT.zip
NOOPT-SERIAL.zip

When I try to use either one of these OVMF_CODE.fd and OVMF_VARS.fd, I do not get any display.

[Pavo-IM]

Nevermind I figured it out on the no display thing, issue was reproduced with NOOPT version and here is the debug log got the NOOPT-Serial version.
opencore-2020-07-18-202650.txt

[Pavo-IM]

Here is the SysReport with ACPI tables.
SysReport.zip

[vit9696]

Serial, I mean prior to OpenCore. When it boots it prints stuff to stdout. E.g. you can add an argument to log serial to file like -serial /path/to.txt

[Pavo-IM]

Here is the output with NOOPT-Serial and using Debug OpenCore after selecting macOS as the Startup Disk.
NOOPT-Serial_OpenCore_Debug_output.txt
OpenCore Debug Log:
opencore-2020-07-19-110754.txt

[vit9696]

OVMF gets ACPI tables straight from QEMU: hw/i386/acpi-build.c, which hardcodes PCI0 UID as 1:

        sb_scope = aml_scope("_SB");
        dev = aml_device("PCI0");
        aml_append(dev, aml_name_decl("_HID", aml_eisaid("PNP0A08")));
        aml_append(dev, aml_name_decl("_CID", aml_eisaid("PNP0A03")));
        aml_append(dev, aml_name_decl("_ADR", aml_int(0)));
        aml_append(dev, aml_name_decl("_UID", aml_int(1)));
        aml_append(dev, build_q35_osc_method());

This is obviously wrong, as pcibus_num for pci_bus_is_root returns 0 for the root PCI bus and subsequent PCI bus instances have 1+ IDs if present (and are handled lower). Besides, there is an interesting case with cpu hotplug, implemented in hw/acpi/cpu_hotplug.c, which contains something completely insane:

    dev = aml_device("PCI0." stringify(CPU_HOTPLUG_RESOURCE_DEVICE));
    aml_append(dev, aml_name_decl("_HID", aml_eisaid("PNP0A06")));
    aml_append(dev,
        aml_name_decl("_UID", aml_string("CPU Hotplug resources"))
    );

All in all, replacing the line in the first code snippet (twice) by UID 0 and recompiling QEMU should solve the issue:

aml_append(dev, aml_name_decl("_UID", aml_int(0)));

Given all that we need to decide:

• Whether to report this to QEMU/OVMF?
• Whether to add workarounds on our side?
• Whether to submit patches fixing this?

@pavo, try this change and report whether it works for you.

[vit9696]

This is now a confirmed bug in QEMU, tracked in the mailing list here:

https://www.mail-archive.com/qemu-devel@nongnu.org/msg724254.html

We need to provide them the details whether the suggested fix works to get this merged into the next version of QEMU. @Pavo-IM, do you need extra help with rebuilding QEMU with this change?

[Pavo-IM]

I will try but it might take sometime since I have never built Qemu before, I am reading the docs as we speak.

[vit9696]

Rebuilding QEMU is straightforward, you just run configure and specify --prefix argument to the installation directory. However, if you have a specific Linux distribution, like Debian or Ubuntu, I believe you may be able to utilise easier way to customise the build process. See this link for an example: https://unix.stackexchange.com/questions/324680/how-to-apply-a-patch-in-a-debian-package

If you tell us the exact distribution you use we could try to help further.

[Pavo-IM]

Roger, the exact distro I am using is Debian 10 (Buster).

[vit9696]

Try this please (you will need to replace system-installed binary, in /usr/bin I guess).

qemu-system-x86_64-patched.zip

[Pavo-IM]

Getting this error after replacing the qemu-system-x86_64 binary at /usr/bin

/usr/bin/kvm: error while loading shared libraries: libvirglrenderer.so.0: cannot open shared object file: No such file or directory
command '/usr/bin/kvm --version' failed: exit code 127
Detected old QEMU binary ('unknown', at least 3.0 is required)

[vit9696]

Are you sure you have the latest version of Debian? Please run sudo apt-get update and then sudo apt-get dist-upgrade and reboot. Also provide the output of uname -a after rebooting if the issue is unresolved.

[Pavo-IM]

Yeah, Proxmox is a custom made Debian 10 distro, it is using Linux pve 5.7.8-1 kernel. I will try and build the custom version of Qemu from Proxmox's repos.

[vit9696]

If you upload qemu-system-x86-64 original binary, I can modify it similarly.

[Pavo-IM]

Here is the original binary.
qemu-system-x86_64.zip

[vit9696]

Done

qemu-system-x86_64-fix.zip

[Pavo-IM]

@vit9696 The patched binary worked beautifully. Default boot selection is working as intended now, thank you so much. I will submit this patch to Proxmox devs so they can add it to their upstream repo.

[vit9696]

It will be good indeed if they backport these changes to your distro. We will report it to QEMU team, so that the change lands to master.

We found a similar issue with Intel G33, and might actually add a workaround to OpenCore as well, but this is out of the scope of this issue. Closing.