[UEFI-Only] Xen crashes before linux is loaded on AMD R7 1800X system

[shawnanastasio]

Qubes OS version:

Qubes OS 4.0 x86_64

Affected component(s):

CPU: AMD R7 1800X
GPU 1 (Main PCIe slot, connected to display): GTX 1080Ti
GPU 2 (Secondary PCIe slot): GTX 1080
RAM: 32GB DDR4
Motherboard: ASUS Prime X370, BIOS version 3805

---

Steps to reproduce the behavior:

1. Boot Qubes USB in UEFI mode, Secure Boot disabled.
2. Observe behavior

Expected behavior:

The installer should boot successfully as it does when booted using UEFI-CSM legacy mode on the same system.

Actual behavior:

Some Xen output is displayed up until the following:

Dom0 has maximum 16 VCPUs
Setup 0000:00:00.2 for d0 failed (-19)
Scrubbing Free RAM on 1 nodes using 8 CPUs
................................done.
Initial low memory virq threshold set at 0x4000 pages.
Std. Loglevel: Erros and warnings
Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
Xen is relinquishing VGA console.

At this point, the screen goes blank and the system is rebooted.

General notes:

The UEFI troubleshooting guide has been followed and the same behavior is produced when using /mapbs, /noexitboot, uefi=attr=uc and any combination of them.

The system boots successfully in legacy mode, and the USB install media has been confirmed working in UEFI mode on another system.

---

Related issues:

[Polygonbugs]

That's interesting that someone using Ryzen with dGPU... Although, Ryzen 1800x is on qubes compatibility list but if you look closely, it is using very old graphic card such as GTX 480 which qubes official document is based on (very old Nvidia graphic card or AMD graphic card. Not modern one. I've only seen someone using Intel xeon processor with maxwell series but not pascal series. I doubt he used custom kernel...).I'm not sure that you are using your own custom kernel to solve it but generally Qubes-OS recommend Intel iGPU.

The system boots successfully in legacy mode, and the USB install media has been confirmed working in UEFI mode on another system.

Don't you have any weird Graphic problem? Such as laggy and very low resolution? Is it possible on fedora-26 and PVH mode of dom0 in Qubes-OS 4?

BTW, there is some compatibilty issue with Ryzen not supporting memory remapping by some qubes-users on that compatibilty list. It could be that problem. Maybe turning on memory remap on bios could solve problem...

p.s I also want to use dGPU on Qubes OS as well. Because most of high end processors has no integrated GPU, that limits the Qubes OS hardware support like ECC memory support and memory size(>64GB), more PCI lanes support from CPU (>16), of course more cores and thread (>12 or >16), better virtualization technique(more performance), and could be physical mitigation from Meltdown.

[Polygonbugs]

That's interesting that someone using Ryzen with dGPU... Although, Ryzen 1800x is on qubes compatibility list but if you look closely, it is using very old graphic card such as GTX 480 which qubes official document is based on (very old Nvidia graphic card or AMD graphic card. Not modern one. I've only seen someone using Intel xeon processor with maxwell series but not pascal series. I doubt he used custom kernel...).I'm not sure that you are using your own custom kernel to solve it but generally Qubes-OS recommend Intel iGPU.

The system boots successfully in legacy mode, and the USB install media has been confirmed working in UEFI mode on another system.

Don't you have any weird Graphic problem? Such as laggy and very low resolution? Is it possible on fedora-26 and PVH mode of dom0 in Qubes-OS 4?

BTW, there is some compatibilty issue with Ryzen not supporting memory remapping by some qubes-users on that compatibilty list. It could be that problem. Maybe turning on memory remap on bios could solve problem...

p.s I also want to use dGPU on Qubes OS as well. Because most of high end processors has no integrated GPU, that limits the Qubes OS hardware support like ECC memory support and memory size(>64GB), more PCI lanes support from CPU (>16), of course more cores and thread (>12 or >16), better virtualization technique(more performance), and could be physical mitigation from Meltdown.

[shawnanastasio]

You're right that even when I am able to boot Qubes, the video acceleration is nearly unusable. It seems nouveau on 4.14 doesn't have very good Pascal support. Since it still boots in legacy mode, though, I doubt that the GPU is the source of this issue.

I'll soon have access to another GPU that I have confirmed to work fully in Qubes 4 (AMD RX 460) and I'll update this issue once I test it in this machine.

As for memory remapping, I don't see any option like that in my UEFI settings. Perhaps my motherboard vendor has renamed it or opted not to include it.

[shawnanastasio]

You're right that even when I am able to boot Qubes, the video acceleration is nearly unusable. It seems nouveau on 4.14 doesn't have very good Pascal support. Since it still boots in legacy mode, though, I doubt that the GPU is the source of this issue.

I'll soon have access to another GPU that I have confirmed to work fully in Qubes 4 (AMD RX 460) and I'll update this issue once I test it in this machine.

As for memory remapping, I don't see any option like that in my UEFI settings. Perhaps my motherboard vendor has renamed it or opted not to include it.

[Polygonbugs]

I'll soon have access to another GPU that I have confirmed to work fully in Qubes 4 (AMD RX 460) and I'll update this issue once I test it in this machine.

AMD has nice support with opensource driver in linux. I hope you work :)

[Polygonbugs]

I'll soon have access to another GPU that I have confirmed to work fully in Qubes 4 (AMD RX 460) and I'll update this issue once I test it in this machine.

AMD has nice support with opensource driver in linux. I hope you work :)

[0spinboson]

Yes, nouveau's Pascal support is horrid, still, sadly, because NV forces them to reinvent the wheel.
Ryzen/TR/Epyc is fully supported by Xen, so there shouldn't be any issues with that (the only issue with 'remapping' was that it wasn't detected properly by the hcl-report tool due to an uppercase letter being used). And so long as you pick the right motherboard, you can get ecc support with any of those CPUs.

As for graphics rendering: it works fine here (4k60 screen via DP on hd7750 and vega56, only problem is that you can't install using the Vega card because you need kernel 4.15 for that). Main drawback of the hd7750 is lack of freesync support, which kinda sucks.

[0spinboson]

Yes, nouveau's Pascal support is horrid, still, sadly, because NV forces them to reinvent the wheel.
Ryzen/TR/Epyc is fully supported by Xen, so there shouldn't be any issues with that (the only issue with 'remapping' was that it wasn't detected properly by the hcl-report tool due to an uppercase letter being used). And so long as you pick the right motherboard, you can get ecc support with any of those CPUs.

As for graphics rendering: it works fine here (4k60 screen via DP on hd7750 and vega56, only problem is that you can't install using the Vega card because you need kernel 4.15 for that). Main drawback of the hd7750 is lack of freesync support, which kinda sucks.

[Polygonbugs]

@0spinboson , That's very good news! So maybe most of the problem come from bad graphic driver support for Nvidia on linux. Even quite brand new graphic driver from AMD(opensource so linux developer can easily implement) works!
But question here, if CPU+motherboard supports well with Qubes-OS and some higher linux kernel is supported by Qubes, then is it very unlikely Qubes Environment(for now it is Xen) affects graphic issue? Just wondering...

[Polygonbugs]

@0spinboson , That's very good news! So maybe most of the problem come from bad graphic driver support for Nvidia on linux. Even quite brand new graphic driver from AMD(opensource so linux developer can easily implement) works!
But question here, if CPU+motherboard supports well with Qubes-OS and some higher linux kernel is supported by Qubes, then is it very unlikely Qubes Environment(for now it is Xen) affects graphic issue? Just wondering...

[0spinboson]

I don't quite understand your question, could you rephrase what has you worried? There is little point to having a 'good' graphics card in dom0, because most of what it displays is x11-forwarded graphical data, so 3d acceleration is wasted on dom0.
What you can do (though it's not trivial) is attach a good GPU to a VM, then connect a monitor to that GPU, and game on it. But for now, dual-boot for gaming is probably preferable.

[0spinboson]

I don't quite understand your question, could you rephrase what has you worried? There is little point to having a 'good' graphics card in dom0, because most of what it displays is x11-forwarded graphical data, so 3d acceleration is wasted on dom0.
What you can do (though it's not trivial) is attach a good GPU to a VM, then connect a monitor to that GPU, and game on it. But for now, dual-boot for gaming is probably preferable.

[Polygonbugs]

Sorry for late reply, I was just wonder about compatibilty issue with Qubes environment(something related to qubes gui component?) in dom0. I've already known that installing high-end GPU in dom0 is waste. BTW, thanks for replying.

[Polygonbugs]

Sorry for late reply, I was just wonder about compatibilty issue with Qubes environment(something related to qubes gui component?) in dom0. I've already known that installing high-end GPU in dom0 is waste. BTW, thanks for replying.