Static GPU pass-through on Xen for Intel GPU

[rootkovska]

Allow to (start and) run Qubes with one of the VMs (which is not Dom0) having direct access to the GPU, protected via VT-d.

"Static" means that we don't want to support taking a GPU from a running VM and giving it to another one. Because traditionally GPU is assigned to Dom0, and Dom0 can never be stopped, this likely means the assignment is set once per boot.

On a laptop which has only one GPU and one monitor (so 99% of hw we're interested in), such a configuration would not make much sense, especially for Qubes (as opposed to Xen generally). However, together with #833, this should provide a basis to implement true GUI domain. Still, it seems like this task could be worked on independently from #833.

[crat0z]

Just a question: Please excuse my ignorance, but would implementing this have any indirect effect on GPU pass-through to HVM domains? As Marcus at Wetware Labs demonstrated last year, it's possible to get GPU pass-through working for HVM domains given that you don't use stubdoms, which is obviously a security risk. In my recent attempts to get a linux HVM to work with my secondary graphics card, it doesn't seem to initalize the GPU BIOS correctly and spits out strange errors in dmesg. So then, if it's a stubdom issue, and if GuiDom is to run in a PV domain (or PVH in the future) with stubdom, I would imagine that there would be similar issues that HVM domains face with stubdoms. Then again, I don't know much about Xen architecture, let alone stubdoms...

[crat0z]

Just a question: Please excuse my ignorance, but would implementing this have any indirect effect on GPU pass-through to HVM domains? As Marcus at Wetware Labs demonstrated last year, it's possible to get GPU pass-through working for HVM domains given that you don't use stubdoms, which is obviously a security risk. In my recent attempts to get a linux HVM to work with my secondary graphics card, it doesn't seem to initalize the GPU BIOS correctly and spits out strange errors in dmesg. So then, if it's a stubdom issue, and if GuiDom is to run in a PV domain (or PVH in the future) with stubdom, I would imagine that there would be similar issues that HVM domains face with stubdoms. Then again, I don't know much about Xen architecture, let alone stubdoms...

[marmarek]

given that you don't use stubdoms, which is obviously a security risk.

This is unacceptable.

[marmarek]

given that you don't use stubdoms, which is obviously a security risk.

This is unacceptable.

[marmarek]

So, yes, main work here is about getting it working with stubdomain. Update stubdomain to qemu-2.7 and use Linux-based stubdomain in Qubes 4.0 will probably help here.

[marmarek]

So, yes, main work here is about getting it working with stubdomain. Update stubdomain to qemu-2.7 and use Linux-based stubdomain in Qubes 4.0 will probably help here.

[ideologysec]

given that you don't use stubdoms, which is obviously a security risk.

This is unacceptable.

Not using stubdoms, or using stubdoms?

On a slightly tangential note, it would be exceedingly handy to be able to pass, say, a dedicated GPU through to a Windows VM for gaming, or to a Linux VM for GPGPU computing or general HW acceleration. I know this was something that might have worked once but does not appear to any longer; I wonder if since the GUI domain and the rest of Qubes 4 are changing so much, if this is something that could reappear. (I'd submit a PR but I don't actually know how to code...yet.)

[ideologysec]

given that you don't use stubdoms, which is obviously a security risk.

This is unacceptable.

Not using stubdoms, or using stubdoms?

On a slightly tangential note, it would be exceedingly handy to be able to pass, say, a dedicated GPU through to a Windows VM for gaming, or to a Linux VM for GPGPU computing or general HW acceleration. I know this was something that might have worked once but does not appear to any longer; I wonder if since the GUI domain and the rest of Qubes 4 are changing so much, if this is something that could reappear. (I'd submit a PR but I don't actually know how to code...yet.)

[rootkovska]

For many GPUs (e.g. those having their own flash for firmware, such as most/all(?) Nvidias, ATI(s)?, etc), shuffling the GPU between untrusted VMs and trusted ones would be a security risk. This is because your "gaming" VM might exploit some bug in the GPU (or even "functionality", not just a bug), and e.g. update its firmware to a bad/buggy one. Then, when you pass it to a trusted VM (GUI domain) the GPU might strike back and compromise it.

[rootkovska]

For many GPUs (e.g. those having their own flash for firmware, such as most/all(?) Nvidias, ATI(s)?, etc), shuffling the GPU between untrusted VMs and trusted ones would be a security risk. This is because your "gaming" VM might exploit some bug in the GPU (or even "functionality", not just a bug), and e.g. update its firmware to a bad/buggy one. Then, when you pass it to a trusted VM (GUI domain) the GPU might strike back and compromise it.

[ideologysec]

Forget gaming, even GPGPU acceleration in Kali for ocl-hashcat would be handy.

I would not imagine reassigning a GPU that was untrusted to a trusted VM or using it for a GUI domain, though I imagine that a mechanism for preventing reassignment would be difficult at best. And impractical given that VMs can be set up and torn down with startling ease, or even linked from a base Template. Most laptops that have a GPU that would be worthwhile to pass through have two graphics cards, an integrated Intel one and a more powerful Nvidia or AMD one.

...so, it would be exceedingly helpful from a productivity standpoint to pass the dedicated card through to a VM, and use the integrated one for the GUI domain, since the other alternatives are either "spend a great deal of money on an airgapped machine," which would also involve carrying two machines and a great deal of information transfer hassle, or "reboot to Windows and compromise everything." Perhaps even just a warning or a permanent degradation of the overall system security status that are under discussion (#6, #2134)

[ideologysec]

Forget gaming, even GPGPU acceleration in Kali for ocl-hashcat would be handy.

I would not imagine reassigning a GPU that was untrusted to a trusted VM or using it for a GUI domain, though I imagine that a mechanism for preventing reassignment would be difficult at best. And impractical given that VMs can be set up and torn down with startling ease, or even linked from a base Template. Most laptops that have a GPU that would be worthwhile to pass through have two graphics cards, an integrated Intel one and a more powerful Nvidia or AMD one.

...so, it would be exceedingly helpful from a productivity standpoint to pass the dedicated card through to a VM, and use the integrated one for the GUI domain, since the other alternatives are either "spend a great deal of money on an airgapped machine," which would also involve carrying two machines and a great deal of information transfer hassle, or "reboot to Windows and compromise everything." Perhaps even just a warning or a permanent degradation of the overall system security status that are under discussion (#6, #2134)

[crat0z]

@Aktariel Marek was referring to not using stubdomains, i.e. stubdomains are necessary.

What would be the security implications of always using a certain GPU assigned to a VM, and never using it for GUI domain purposes? For example, if I had a second GPU in my desktop computer, and only used it in untrusted domains. I'm not sure how rd.qubes.hide_all_usb works, but maybe pciback could be used to isolate the GPU in a similar way, albeit with the additional complexity that GPUs bring.

Correct me if I'm wrong, but this sounds very similar to what the GUI domain would be capable of, but instead in the context of another domain. My suspicion is that when the GUI domain comes to fruition, it wouldn't be much more difficult to have GPU passthrough to other VMs. However, I acknowledge that with GPU passthrough the user would have to be much more careful to not compromise themselves.

[crat0z]

@Aktariel Marek was referring to not using stubdomains, i.e. stubdomains are necessary.

What would be the security implications of always using a certain GPU assigned to a VM, and never using it for GUI domain purposes? For example, if I had a second GPU in my desktop computer, and only used it in untrusted domains. I'm not sure how rd.qubes.hide_all_usb works, but maybe pciback could be used to isolate the GPU in a similar way, albeit with the additional complexity that GPUs bring.

Correct me if I'm wrong, but this sounds very similar to what the GUI domain would be capable of, but instead in the context of another domain. My suspicion is that when the GUI domain comes to fruition, it wouldn't be much more difficult to have GPU passthrough to other VMs. However, I acknowledge that with GPU passthrough the user would have to be much more careful to not compromise themselves.