Document requirements for secure GPU-accelerated rendering

[DemiMarie]

Qubes OS version:

R4.0

Affected component(s):

Documentation

---

Steps to reproduce the behavior:

Search for documentation for what exactly would be necessary for secure, hardware-accelerated rendering in untrusted AppVMs

Expected behavior:

Some documentation as to why it is not possible currently, and what would be required to change that.

Actual behavior:

No such documentation.

General notes:

One obvious use of a system like Qubes is running untrusted Steam games. Because Qubes doesn’t support hardware-accelerated rendering, this doesn’t work. It would be nice to have documentation as to why this is not possible, and what would be necessary to change this.

---

Related issues:

[andrewdavidwong]

The documentation is a community effort. Please help us improve it by submitting a pull request:

https://www.qubes-os.org/doc/doc-guidelines/

[DemiMarie]

@andrewdavidwong I would be happy to, but to do so I need to know what the QubesOS maintainers would require of such a solution.

[andrewdavidwong]

@DemiMarie:

I would be happy to, but to do so I need to know what the QubesOS maintainers would require of such a solution.

I suggest searching qubes-devel to see whether the information you need is already there and, if not, starting a thread asking specific questions.

[teoman002]

A known problem why gpu acceleration in VMs is a secruity risk:
It is not possible due to VRAM Leaks of graphic cards, read this report with many pictures as a proof.
https://hsmr.cc/palinopsia/ (cited from whonix footnotes)
https://www.whonix.org/wiki/Virtualization_Platform_Security#cite_note-3

Description of the problem:
VRAM leakage enables guest operating systems to access VRAM content from Qubes dom0 area, which can be a secruity risk, if dom0 is spied on in the right time or all the time.
This can be done by allocating VRAM without initializing the content.
The reason why GPU manufacturers don't implement initialization of VRAM content might be, because it decreases either the lifespan of the memory or slows down performance, although I can't site any sources on this. It's also possible that they had no reason to do so till now.

Possible approach:
In order to change the situation one has to tell the graphic card to initialize after memory allocation. AMD has linux open source drivers, maybe a driver developer can investigate the altered behavior when such a function is implemented. But changing such a fundamental behavior could have devastating effects on the API, because everyone expect memory to behave the old way.

[sajkbflksadbfkasdjf]

A known problem why gpu acceleration in VMs is a secruity risk: It is not possible due to VRAM Leaks of graphic cards, read this report with many pictures as a proof. https://hsmr.cc/palinopsia/ (cited from whonix footnotes) https://www.whonix.org/wiki/Virtualization_Platform_Security#cite_note-3

Description of the problem: VRAM leakage enables guest operating systems to access VRAM content from Qubes dom0 area, which can be a secruity risk, if dom0 is spied on in the right time or all the time. This can be done by allocating VRAM without initializing the content. The reason why GPU manufacturers don't implement initialization of VRAM content might be, because it decreases either the lifespan of the memory or slows down performance, although I can't site any sources on this. It's also possible that they had no reason to do so till now.

Possible approach: In order to change the situation one has to tell the graphic card to initialize after memory allocation. AMD has linux open source drivers, maybe a driver developer can investigate the altered behavior when such a function is implemented. But changing such a fundamental behavior could have devastating effects on the API, because everyone expect memory to behave the old way.

Why would this be a blocking issue? Couldn't we construct an additional virtualisation layer (maybe just draw a white screen to the buffer) so that no additional information is exposed? It should definitely be doable and wasting one of the two most powerful chips in your computer because the virtualisation is too hard to get right seems wasteful to me.

[thw0rted]

If more use cases help at all: I'm a web developer writing a WebGL-based application. Some of my application's users are on a Qubes-based desktop environment. Their performance is currently terrible, and I think this issue is the reason.

Users on a physical desktop with a mid-grade dedicated GPU can get a smooth 60FPS performance with minimal load on the GPU. Even those with Intel integrated graphics from several years ago can manage a pretty stable 30FPS, as long as they're using the right drivers. Users on the Qubes platform are stuck with Chrome's "SwiftShader" pure-software renderer, which takes more than one second per frame (!).

I think this problem is only likely to become more prevalent in the future, as more applications move to the browser, and browsers become more reliant on hardware-accelerated compositing and rendering.

[DemiMarie]

If more use cases help at all: I'm a web developer writing a WebGL-based application. Some of my application's users are on a Qubes-based desktop environment. Their performance is currently terrible, and I think this issue is the reason.

First, thanks for letting us know. The current situation sucks, but it is the best we can do right now. Sadly, supporting it securely on cards that people can actually afford (as opposed to super-expensive enterprise cards) is still an open problem. The good news is that people are working on it; the bad news is that these efforts could take quite a while to come to fruition. If Qubes can ever ship hardware accelerated rendering on by default without violating our user’s security expectiations, we will.

Are you able to link to the web application, by any chance?

Users on the Qubes platform are stuck with Chrome's "SwiftShader" pure-software renderer, which takes more than one second per frame (!).

Yea, this sucks. Is your application particularly heavy on textures? Texture sampling is notoriously slow in software renderers; there is a reason GPUs have dedicated hardware for it. 3D transformations are also quite expensive. Simple pixel shaders should (at least in theory) be able to be handled pretty efficiently using SIMD instructions on the CPU, though Qubes disabling SMT (simultaneous multithreading AKA hyper-threading) probably doesn’t help there.

I also suggest filing a ticket with SwiftShader; you could be hitting a pathological case there. At they very least, they might have suggestions as to what part of the code is particularly expensive.

[thw0rted]

I don't have the Qubes-based desktop environment available right now, but I think you could get a good idea of the problem using this demo page. (Our application is proprietary, but built on CesiumJS.)

If you paste in viewer.scene.debugShowFramesPerSecond = true; just after the first line of code, then click Run (F8), it will reload the demo with a FPS counter in the corner. With an Intel HD620 (very basic iGPU, ~5-6 years old) I get reasonable framerates (15-30 FPS), but as soon as I switch to SwiftShader it's down around 1FPS.

I will look at filing a ticket with Chromium and/or Cesium, but I kind of expect each to point the finger at the other...

[marmarek]

With an Intel HD620 (very basic iGPU, ~5-6 years old) I get reasonable framerates (15-30 FPS), but as soon as I switch to SwiftShader it's down around 1FPS.

FWIW I get ~9FPS in Firefox and ~4FPS in Chrome. Both are bad, but one is clearly worse...