No virtualization is available in a HVM qube

[SurinameClubcard]

Qubes OS version:

R4.0

Affected component(s):

Depending on the outcome of a risk and threat assessment, it might be allowable to enable nested virtualization in HVM mode. E.g., I'd like to run GNS3 in a HVM based Linux distribution. Without proper support for nested virtualization, that won't work / perform.

---

Steps to reproduce the behavior:

• Install your favorite Linux distribution in a HVM-based qube.
• Install VirtualBox in that HVM.
• VirtualBox will complain that VT-x/AMD-V hardware acceleration is missing.

Expected behavior:

Hardware acceleration should be availabe in HVM.

Actual behavior:

Currently I'm unable to run GNS3 in a Linux based HVM with acceptable performance.

General notes:

I will not participate in a flame war on nested virtualization. ;-)

I'm aware of risks introduced by nested virtualization. In my situation, I've done the assessment and I accept these risks.

---

Related issues:

I'm more than happy to patch Qubes myself to make this possible. E.g., there is https://groups.google.com/forum/#!msg/qubes-devel/UzO0BsIfIow/wWjpd3IPAgAJ which already contains a patch. Is it really that simple to enable it? Or did things change massively between then and now.

There is also https://wiki.xenproject.org/wiki/Nested_Virtualization_in_Xen#Current_status, which I don't understand, because it looks to me that newer versions of Xen won't work but older versions do?

Maybe I greatly underestimate the complexity of this matter ...

[marmarek]

There is also https://wiki.xenproject.org/wiki/Nested_Virtualization_in_Xen#Current_status, which I don't understand, because it looks to me that newer versions of Xen won't work but older versions do?

It is exactly as you read it - nested virtualization in Xen is in "preview" state for a long time, and it doesn't seems to be really maintained.

This is also the reason why we don't provide easy way to enable it - we choose to spend time on feature that actually improve security and stability, not do the opposite.

If you still want to enable, see /usr/share/qubes/templates/libvirt/xen.xml and https://dev.qubes-os.org/projects/core-admin/en/latest/libvirt.html

[marmarek]

There is also https://wiki.xenproject.org/wiki/Nested_Virtualization_in_Xen#Current_status, which I don't understand, because it looks to me that newer versions of Xen won't work but older versions do?

It is exactly as you read it - nested virtualization in Xen is in "preview" state for a long time, and it doesn't seems to be really maintained.

This is also the reason why we don't provide easy way to enable it - we choose to spend time on feature that actually improve security and stability, not do the opposite.

If you still want to enable, see /usr/share/qubes/templates/libvirt/xen.xml and https://dev.qubes-os.org/projects/core-admin/en/latest/libvirt.html

[andrewdavidwong]

Closing as "won't do." If anyone has a new reason for why this should be done, please leave a comment, and we'll be happy to take another look. Thank you.

[andrewdavidwong]

Closing as "won't do." If anyone has a new reason for why this should be done, please leave a comment, and we'll be happy to take another look. Thank you.

[SurinameClubcard]

I understand the "won't do" closure and I fully appreciate the fact that you're spending time on features that improve security and stability. To me, Qubes is about stability and security, but also about freedom and flexibility. That said, nested virtualization does serve a purpose. The possibility to run ESXi in ESXi is awesome for some use cases, especially for teaching purposes. Nesting Qubes in Qubes also seems so logical, for example, to test-drive a new version. Using Vagrant for getting OWASP WebGoat up-and-running springs to my mind. My original GNS3 example. Docker. I'm more-or-less seeing Qubes as an operating system for operation systems. If Qubes would be able to support nested virtualization, it would be a more complete product. And I currently don't see any threats that will make the outcome of my risk assessment a negative one. But YMMV ofc.

If I understand marmarek's reply correctly, there is no need to patch? The functionality is already in Xen and the only thing to do is enabling it to the tools? I'd like to give it try anyway... ;-)

[SurinameClubcard]

I understand the "won't do" closure and I fully appreciate the fact that you're spending time on features that improve security and stability. To me, Qubes is about stability and security, but also about freedom and flexibility. That said, nested virtualization does serve a purpose. The possibility to run ESXi in ESXi is awesome for some use cases, especially for teaching purposes. Nesting Qubes in Qubes also seems so logical, for example, to test-drive a new version. Using Vagrant for getting OWASP WebGoat up-and-running springs to my mind. My original GNS3 example. Docker. I'm more-or-less seeing Qubes as an operating system for operation systems. If Qubes would be able to support nested virtualization, it would be a more complete product. And I currently don't see any threats that will make the outcome of my risk assessment a negative one. But YMMV ofc.

If I understand marmarek's reply correctly, there is no need to patch? The functionality is already in Xen and the only thing to do is enabling it to the tools? I'd like to give it try anyway... ;-)