Cant install R4.0-rc1 on R3.2 HVM

[kumavis]

Qubes OS version (e.g., R3.2):

R4.0-r1 on R3.2

Expected behavior:

able to install R4.0-r1 on R3.2 HVM

Actual behavior:

fails during installation

Steps to reproduce the behavior:

• create hvm
• attach R4.0-r1 ISO as cdrom

General notes:

not sure if this is actually supported
thought it would be an easy way to play around with R4.0-r1 while still on R3.2

• Fails to start Show Plymouth Boot Screen
  • tries to run /usr/sbin/plymouthd but does not exist
  • /bin/plymouth does exist
• Failed to start Switch Root
  • Failed to switch root: Specified switch root path /sysroot does not seem to be an OS tree.
  • /sysroot only contains boot dir
• rdsoreport: command not found

[andrewdavidwong]

I'm pretty sure this is not supported. @marmarek?

[andrewdavidwong]

I'm pretty sure this is not supported. @marmarek?

[kumavis]

reasonable if it is not supported.
Though I would like to know what is preventing it from working, if it just some needs some debugging / or some environment detection is in order I would attempt a PR.
tho I'm no systems wizard-- specialty is far and away in browser and blockchain. but learning more and more through my time with qubes 😸

[kumavis]

reasonable if it is not supported.
Though I would like to know what is preventing it from working, if it just some needs some debugging / or some environment detection is in order I would attempt a PR.
tho I'm no systems wizard-- specialty is far and away in browser and blockchain. but learning more and more through my time with qubes 😸

[kumavis]

is this where the R4.0-rc1 installer ISO is built? https://github.com/QubesOS/qubes-installer-qubes-os

[kumavis]

is this where the R4.0-rc1 installer ISO is built? https://github.com/QubesOS/qubes-installer-qubes-os

[kumavis]

checking the system requirements https://www.qubes-os.org/doc/system-requirements/

Qubes Release 4.x
Minimum
64-bit Intel or AMD processor (x86_64 aka x64 aka AMD64)
Intel VT-x with EPT or AMD-V with RVI
Intel VT-d or AMD-Vi (aka AMD IOMMU)
4 GB RAM
32 GB disk space

seems that in R4.x VT-x and VT-d is mandatory

according to https://www.howtogeek.com/howto/linux/linux-tip-how-to-tell-if-your-processor-supports-vt/
you can check for VT-d support with cat /proc/cpuinfo | grep svm

If that is correct, it seems that the R3.2 vanilla HVM does not support this.
Note: I also got a negative result for cat /proc/cpuinfo | grep svm in R3.2's Dom0 (?)
Is Dom0 virtualized? If so how do I access the actual hardware /proc/cpuinfo

[kumavis]

checking the system requirements https://www.qubes-os.org/doc/system-requirements/

Qubes Release 4.x
Minimum
64-bit Intel or AMD processor (x86_64 aka x64 aka AMD64)
Intel VT-x with EPT or AMD-V with RVI
Intel VT-d or AMD-Vi (aka AMD IOMMU)
4 GB RAM
32 GB disk space

seems that in R4.x VT-x and VT-d is mandatory

according to https://www.howtogeek.com/howto/linux/linux-tip-how-to-tell-if-your-processor-supports-vt/
you can check for VT-d support with cat /proc/cpuinfo | grep svm

If that is correct, it seems that the R3.2 vanilla HVM does not support this.
Note: I also got a negative result for cat /proc/cpuinfo | grep svm in R3.2's Dom0 (?)
Is Dom0 virtualized? If so how do I access the actual hardware /proc/cpuinfo

[Jeeppler]

@kumavis

The How-To-Geek article you referenced mentions for Intel x86 processors this command:

cat /proc/cpuinfo | grep vmx

and for AMD based processors:

cat /proc/cpuinfo | grep svm

Another post on Unix Stackexchanges explains most - if not all - flags found in cat /proc/cpuinfo.

In addition, the Linux Kernel cpufeatures.h header file for version 4.9 specifies:

/* Intel-defined CPU features, CPUID level 0x00000001 (ecx), word 4 */
...
#define X86_FEATURE_VMX		( 4*32+ 5) /* Hardware virtualization */

and

/* AMD SVM Feature Identification, CPUID level 0x8000000a (edx), word 15 */
...
#define X86_FEATURE_SVM		( 6*32+ 2) /* Secure virtual machine */

[Jeeppler]

@kumavis

The How-To-Geek article you referenced mentions for Intel x86 processors this command:

cat /proc/cpuinfo | grep vmx

and for AMD based processors:

cat /proc/cpuinfo | grep svm

Another post on Unix Stackexchanges explains most - if not all - flags found in cat /proc/cpuinfo.

In addition, the Linux Kernel cpufeatures.h header file for version 4.9 specifies:

/* Intel-defined CPU features, CPUID level 0x00000001 (ecx), word 4 */
...
#define X86_FEATURE_VMX		( 4*32+ 5) /* Hardware virtualization */

and

/* AMD SVM Feature Identification, CPUID level 0x8000000a (edx), word 15 */
...
#define X86_FEATURE_SVM		( 6*32+ 2) /* Secure virtual machine */

[kumavis]

Thank you @Jeeppler
My cpu is a i7 6700K which supports VT-d and VT-x but I dont see it under dom0's cat /proc/cpuinfo
I must need to enable it in BIOS, will look into that

[kumavis]

Thank you @Jeeppler
My cpu is a i7 6700K which supports VT-d and VT-x but I dont see it under dom0's cat /proc/cpuinfo
I must need to enable it in BIOS, will look into that

[kumavis]

Next task is to see if its possible to create an HVM with VT-d and VT-x support

been digging through qvm-create and related scripts trying to see how everything connects together. Followed it down to libvirt.py calls, now reading more about libvirt and exploring with virsh

its a little confusing to find the actual qubes repos on github as the search feature indexes against master, but oh well -- not a blocker

[kumavis]

Next task is to see if its possible to create an HVM with VT-d and VT-x support

been digging through qvm-create and related scripts trying to see how everything connects together. Followed it down to libvirt.py calls, now reading more about libvirt and exploring with virsh

its a little confusing to find the actual qubes repos on github as the search feature indexes against master, but oh well -- not a blocker

[kumavis]

some good hints here https://kashyapc.wordpress.com/2012/01/14/nested-virtualization-with-kvm-intel/

[kumavis]

some good hints here https://kashyapc.wordpress.com/2012/01/14/nested-virtualization-with-kvm-intel/

[Jeeppler]

@kumavis

Yes, you most likely have to enable VT-d support in the BIOS. At least I had to.

The Qubes OS architecture is roughly the following:

---------------------------------
| Qubes OS      |  Qube A...N   |
|     Dom 0     |      DomU     |
---------------------------------
|              Libvirt          |
---------------------------------
|                 Xen           |
---------------------------------

Qubes OS refers to the tools and the Dom0 VM, which controls the windows.

If I understand it correct you want to run Qubes OS 4.0-RC1 on top of Qubes OS 3.2. This is not recommended and I assume not well tested. However, if you want to do it any ways have a look at the Qubes OS/libvirt config files for the HVM you want to run Qubes OS 4.0-RC1 in.

Furthermore, have a look at: https://wiki.xenproject.org/wiki/Nested_Virtualization_in_Xen

[Jeeppler]

@kumavis

Yes, you most likely have to enable VT-d support in the BIOS. At least I had to.

The Qubes OS architecture is roughly the following:

---------------------------------
| Qubes OS      |  Qube A...N   |
|     Dom 0     |      DomU     |
---------------------------------
|              Libvirt          |
---------------------------------
|                 Xen           |
---------------------------------

Qubes OS refers to the tools and the Dom0 VM, which controls the windows.

If I understand it correct you want to run Qubes OS 4.0-RC1 on top of Qubes OS 3.2. This is not recommended and I assume not well tested. However, if you want to do it any ways have a look at the Qubes OS/libvirt config files for the HVM you want to run Qubes OS 4.0-RC1 in.

Furthermore, have a look at: https://wiki.xenproject.org/wiki/Nested_Virtualization_in_Xen

[kumavis]

updated BIOS and enabled "Intel Virtualization Technology" and "VT-d" options but still not seeing vmx in dom0's /proc/cpuinfo 😕

[kumavis]

updated BIOS and enabled "Intel Virtualization Technology" and "VT-d" options but still not seeing vmx in dom0's /proc/cpuinfo 😕

[kumavis]

is dom0's /proc/cpuinfo accurate for the physical hardware settings? or do i need to ask xen for the actual cpuinfo?

[kumavis]

is dom0's /proc/cpuinfo accurate for the physical hardware settings? or do i need to ask xen for the actual cpuinfo?

[Jeeppler]

@kumavis do you have a working GNU/Linux live CD/USB around? If yes, boot into the live CD/USB and then use cat /proc/cpuinfo.

I actually, don't have a vmx flag set in ´Dom0or anyVM, instead I see is ahypervisorflag. Which I don't see if I just usecat /proc/cpuinfo` in a non-virtualized environment.

[Jeeppler]

@kumavis do you have a working GNU/Linux live CD/USB around? If yes, boot into the live CD/USB and then use cat /proc/cpuinfo.

I actually, don't have a vmx flag set in ´Dom0or anyVM, instead I see is ahypervisorflag. Which I don't see if I just usecat /proc/cpuinfo` in a non-virtualized environment.

[marmarek]

Dom0 will not see VT-x or VT-d, Xen (running below dom0) will. See xl info to verify (virt_caps field - hvm is about VT-x and hvm_directio is about VT-d). So, the content of /proc/cpuinfo is expected.

As for running R4.0 in HVM on R3.2, that would require virtualizing VT-x and VT-d (aka nested virtualization), which we don't do for security purposes (very complex code). And it isn't very stable, at least: https://wiki.xenproject.org/wiki/Nested_Virtualization_in_Xen.

In short - this configuration isn't supported.

[marmarek]

Dom0 will not see VT-x or VT-d, Xen (running below dom0) will. See xl info to verify (virt_caps field - hvm is about VT-x and hvm_directio is about VT-d). So, the content of /proc/cpuinfo is expected.

As for running R4.0 in HVM on R3.2, that would require virtualizing VT-x and VT-d (aka nested virtualization), which we don't do for security purposes (very complex code). And it isn't very stable, at least: https://wiki.xenproject.org/wiki/Nested_Virtualization_in_Xen.

In short - this configuration isn't supported.