PCI permissive mode broken in Qubes R4.0

[mossy-nw]

Qubes OS version:

R4.0 (tested in rc3)

Affected TemplateVMs:

---

Steps to reproduce the behavior:

Follow advice here to get e.g. network card working (successful approach in R3.2)
https://www.qubes-os.org/doc/assigning-devices/#pci-passthrough-issues

Expected behavior:

wireless adapter recognized in sys-net after setting PCI permissive mode

Actual behavior:

wireless adapter not recognized despite PCI permissive mode

General notes:

See qubes-users thread for bug details, example logs, and developer discussion @marmarek et al

https://groups.google.com/forum/#!topic/qubes-users/Klf2q3tEbOI

---

Related issues:

[mossy-nw]

Some (hopefully most relevant) excerpts from that thread so far

According to logs provided by mossy-nw permissive mode is correctly
enabled in xen-pciback in dom0 for this device. The question here is what
else is needed for HVM (using qemu in stubdomain).

My dom0 log shows that too, but the guest-dm/debug log showed the PCI
device always getting added with permissive=false. Maybe that's what you
are saying, qemu isn't passing the value along?

Yes that's the problem. The guide sets permissive mode in pciback and
not via libxl. So the stubdomain never learns about it. Will take a look
today.

If that's the case, it looks we need another option for PCI device, like
this:

qvm-pci attach sys-net dom0:xx_yy.zz -o permissive=True

Yes, exactly. And if I didn't miss something we first need to teach
libvirt about the permissive option.

[mossy-nw]

Some (hopefully most relevant) excerpts from that thread so far

According to logs provided by mossy-nw permissive mode is correctly
enabled in xen-pciback in dom0 for this device. The question here is what
else is needed for HVM (using qemu in stubdomain).

My dom0 log shows that too, but the guest-dm/debug log showed the PCI
device always getting added with permissive=false. Maybe that's what you
are saying, qemu isn't passing the value along?

Yes that's the problem. The guide sets permissive mode in pciback and
not via libxl. So the stubdomain never learns about it. Will take a look
today.

If that's the case, it looks we need another option for PCI device, like
this:

qvm-pci attach sys-net dom0:xx_yy.zz -o permissive=True

Yes, exactly. And if I didn't miss something we first need to teach
libvirt about the permissive option.

[mossy-nw]

finally an outline of qvm-start control flow:

1. qvm-start sends a request to qubesd, using Admin API
2. qubesd starts required netvm (recursively), if needed
3. qubesd request qmemman to allocate needed memory for new VM
   (according to VM's 'memory' property)
4. qubesd calls into appropriate storage pool driver to prepare for VM
   startup (create copy-on-write layers etc)
5. qubesd gathers needed VM
   properties etc and builds libvirt VM configuration (XML format, can be
   seen using virsh dumpxml)
6. qubesd calls into libvirt to start the VM
   (but in paused mode)
7. libvirt setup the VM using libxl, this include starting stubdomain if
   needed
8. qubesd start auxiliary processes, including:

• qrexec-daemon
• qubesdb-daemon (and fill its content)

9. libvirt unpause the VM
10. qvm-start-gui process (running separately from qubesd, as part of
    dom0 user GUI session) starts gui daemon

most of the above is in "start" method:
https://dev.qubes-os.org/projects/core-admin/en/latest/qubes-vm/qubesvm.html#qubes.vm.qubesvm.QubesVM.start
(there is "source" link at the right side)

[mossy-nw]

finally an outline of qvm-start control flow:

1. qvm-start sends a request to qubesd, using Admin API
2. qubesd starts required netvm (recursively), if needed
3. qubesd request qmemman to allocate needed memory for new VM
   (according to VM's 'memory' property)
4. qubesd calls into appropriate storage pool driver to prepare for VM
   startup (create copy-on-write layers etc)
5. qubesd gathers needed VM
   properties etc and builds libvirt VM configuration (XML format, can be
   seen using virsh dumpxml)
6. qubesd calls into libvirt to start the VM
   (but in paused mode)
7. libvirt setup the VM using libxl, this include starting stubdomain if
   needed
8. qubesd start auxiliary processes, including:

• qrexec-daemon
• qubesdb-daemon (and fill its content)

9. libvirt unpause the VM
10. qvm-start-gui process (running separately from qubesd, as part of
    dom0 user GUI session) starts gui daemon

most of the above is in "start" method:
https://dev.qubes-os.org/projects/core-admin/en/latest/qubes-vm/qubesvm.html#qubes.vm.qubesvm.QubesVM.start
(there is "source" link at the right side)

[HW42]

Ah, I missed that you created a ticket. Fix: QubesOS/qubes-core-admin#184

[HW42]

Ah, I missed that you created a ticket. Fix: QubesOS/qubes-core-admin#184

[qubesos-bot]

Automated announcement from builder-github

The package libvirt-3.3.0-6.fc25 has been pushed to the r4.0 testing repository for dom0.
To test this update, please install it with the following command:

sudo qubes-dom0-update --enablerepo=qubes-dom0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package libvirt-3.3.0-6.fc25 has been pushed to the r4.0 testing repository for dom0.
To test this update, please install it with the following command:

sudo qubes-dom0-update --enablerepo=qubes-dom0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-dom0-4.0.19-1.fc25 has been pushed to the r4.0 testing repository for dom0.
To test this update, please install it with the following command:

sudo qubes-dom0-update --enablerepo=qubes-dom0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-dom0-4.0.19-1.fc25 has been pushed to the r4.0 testing repository for dom0.
To test this update, please install it with the following command:

sudo qubes-dom0-update --enablerepo=qubes-dom0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package libvirt-3.3.0-7.fc25 has been pushed to the r4.0 stable repository for dom0.
To install this update, please use the standard update command:

sudo qubes-dom0-update

Or update dom0 via Qubes Manager.

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package libvirt-3.3.0-7.fc25 has been pushed to the r4.0 stable repository for dom0.
To install this update, please use the standard update command:

sudo qubes-dom0-update

Or update dom0 via Qubes Manager.

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-dom0-4.0.21-1.fc25 has been pushed to the r4.0 stable repository for dom0.
To install this update, please use the standard update command:

sudo qubes-dom0-update

Or update dom0 via Qubes Manager.

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-dom0-4.0.21-1.fc25 has been pushed to the r4.0 stable repository for dom0.
To install this update, please use the standard update command:

sudo qubes-dom0-update

Or update dom0 via Qubes Manager.

Changes included in this update

[mossy-nw]

following this was (also?) necessary for passthrough to work in R4.0_rc4 --
https://github.com/awokd/qubes-doc/blob/patch-15/configuration/assigning-devices.md/#pci-passthrough-issues
specifically for my PCI wifi adapter:

For example, if 00_1a.0 is the BDF of the device you want to assign to the "personal" domain, and it is particularly difficult to pass through you would do this:
qvm-pci attach --persistent --option permissive=true dom0:00_1a.0

not sure if/when this improvement to the docs will be merged? thanks!

[mossy-nw]

following this was (also?) necessary for passthrough to work in R4.0_rc4 --
https://github.com/awokd/qubes-doc/blob/patch-15/configuration/assigning-devices.md/#pci-passthrough-issues
specifically for my PCI wifi adapter:

For example, if 00_1a.0 is the BDF of the device you want to assign to the "personal" domain, and it is particularly difficult to pass through you would do this:
qvm-pci attach --persistent --option permissive=true dom0:00_1a.0

not sure if/when this improvement to the docs will be merged? thanks!

[andrewdavidwong]

@mossy-nw:

not sure if/when this improvement to the docs will be merged? thanks!

If there's an open PR with that content, it'll get merged once it passes review:

https://www.qubes-os.org/doc/doc-guidelines/

[andrewdavidwong]

@mossy-nw:

not sure if/when this improvement to the docs will be merged? thanks!

If there's an open PR with that content, it'll get merged once it passes review:

https://www.qubes-os.org/doc/doc-guidelines/