Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.
Sign upQubes 4.0-rc4 installation with setting "create sys-usb" first run error #3518
Comments
andrewdavidwong
added
bug
C: installer
labels
Feb 3, 2018
andrewdavidwong
added this to the Release 4.0 milestone
Feb 3, 2018
andrewdavidwong
modified the milestones:
Release 4.0,
Release 4.0 updates
Mar 31, 2018
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
pachulo
May 27, 2018
I'm facing the same issue.
Apart from using virt_mod pv, detaching the offending controller makes sys-usb work with default config.
pachulo
commented
May 27, 2018
|
I'm facing the same issue. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
marmarek
May 27, 2018
Member
@pachulo do you also have exact this message:
2018-02-02 07:57:23.651+0000: libxl: libxl_pci.c:993:qemu_pci_add_xenstore: qemu refused to add device: 0000:00:14.0,msitranslate=0,power_mgmt=0
2018-02-02 07:57:23.652+0000: libxl: libxl_pci.c:1338:libxl__add_pcidevs: libxl_device_pci_add failed: -3
?
Can you find related fragments of /var/log/xen/console/hypervisor.log (if any) and /var/log/xen/console/guest-sys-usb-dm.log from that time?
|
@pachulo do you also have exact this message: ? Can you find related fragments of |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
pachulo
Jun 6, 2018
Hey @marmarek !
Just to make sure that all the information is here: When I do:
$ sudo systemctl status qubes-vm@sys-usb
I see:
Start failed: internal error: Unable to reset PCI device 0000:00:14.0: internal error: libxenlight failed to create new domain sys-usb
Those are the messages in /var/log/libvirt/lixl/libxl-driver.log:
2018-06-06 20:23:25.327+0000: libxl: libxl_pci.c:1176:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:00:14.0
2018-06-06 20:24:25.473+0000: libxl: libxl_exec.c:226:libxl__xenstore_child_wait_deprecated: Device Model not ready
2018-06-06 20:24:25.475+0000: libxl: libxl_pci.c:993:qemu_pci_add_xenstore: qemu refused to add device: 0000:00:14.0,msitranslate=0,power_mgmt=0
2018-06-06 20:24:25.475+0000: libxl: libxl_pci.c:1338:libxl__add_pcidevs: libxl_device_pci_add failed: -3
2018-06-06 20:24:25.475+0000: libxl: libxl_create.c:1512:domcreate_attach_devices: unable to add pci devices
2018-06-06 20:24:25.834+0000: libxl: libxl_pci.c:1176:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:00:14.0
Then, in /var/log/xen/console/hypervisor.log:
(XEN) [VT-D] It's risky to assign 0000:00:1a.0 with shared RMRR at b9c0f000 for Dom1.
(XEN) pt_irq_create_bind failed (-16) for dom1
And in /var/log/xen/console/guest-sys-usb-dm.log:
+ mv /tmp/qmp/new-req_f8b19b0a-603f-4d7f-94f3-a1ed9ec85648 /tmp/qmp/req/f8b19b0a-603f-4d7f-94f3-a1ed9ec85648
+ [ -f /tmp/qmp/res/f8b19b0a-603f-4d7f-94f3-a1ed9ec85648 ]
+ sleep 0.1
f8b19b0a-603f-4d7f-94f3-a1ed9ec85648
{"execute": "device_add", "arguments": {"driver": "xen-pci-passthrough", "id": "xen-pci-pt_0000-00-14.0", "hostaddr": "0000:00:01.00", "machine_addr": "0000:00:14.0", "permissive": false}}
[00:06.0] xen_pt_realize: Assigning real physical device 00:01.0 to devfn 0x30
+ [ -f /tmp/qmp/res/f8b19b0a-603f-4d7f-94f3-a1ed9ec85648 ]
+ sleep 0.1
[00:06.0] xen_pt_register_regions: IO region 0 registered (size=0x00010000 base_addr=0xd4720000 type: 0x4)
[00:06.0] xen_pt_config_reg_init: Offset 0x0006 mismatch! Emulated=0x0010, host=0x0290, syncing to 0x0290.
[00:06.0] xen_pt_config_reg_init: Offset 0x000e mismatch! Emulated=0x0080, host=0x0000, syncing to 0x0000.
[00:06.0] xen_pt_config_reg_init: Offset 0x0010 mismatch! Emulated=0x0000, host=0xd4720004, syncing to 0xd4720004.
[00:06.0] xen_pt_config_reg_init: Offset 0x0072 mismatch! Emulated=0x0000, host=0x01c2, syncing to 0x0002.
[00:06.0] xen_pt_config_reg_init: Offset 0x0082 mismatch! Emulated=0x0000, host=0x0086, syncing to 0x0080.
[00:06.0] xen_pt_pci_intx: intx=1
[00:06.0] xen_pt_realize: Real physical device 00:01.0 registered successfully
{"error": {"class": "GenericError", "desc": "Binding of interrupt 0 failed: Device or resource busy"}}
qemu: /home/user/qubes-src/vmm-xen-stubdom-linux/build/qemu/memory.c:2634: do_address_space_destroy: Assertion `QTAILQ_EMPTY(&as->listeners)' failed.
u2mfn_release, priv= (null)
+ [ -f /tmp/qmp/res/f8b19b0a-603f-4d7f-94f3-a1ed9ec85648 ]
+ cat /tmp/qmp/res/f8b19b0a-603f-4d7f-94f3-a1ed9ec85648
+ rm /tmp/qmp/res/f8b19b0a-603f-4d7f-94f3-a1ed9ec85648
+ sed -n /"qdev_id": "xen-pci-pt_0000-00-14.0"/{s/.*"slot": \([0-9]\+\),.*"function": \([0-9]\+\),.*/\1,\2/;p}
+ sed s/"qdev_id"/\n\0/g
+ qmp_req {"execute": "query-pci"}
+ cat /proc/sys/kernel/random/uuid
+ local id=bbdf1958-98dd-41d7-b77a-aaa74cd8ce32
+ echo {"execute": "query-pci"}
+ mv /tmp/qmp/new-req_bbdf1958-98dd-41d7-b77a-aaa74cd8ce32 /tmp/qmp/req/bbdf1958-98dd-41d7-b77a-aaa74cd8ce32
+ [ -f /tmp/qmp/res/bbdf1958-98dd-41d7-b77a-aaa74cd8ce32 ]
+ sleep 0.1
+ [ -f /tmp/qmp/res/bbdf1958-98dd-41d7-b77a-aaa74cd8ce32 ]
+ sleep 0.1
bbdf1958-98dd-41d7-b77a-aaa74cd8ce32
{"execute": "query-pci"}
+ [ -f /tmp/qmp/res/bbdf1958-98dd-41d7-b77a-aaa74cd8ce32 ]
+ cat /tmp/qmp/res/bbdf1958-98dd-41d7-b77a-aaa74cd8ce32
+ rm /tmp/qmp/res/bbdf1958-98dd-41d7-b77a-aaa74cd8ce32
+ slot_func=
+ cut -d , -f 1
+ printf %s
+ slot=
+ cut -d , -f 2
+ printf %s
+ func=
/init: line 1: arithmetic syntax error
pcifront pci-0: Rescanning PCI Frontend Bus 0000:00
pcifront pci-0: Rescanning PCI Frontend Bus 0000:00
Hope it helps!
pachulo
commented
Jun 6, 2018
•
edited
Edited 3 times
-
pachulo
edited Jun 10, 2018 (most recent)
-
pachulo
edited Jun 6, 2018
-
pachulo
edited Jun 6, 2018
-
pachulo
created Jun 6, 2018
edited
-
Jun 10, 2018 (most recent) -
Jun 6, 2018 -
Jun 6, 2018 -
Jun 6, 2018
|
Hey @marmarek ! Just to make sure that all the information is here: When I do:
I see:
Those are the messages in /var/log/libvirt/lixl/libxl-driver.log: Then, in /var/log/xen/console/hypervisor.log: And in /var/log/xen/console/guest-sys-usb-dm.log: Hope it helps! |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
sudomain
Jun 20, 2018
I just want to add that I'm experiencing this on the final 4.0 release and my logs basically mirror @pachulo 's logs. A major difference for me is that I didn't create sys-usb at first boot. I installed dom0 updates then tried creating sys-usb using:
sudo qubesctl state.sls qvm.sys-usb
(per the docs).
I have 3 USB controllers and have tried giving various combinations of the 3 to sys-usb (as well as toggling no-strict-reset on some of them). For me, detaching the offending controller does not result in a working sys-usb, but thankfully switching to PV mode does.
Trying to run sys-usb without any of the controllers attached results in this error:
qrexec-daemon startup failed: connection to the vm failed
sudomain
commented
Jun 20, 2018
|
I just want to add that I'm experiencing this on the final 4.0 release and my logs basically mirror @pachulo 's logs. A major difference for me is that I didn't create sys-usb at first boot. I installed dom0 updates then tried creating sys-usb using: |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
marmarek
Jun 20, 2018
Member
|
(XEN) [VT-D] It's risky to assign 0000:00:1a.0 with shared RMRR at b9c0f000 for Dom1.
Note it's about different device.
(XEN) pt_irq_create_bind failed (-16) for dom1
Can you provide more details about this device (00:14.0)? Like
this: `sudo lspci -vvs 00:14.0` in dom0. Especially interesting part is
about interrupt and MSI capabilities.
You can also add `pci=nomsi` kernel option for that VM (qvm-prefs, see
kernelopts property).
|
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
sudomain
Jun 20, 2018
You can also add
pci=nomsikernel option for that VM (qvm-prefs, see kernelopts property).
I created a fresh sys-usb and appended the 'pci=nomsi' kernel option to the existing options using this command (in dom0):
qvm-prefs -s sys-usb kernelopts "nopat iommu=soft swiotlb=8192 pci=nomsi"
Is this the correct way to add that kernel option?
I got the same error as before (Unable to reset PCI device 0000:00:14.0)
Can you provide more details about this device (00:14.0)? Like this:
sudo lspci -vvs 00:14.0in dom0.
I realize this wasn't directed toward me since I didn't mention a BDF address in my first comment, but maybe info about my faulty PCI device (which also happens to be 00:14.0) will be helpful:
00:14.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller (rev 04) (prog-if 30 [XHCI]) Subsystem: Hewlett-Packard Company Device 179b Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Interrupt: pin A routed to IRQ 10 Region 0: Memory at d4720000 (64-bit, non-prefetchable) [disabled] [size=64K] Capabilities: [70] Power Management version 2 Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0-,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Capabilities: [80] MSI: Enable- Count=1/8 Maskable- 64bit+ Address: 0000000000000000 Data: 0000 Kernel driver in use: pciback Kernel modules: xhci_pci
I think I'm having the same issue given the similarity of our logs, but it may be different due to the slightly different behavior that I mentioned in my first comment (sys-usb not starting even with all devices removed). Should I open a separate issue given this difference in behavior?
sudomain
commented
Jun 20, 2018
•
edited
Edited 3 times
-
sudomain
edited Jun 20, 2018 (most recent)
-
sudomain
edited Jun 20, 2018
-
sudomain
edited Jun 20, 2018
-
sudomain
created Jun 20, 2018
edited
-
Jun 20, 2018 (most recent) -
Jun 20, 2018 -
Jun 20, 2018 -
Jun 20, 2018
I created a fresh sys-usb and appended the 'pci=nomsi' kernel option to the existing options using this command (in dom0):
I realize this wasn't directed toward me since I didn't mention a BDF address in my first comment, but maybe info about my faulty PCI device (which also happens to be 00:14.0) will be helpful:
I think I'm having the same issue given the similarity of our logs, but it may be different due to the slightly different behavior that I mentioned in my first comment (sys-usb not starting even with all devices removed). Should I open a separate issue given this difference in behavior? |
errin07 commentedFeb 2, 2018
Qubes OS version:
R4.0-rc4
Affected TemplateVMs:
dom0
Steps to reproduce the behavior:
Select "create sys-usb" from first boot/setup menu.
Expected behavior:
Installation finished successful.
Actual behavior:
Start failed: internal error: Unable to reset PCI device 0000:00:14.0: internal error: libxenlight failed to create new domain sys-usb
libxl-driver.log:
2018-02-02 07:57:23.651+0000: libxl: libxl_pci.c:993:qemu_pci_add_xenstore: qemu refused to add device: 0000:00:14.0,msitranslate=0,power_mgmt=0 2018-02-02 07:57:23.652+0000: libxl: libxl_pci.c:1338:libxl__add_pcidevs: libxl_device_pci_add failed: -3 2018-02-02 07:57:23.652+0000: libxl: libxl_create.c:1512:domcreate_attach_devices: unable to add pci devices 2018-02-02 07:57:23.658+0000: libxl: libxl_pci.c:1176:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:00:14.0qvm-pci:
dom0:00_14.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller sys-usb (no-strict-reset=True)General notes:
Fresh Installation R4.0-rc4
Setting 'sys-usb virt_mode pv' works.
Related issues: