libvirt.libvirtError: internal error: Unable to reset PCI device 0000:05:00.2: internal error: Active 0000:05:00.0 devices on bus with 0000:05:00.2, not doing bus reset

[adrelanos]

Reported by troubadour in Whonix forum.

With Qubes-R3.0-x86_64-DVD.iso.

The output of qubes-dom0-update:

Using sys-firewall as UpdateVM to download updates for Dom0; this may take some time...
Running command on VM: 'sys-firewall'...
Starting the VM 'sys-firewall'...
--> Starting NetVM sys-net...
--> Creating volatile image: /var/lib/qubes/servicevms/sys-net/volatile.img...
--> Loading the VM (type = NetVM)...
Traceback (most recent call last):
  File "/bin/qvm-run", line 199, in <module>
    main()
  File "/bin/qvm-run", line 194, in main
    r = vm_run_cmd(vm, cmdstr, options)
  File "/bin/qvm-run", line 64, in vm_run_cmd
    gui = options.gui, filter_esc = options.filter_esc)
  File "/usr/lib64/python2.7/site-packages/qubes/modules/000QubesVm.py", line 1529, in run
    self.start(verbose=verbose, start_guid = gui, notify_function=notify_function)
  File "/usr/lib64/python2.7/site-packages/qubes/modules/006QubesProxyVm.py", line 82, in start
    retcode = super(QubesProxyVm, self).start(**kwargs)
  File "/usr/lib64/python2.7/site-packages/qubes/modules/005QubesNetVm.py", line 121, in start
    xid=super(QubesNetVm, self).start(**kwargs)
  File "/usr/lib64/python2.7/site-packages/qubes/modules/000QubesVm.py", line 1733, in start
    self.netvm.start(verbose = verbose, start_guid = start_guid, notify_function = notify_function)
  File "/usr/lib64/python2.7/site-packages/qubes/modules/005QubesNetVm.py", line 121, in start
    xid=super(QubesNetVm, self).start(**kwargs)
  File "/usr/lib64/python2.7/site-packages/qubes/modules/000QubesVm.py", line 1773, in start
    self.libvirt_domain.createWithFlags(libvirt.VIR_DOMAIN_START_PAUSED)
  File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1037, in createWithFlags
    if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
libvirt.libvirtError: internal error: Unable to reset PCI device 0000:05:00.2: internal error: Active 0000:05:00.0 devices on bus with 0000:05:00.2, not doing bus reset

This error shows also in a narrow window after installing the service vm's at first boot.

May be anecdotal, may be not, during installation the mouse is not responding (it does work with Qubes R2).

[adrelanos]

@troubadoour's

Unable to reset PCI device 0000:05:00.2: internal error: Active 0000:05:00.0 devices on bus with 0000:05:00.2, not doing bus reset

From https://groups.google.com/forum/#!searchin/qubes-users/Unable$20to$20reset$20PCI$20device/qubes-users/o8eahbAg3q0/v1Ztl8aU-UkJ

Unable to reset PCI device 0000:04:00.2: internal error: Active 0000:04:00.0 devices on bus with 0000:04:00.2 not doing bus reset

So this is likely the same issue.

[adrelanos]

@troubadoour's

Unable to reset PCI device 0000:05:00.2: internal error: Active 0000:05:00.0 devices on bus with 0000:05:00.2, not doing bus reset

From https://groups.google.com/forum/#!searchin/qubes-users/Unable$20to$20reset$20PCI$20device/qubes-users/o8eahbAg3q0/v1Ztl8aU-UkJ

Unable to reset PCI device 0000:04:00.2: internal error: Active 0000:04:00.0 devices on bus with 0000:04:00.2 not doing bus reset

So this is likely the same issue.

[adrelanos]

Does sys-net run already? You'll see a green indicator under state in Qubes VM Manager. Alternatively you can run in dom0 qvm-ls to see that. I guess sys-net is not yet running?

If not... If you are running in dom0 qvm-start sys-net, it should show the error you posted above?

In that case, go to Qubes VM Manager -> right click on sys-net -> VM Settings -> Devices. Assign both 0000:04:00.2 and 0000:04:00.0. See if that works.

It would be interesting if you could post your dom0 lspci output.

Alternative attempt. In dom0.

sudo su
echo -n "1" > /sys/bus/pci/devices/0000\:05\:00.0/remove 

Pure theory from reading https://groups.google.com/forum/#!searchin/qubes-users/Unable$20to$20reset$20PCI$20device/qubes-users/o8eahbAg3q0/v1Ztl8aU-UkJ.

@marmarek do you think this can be somehow fixed and made work out of the box?

[adrelanos]

Does sys-net run already? You'll see a green indicator under state in Qubes VM Manager. Alternatively you can run in dom0 qvm-ls to see that. I guess sys-net is not yet running?

If not... If you are running in dom0 qvm-start sys-net, it should show the error you posted above?

In that case, go to Qubes VM Manager -> right click on sys-net -> VM Settings -> Devices. Assign both 0000:04:00.2 and 0000:04:00.0. See if that works.

It would be interesting if you could post your dom0 lspci output.

Alternative attempt. In dom0.

sudo su
echo -n "1" > /sys/bus/pci/devices/0000\:05\:00.0/remove 

Pure theory from reading https://groups.google.com/forum/#!searchin/qubes-users/Unable$20to$20reset$20PCI$20device/qubes-users/o8eahbAg3q0/v1Ztl8aU-UkJ.

@marmarek do you think this can be somehow fixed and made work out of the box?

[marmarek]

If we identify what devices are those, we may add a code to always
assign both of them. But doing that blindly isn't a good idea - the
other device (since we don't know otherwise) can be anything, including
something we don't want in sys-net.

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[marmarek]

If we identify what devices are those, we may add a code to always
assign both of them. But doing that blindly isn't a good idea - the
other device (since we don't know otherwise) can be anything, including
something we don't want in sys-net.

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[troubadoour]

sys-net does not run.

Alternative attempt. In dom0.

sudo su
echo -n "1" > /sys/bus/pci/devices/0000:05:00.0/remove

That works, sys-net can then be started. I updated dom0.

I'll use the trick for the time being, pending to investigation.

[troubadoour]

sys-net does not run.

Alternative attempt. In dom0.

sudo su
echo -n "1" > /sys/bus/pci/devices/0000:05:00.0/remove

That works, sys-net can then be started. I updated dom0.

I'll use the trick for the time being, pending to investigation.

[adrelanos]

Good.

Curious, what kind of device is 0000:05:00.0? Can you check please in dom0 lspci?

[adrelanos]

Good.

Curious, what kind of device is 0000:05:00.0? Can you check please in dom0 lspci?

[adrelanos]

If you like to automate this workaround, have it applied on boot... Theoretically, the following should work... dom0...

Edit /etc/rc.local with root rights:

#!/bin/bash
echo -n "1" > /sys/bus/pci/devices/0000:05:00.0/remove 

sudo chmod +x /etc/rc.local
sudo systemctl enable rc-local.service

[adrelanos]

If you like to automate this workaround, have it applied on boot... Theoretically, the following should work... dom0...

Edit /etc/rc.local with root rights:

#!/bin/bash
echo -n "1" > /sys/bus/pci/devices/0000:05:00.0/remove 

sudo chmod +x /etc/rc.local
sudo systemctl enable rc-local.service

[marmarek]

/etc/rc.local would be too late. Take a look here for a better way (that one about permissive mode, but the same can be used for other pre-netvm tasks).

[marmarek]

/etc/rc.local would be too late. Take a look here for a better way (that one about permissive mode, but the same can be used for other pre-netvm tasks).

[troubadoour]

Device 0000:05:00.0 is an unassigned Realtek Express Card Reader (there is no card reader on the laptop). Tried several things, including different rules in /etc/udev/rules.d, to no avail.

As a temporary workaround, the device is removed and I'm on a wired connection to finish setting up Qubes (some funny behavior otherwise, will summarize when I find the time).

Will try PCI passthrough issues when everything is up and running properly.

[troubadoour]

Device 0000:05:00.0 is an unassigned Realtek Express Card Reader (there is no card reader on the laptop). Tried several things, including different rules in /etc/udev/rules.d, to no avail.

As a temporary workaround, the device is removed and I'm on a wired connection to finish setting up Qubes (some funny behavior otherwise, will summarize when I find the time).

Will try PCI passthrough issues when everything is up and running properly.

[marmarek]

Hmm, all the cases of this problem I've seen was about 'Realtek Express Card Reader'. Maybe we should add this for User FAQ at least?

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[marmarek]

Hmm, all the cases of this problem I've seen was about 'Realtek Express Card Reader'. Maybe we should add this for User FAQ at least?

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[troubadoour]

The "PCI passthrough issues" solution does not seem to work,
systemctl status qubes-pre-netvm.service reports

Nov 10 17:59:06 dom0 systemd[1]: Starting Netvm fixup...                                                       
Nov 10 17:59:06 dom0 sh[1092]: /bin/sh: line 0: echo: write error: No such device                              
Nov 10 17:59:06 dom0 systemd[1]: qubes-pre-netvm.service: main process exited, code=...URE                     

Running the command echo 0000:05:00.0 > /sys/bus/pci/drivers/pciback/permissive directly or editing the "permissive" file return the same error No such device. (I'm able to edit it if 0000:05:00.0 is added to sys-net devices, but in this configuration, qrexec won't start...)

A workaround is to change the ExecStart line in qubes-pre-netvm.service to the command working from the terminal.
ExecStart=/bin/sh -c 'echo -n "1" > /sys/bus/pci/devices/0000:05:00.0/remove'

And that's where the funny behavior mentioned earlier starts. If the ethernet cable is not connected, the network is gone (popup "Disconnected. The network connection has been disconnected"). It might be an unrelated issue. It has shown up right after I installed whonix-gw template (would have to reinstall Qubes to confirm).

I'll let aside this problem for a while. For the moment, I can work with the wire.

[troubadoour]

The "PCI passthrough issues" solution does not seem to work,
systemctl status qubes-pre-netvm.service reports

Nov 10 17:59:06 dom0 systemd[1]: Starting Netvm fixup...                                                       
Nov 10 17:59:06 dom0 sh[1092]: /bin/sh: line 0: echo: write error: No such device                              
Nov 10 17:59:06 dom0 systemd[1]: qubes-pre-netvm.service: main process exited, code=...URE                     

Running the command echo 0000:05:00.0 > /sys/bus/pci/drivers/pciback/permissive directly or editing the "permissive" file return the same error No such device. (I'm able to edit it if 0000:05:00.0 is added to sys-net devices, but in this configuration, qrexec won't start...)

A workaround is to change the ExecStart line in qubes-pre-netvm.service to the command working from the terminal.
ExecStart=/bin/sh -c 'echo -n "1" > /sys/bus/pci/devices/0000:05:00.0/remove'

And that's where the funny behavior mentioned earlier starts. If the ethernet cable is not connected, the network is gone (popup "Disconnected. The network connection has been disconnected"). It might be an unrelated issue. It has shown up right after I installed whonix-gw template (would have to reinstall Qubes to confirm).

I'll let aside this problem for a while. For the moment, I can work with the wire.

[waiting-for-dev]

Just in case it could be useful I will tell my experience with a very similar issue.

First, upon installing qubes I got a pop-up with following error.

Setting up neworing failure!

['/usr/sbin/service', 'qubes-netv', 'start'] failed: Redirecting to /bin/systemctl start qubes-netvm.service job for qubes-netvm.service failed. See 'systemctl status qubes-netvm.service' and 'journalctl -xn' for details

Then, trying qvm-start sys-net from console:

libvirt.libvirtError: internal error: Unable to reset PCI device 0000:03:00.1: internal error: Active 0000:03:00.0 devices on bus with 0000:03:00.1, not doing bus reset

• 0000:03:00.1 is Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 12)
• 0000:03:00.0 is Audio device: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor HD Audio Controller (rev 06)

1 was already added to sys-net but not 0 (what seems correct, because an audio device shouldn't have nothing to do). Anyway, as stated in this issue I tried to add it to sys-net. Then, it launched successfully but, suddenly, a pop-up appeared noticing me that network connection had been lost.

Then, I tried:

echo -n "1" > /sys/bus/pci/devices/0000:03:00.0/remove`

and then, eureka, sys-net worked. As also stated in this issue this was not persistent, so I created a file /etc/systemd/system/qubes-pre-netvm.service with:

[Unit]
Description=Netvm fixup
Before=qubes-netvm.service

[Service]
ExecStart=/bin/sh -c 'echo -n "1" > /sys/bus/pci/devices/0000:03:00.0/remove`'
Type=oneshot
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

I ran systemctl enable qubes-pre-netvm.service and now it persists between reboots.

Now, it seems everything is ok. At least for now, it has been my only problem installing Qubes.

By the way, audio is working anyway. I have another audio entry in lspci. Not sure about their difference...

Thanks for your hard work!!!

[waiting-for-dev]

Just in case it could be useful I will tell my experience with a very similar issue.

First, upon installing qubes I got a pop-up with following error.

Setting up neworing failure!

['/usr/sbin/service', 'qubes-netv', 'start'] failed: Redirecting to /bin/systemctl start qubes-netvm.service job for qubes-netvm.service failed. See 'systemctl status qubes-netvm.service' and 'journalctl -xn' for details

Then, trying qvm-start sys-net from console:

libvirt.libvirtError: internal error: Unable to reset PCI device 0000:03:00.1: internal error: Active 0000:03:00.0 devices on bus with 0000:03:00.1, not doing bus reset

• 0000:03:00.1 is Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 12)
• 0000:03:00.0 is Audio device: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor HD Audio Controller (rev 06)

1 was already added to sys-net but not 0 (what seems correct, because an audio device shouldn't have nothing to do). Anyway, as stated in this issue I tried to add it to sys-net. Then, it launched successfully but, suddenly, a pop-up appeared noticing me that network connection had been lost.

Then, I tried:

echo -n "1" > /sys/bus/pci/devices/0000:03:00.0/remove`

and then, eureka, sys-net worked. As also stated in this issue this was not persistent, so I created a file /etc/systemd/system/qubes-pre-netvm.service with:

[Unit]
Description=Netvm fixup
Before=qubes-netvm.service

[Service]
ExecStart=/bin/sh -c 'echo -n "1" > /sys/bus/pci/devices/0000:03:00.0/remove`'
Type=oneshot
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

I ran systemctl enable qubes-pre-netvm.service and now it persists between reboots.

Now, it seems everything is ok. At least for now, it has been my only problem installing Qubes.

By the way, audio is working anyway. I have another audio entry in lspci. Not sure about their difference...

Thanks for your hard work!!!

[ampanasiuk]

Issue still with 4.0 rc3; workaround still fixes the problem.

[ampanasiuk]

Issue still with 4.0 rc3; workaround still fixes the problem.

[fosslinux]

@ampanasiuk which one? There are a few in this thread.

[fosslinux]

@ampanasiuk which one? There are a few in this thread.

[ampanasiuk]

The one with the workaround of echo -n "1" > /sys/bus/pci/devices/0000:05:00.0/remove .

…

@ampanasiuk which one? There are a few in this thread. -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #1393 (comment)

[ampanasiuk]

The one with the workaround of echo -n "1" > /sys/bus/pci/devices/0000:05:00.0/remove .

…

@ampanasiuk which one? There are a few in this thread. -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #1393 (comment)

[e6lk7dqzm83p]

Hi,

I also got an error similar to this (slightly different) when setting up Qubes

Setting up networking failure!

['/usr/sbin/service', 'qubes-netv', 'start'] failed: Redirecting to /bin/systemctl start qubes-netvm.service job for qubes-netvm.service failed. See 'systemctl status qubes-netvm.service' and 'journalctl -xn' for details

When I run:

echo -n "1" > /sys/bus/pci/devices/0000:3a:00.0/remove .

I get a permission denied error, even when using sudo. I have no network connectivity so I'm unable to copy and paste all the details.

Any help would be appreciated, thanks!

[e6lk7dqzm83p]

Hi,

I also got an error similar to this (slightly different) when setting up Qubes

Setting up networking failure!

['/usr/sbin/service', 'qubes-netv', 'start'] failed: Redirecting to /bin/systemctl start qubes-netvm.service job for qubes-netvm.service failed. See 'systemctl status qubes-netvm.service' and 'journalctl -xn' for details

When I run:

echo -n "1" > /sys/bus/pci/devices/0000:3a:00.0/remove .

I get a permission denied error, even when using sudo. I have no network connectivity so I'm unable to copy and paste all the details.

Any help would be appreciated, thanks!

[ampanasiuk]

Are you doing sudo echo -n "1" > /sys/bus/pci/devices/0000:3a:00.0/remove ? The redirection is being done outside of sudo. Do "sudo su" first, then the echo.

…

Hi, I also got an error similar to this (slightly different) when setting up Qubes ``` Setting up networking failure! ['/usr/sbin/service', 'qubes-netv', 'start'] failed: Redirecting to /bin/systemctl start qubes-netvm.service job for qubes-netvm.service failed. See 'systemctl status qubes-netvm.service' and 'journalctl -xn' for details ``` When I run: echo -n "1" > /sys/bus/pci/devices/0000:3a:00.0/remove . I get a permission denied error, even when using sudo. I have no network connectivity so I'm unable to copy and paste all the details. Any help would be appreciated, thanks! -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #1393 (comment)

[ampanasiuk]

Are you doing sudo echo -n "1" > /sys/bus/pci/devices/0000:3a:00.0/remove ? The redirection is being done outside of sudo. Do "sudo su" first, then the echo.

…

Hi, I also got an error similar to this (slightly different) when setting up Qubes ``` Setting up networking failure! ['/usr/sbin/service', 'qubes-netv', 'start'] failed: Redirecting to /bin/systemctl start qubes-netvm.service job for qubes-netvm.service failed. See 'systemctl status qubes-netvm.service' and 'journalctl -xn' for details ``` When I run: echo -n "1" > /sys/bus/pci/devices/0000:3a:00.0/remove . I get a permission denied error, even when using sudo. I have no network connectivity so I'm unable to copy and paste all the details. Any help would be appreciated, thanks! -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #1393 (comment)

[e6lk7dqzm83p]

Hi,

Thanks, that worked, however the method mentioned @waiting-for-dev for persistence doesn't seem to work for me.

On top of this Qubes is incredibly slow (even with a brand new quad core with 32 GB of RAM); I'm not sure if that's relevant or not.

[e6lk7dqzm83p]

Hi,

Thanks, that worked, however the method mentioned @waiting-for-dev for persistence doesn't seem to work for me.

On top of this Qubes is incredibly slow (even with a brand new quad core with 32 GB of RAM); I'm not sure if that's relevant or not.

[e6lk7dqzm83p]

Does anyone have a persistent solution to this? Do I just throw a script somewhere that executes that command on startup (sorry, I'm trying to get Qubes working after moving from Ubuntu)?

[e6lk7dqzm83p]

Does anyone have a persistent solution to this? Do I just throw a script somewhere that executes that command on startup (sorry, I'm trying to get Qubes working after moving from Ubuntu)?

[e6lk7dqzm83p]

I realized there was a typo above, using this script worked:

[Unit]
Description=Netvm fixup
Before=qubes-netvm.service

[Service]
ExecStart=/bin/sh -c 'echo -n "1" > /sys/bus/pci/devices/0000:03:00.0/remove' #Note the missing accent mark
Type=oneshot
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

[e6lk7dqzm83p]

I realized there was a typo above, using this script worked:

[Unit]
Description=Netvm fixup
Before=qubes-netvm.service

[Service]
ExecStart=/bin/sh -c 'echo -n "1" > /sys/bus/pci/devices/0000:03:00.0/remove' #Note the missing accent mark
Type=oneshot
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target