qvm-shutdown --timeout exits with libvirtError

[tasket]

Qubes OS version:

R4.0rc5

Affected component(s):

dom0 tools

---

Steps to reproduce the behavior:

qvm-shutdown --all --timeout 20

Expected behavior:

All VMs are shutdown or killed without errors.

Actual behavior:

One or more VMs are left running and the command exits with an error:

Traceback (most recent call last):
  File "/usr/lib/python3.5/site-packages/qubesadmin/tools/qvm_shutdown.py", line 87, in main
    args.timeout))
  File "/usr/lib64/python3.5/asyncio/base_events.py", line 467, in run_until_complete
    return future.result()
  File "/usr/lib64/python3.5/asyncio/futures.py", line 294, in result
    raise self._exception
  File "/usr/lib64/python3.5/asyncio/tasks.py", line 240, in _step
    result = coro.send(None)
  File "/usr/lib64/python3.5/asyncio/tasks.py", line 410, in wait_for
    raise futures.TimeoutError()
concurrent.futures._base.TimeoutError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/qvm-shutdown", line 5, in <module>
    sys.exit(main())
  File "/usr/lib/python3.5/site-packages/qubesadmin/tools/qvm_shutdown.py", line 91, in main
    vm.kill()
  File "/usr/lib/python3.5/site-packages/qubesadmin/vm/__init__.py", line 120, in kill
    self.qubesd_call(self._method_dest, 'admin.vm.Kill')
  File "/usr/lib/python3.5/site-packages/qubesadmin/base.py", line 68, in qubesd_call
    payload_stream)
  File "/usr/lib/python3.5/site-packages/qubesadmin/app.py", line 483, in qubesd_call
    return self._parse_qubesd_response(return_data)
  File "/usr/lib/python3.5/site-packages/qubesadmin/base.py", line 81, in _parse_qubesd_response
    'Got empty response from qubesd. See journalctl in dom0 for '
qubesadmin.exc.QubesDaemonNoResponseError: Got empty response from qubesd. See journalctl in dom0 for details.

Here is the log output:

Mar 28 06:44:57 dom0 qubesd[15972]: unhandled exception while calling src=b'dom0' meth=b'admin.vm.Kill' dest=b'sys-vpn' arg=b'' len(untrusted_payload)=0
Mar 28 06:44:57 dom0 qubesd[15972]: Traceback (most recent call last):
Mar 28 06:44:57 dom0 qubesd[15972]:   File "/usr/lib/python3.5/site-packages/qubes/api/__init__.py", line 262, in respond
Mar 28 06:44:57 dom0 qubesd[15972]:     untrusted_payload=untrusted_payload)
Mar 28 06:44:57 dom0 qubesd[15972]:   File "/usr/lib64/python3.5/asyncio/futures.py", line 381, in __iter__
Mar 28 06:44:57 dom0 qubesd[15972]:     yield self  # This tells Task to wait for completion.
Mar 28 06:44:57 dom0 qubesd[15972]:   File "/usr/lib64/python3.5/asyncio/tasks.py", line 310, in _wakeup
Mar 28 06:44:57 dom0 qubesd[15972]:     future.result()
Mar 28 06:44:57 dom0 qubesd[15972]:   File "/usr/lib64/python3.5/asyncio/futures.py", line 294, in result
Mar 28 06:44:57 dom0 qubesd[15972]:     raise self._exception
Mar 28 06:44:57 dom0 qubesd[15972]:   File "/usr/lib64/python3.5/asyncio/tasks.py", line 240, in _step
Mar 28 06:44:57 dom0 qubesd[15972]:     result = coro.send(None)
Mar 28 06:44:57 dom0 qubesd[15972]:   File "/usr/lib/python3.5/site-packages/qubes/api/admin.py", line 810, in vm_kill
Mar 28 06:44:57 dom0 qubesd[15972]:     yield from self.dest.kill()
Mar 28 06:44:57 dom0 qubesd[15972]:   File "/usr/lib64/python3.5/asyncio/coroutines.py", line 210, in coro
Mar 28 06:44:57 dom0 qubesd[15972]:     res = func(*args, **kw)
Mar 28 06:44:57 dom0 qubesd[15972]:   File "/usr/lib/python3.5/site-packages/qubes/vm/qubesvm.py", line 1027, in kill
Mar 28 06:44:57 dom0 qubesd[15972]:     self.libvirt_domain.destroy()
Mar 28 06:44:57 dom0 qubesd[15972]:   File "/usr/lib/python3.5/site-packages/qubes/app.py", line 94, in wrapper
Mar 28 06:44:57 dom0 qubesd[15972]:     return attr(*args, **kwargs)
Mar 28 06:44:57 dom0 qubesd[15972]:   File "/usr/lib64/python3.5/site-packages/libvirt.py", line 1136, in destroy
Mar 28 06:44:57 dom0 qubesd[15972]:     if ret == -1: raise libvirtError ('virDomainDestroy() failed', dom=self)
Mar 28 06:44:57 dom0 qubesd[15972]: libvirt.libvirtError: Requested operation is not valid: Domain is not running

General notes:

I'd suggest that qvm_shutdown trap its qvm_kill calls and maybe set a flag noting when an exception occurs. Then a follow-up loop can check VM status and exit with an exception only if VMs remain and could not be killed.

---

Related issues:

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-dom0-4.0.26-1.fc25 has been pushed to the r4.0 testing repository for dom0.
To test this update, please install it with the following command:

sudo qubes-dom0-update --enablerepo=qubes-dom0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-dom0-4.0.26-1.fc25 has been pushed to the r4.0 testing repository for dom0.
To test this update, please install it with the following command:

sudo qubes-dom0-update --enablerepo=qubes-dom0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-dom0-4.0.27-1.fc25 has been pushed to the r4.0 stable repository for dom0.
To install this update, please use the standard update command:

sudo qubes-dom0-update

Or update dom0 via Qubes Manager.

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-dom0-4.0.27-1.fc25 has been pushed to the r4.0 stable repository for dom0.
To install this update, please use the standard update command:

sudo qubes-dom0-update

Or update dom0 via Qubes Manager.

Changes included in this update