qvm-ls in 4.0-rc1 throw QubesDaemonNoResponseError if no anyvm rule for admin.vm.List

[pietrushnic]

Qubes OS version:

R4.0

Affected TemplateVMs:

debian-8 and fedora-25

---

Steps to reproduce the behavior:

Follow introduction

1. Modify files:

/etc/qubes-rpc/policy/admin.vm.Create.AppVM:test-mgmt $adminvm allow,target=$adminvm
/etc/qubes-rpc/policy/include/admin-local-rwx:test-mgmt $tag:created-by-test-mgmt allow,target=$adminvm
/etc/qubes-rpc/policy/admin.label.List:test-mgmt $adminvm allow,target=$adminvm
/etc/qubes-rpc/policy/admin.vm.List:test-mgmt $adminvm allow,target=$adminvm

2. qvm-create --label green --template debian-8 test-mgmt - repro also with fedora-25
3. qvm-run test-mgmt gnome-terminal
4. In test-mgmt:

sudo apt update
sudo apt upgrade
sudo apt install qubes-core-admin-client

5. Run qvm-ls

user@test-mgmt:~$ qvm-ls
Traceback (most recent call last):
  File "/usr/bin/qvm-ls", line 9, in <module>
    load_entry_point('qubesadmin==4.0.4', 'console_scripts', 'qvm-ls')()
  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 356, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 2476, in load_entry_point
    return ep.load()
  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 2190, in load
    ['__name__'])
  File "/usr/lib/python3/dist-packages/qubesadmin/tools/__init__.py", line 34, in <module>
    import qubesadmin.log
  File "/usr/lib/python3/dist-packages/qubesadmin/log.py", line 30, in <module>
    import dbus
ImportError: No module named 'dbus'
user@test-mgmt:~$ qvm-ls
Traceback (most recent call last):
  File "/usr/bin/qvm-ls", line 9, in <module>
    load_entry_point('qubesadmin==4.0.4', 'console_scripts', 'qvm-ls')()
  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 356, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 2476, in load_entry_point
    return ep.load()
  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 2190, in load
    ['__name__'])
  File "/usr/lib/python3/dist-packages/qubesadmin/tools/__init__.py", line 34, in <module>
    import qubesadmin.log
  File "/usr/lib/python3/dist-packages/qubesadmin/log.py", line 30, in <module>
    import dbus
ImportError: No module named 'dbus'

6. Install python3-dbus and run again

user@test-mgmt:~$ qvm-ls
please wait... |Traceback (most recent call last):
  File "/usr/bin/qvm-ls", line 9, in <module>
    load_entry_point('qubesadmin==4.0.4', 'console_scripts', 'qvm-ls')()
  File "/usr/lib/python3/dist-packages/qubesadmin/tools/qvm_ls.py", line 585, in main
    table.write_table(sys.stdout)
  File "/usr/lib/python3/dist-packages/qubesadmin/tools/qvm_ls.py", line 423, in write_table
    table_data.append(self.get_row(vm))
  File "/usr/lib/python3/dist-packages/qubesadmin/tools/qvm_ls.py", line 407, in get_row
    ret.append(col.cell(vm))
  File "/usr/lib/python3/dist-packages/qubesadmin/tools/qvm_ls.py", line 85, in cell
    value = self.format(vm) or '-'
  File "/usr/lib/python3/dist-packages/qubesadmin/tools/qvm_ls.py", line 106, in format
    ret = self._attr(vm)
  File "/usr/lib/python3/dist-packages/qubesadmin/tools/qvm_ls.py", line 337, in <lambda>
    attr=(lambda vm: vm.get_power_state()),
  File "/usr/lib/python3/dist-packages/qubesadmin/vm/__init__.py", line 196, in get_power_state
    ).decode('ascii').split('\n')
  File "/usr/lib/python3/dist-packages/qubesadmin/base.py", line 69, in qubesd_call
    payload_stream)
  File "/usr/lib/python3/dist-packages/qubesadmin/app.py", line 560, in qubesd_call
    'Service call error: %s', stderr.decode())
qubesadmin.exc.QubesDaemonNoResponseError: Service call error: Request refused

Expected behavior:

qvm-ls should not crash and list qubes that test-mgmt created.

Actual behavior:

qvm-ls crashing as above.

General notes:

test-mon example from introduction works fine.

After investigation it looks like fault is in lack of:

test-mgmt $anyvm allow,target=$adminvm

Adding lead to test-mgmt working.

BTW I see more problems in mentioned article, do you think issues here are best method to report this problems ?

---

Related issues:

[marmarek]

Thanks for the report. Yes, this is appropriate place for such things. If you have list of shorter items, it may be worth creating a single issue with them.

As for the fix, see linked pull request, especially marmarek/qubes-core-admin-client@2f7d1ca
There are two issues here:

1. Missing policy line in the article.
2. Even without it, qvm-ls shouldn't crash, especially with such message. It should just list NA as power state of all the VMs in that case.

The pull request fixes the second thing.

[marmarek]

Thanks for the report. Yes, this is appropriate place for such things. If you have list of shorter items, it may be worth creating a single issue with them.

As for the fix, see linked pull request, especially marmarek/qubes-core-admin-client@2f7d1ca
There are two issues here:

1. Missing policy line in the article.
2. Even without it, qvm-ls shouldn't crash, especially with such message. It should just list NA as power state of all the VMs in that case.

The pull request fixes the second thing.

[pietrushnic]

@marmarek I would be glad to test that, what is the easiest way except direct file replacement ?

[pietrushnic]

@marmarek I would be glad to test that, what is the easiest way except direct file replacement ?

[pietrushnic]

@marmarek ok I did that through sudo python3 setup.py install and can confirm that it fix mentioned problem.

[pietrushnic]

@marmarek ok I did that through sudo python3 setup.py install and can confirm that it fix mentioned problem.

[pietrushnic]

@marmarek I assume that this rules:

/etc/qubes-rpc/policy/admin.vm.Create.AppVM:test-mgmt $adminvm allow,target=$adminvm
/etc/qubes-rpc/policy/include/admin-local-rwx:test-mgmt $tag:created-by-test-mgmt allow,target=$adminvm
/etc/qubes-rpc/policy/admin.label.List:test-mgmt $adminvm allow,target=$adminvm
/etc/qubes-rpc/policy/admin.vm.List:test-mgmt $adminvm allow,target=$adminvm

Should give me list only VM create by my test-mgmt, but I get all vms:

[user@test-mgmt qubes-core-admin-client]$ qvm-ls
NAME              STATE    CLASS       LABEL  TEMPLATE   NETVM
debian-9          NA       TemplateVM  -      -          -
dev               NA       TemplateVM  -      -          -
dom0              Running  AdminVM     -      -          -
fedora-25         NA       TemplateVM  -      -          -
fedora-25-dvm     NA       AppVM       -      -          -
managed-research  NA       AppVM       red    fedora-25  sys-firewall
managed-vpn       NA       AppVM       green  fedora-25  sys-firewall
managed-work      NA       AppVM       green  fedora-25  sys-firewall
personal          NA       AppVM       -      -          -
sys-firewall      NA       AppVM       -      -          -
sys-net           NA       AppVM       -      -          -
test-mgmt         NA       AppVM       -      -          -
test-mon          NA       AppVM       -      -          -
untrusted         NA       AppVM       -      -          -
vault             NA       AppVM       -      -          -
work              NA       AppVM       -      -          -

[pietrushnic]

@marmarek I assume that this rules:

/etc/qubes-rpc/policy/admin.vm.Create.AppVM:test-mgmt $adminvm allow,target=$adminvm
/etc/qubes-rpc/policy/include/admin-local-rwx:test-mgmt $tag:created-by-test-mgmt allow,target=$adminvm
/etc/qubes-rpc/policy/admin.label.List:test-mgmt $adminvm allow,target=$adminvm
/etc/qubes-rpc/policy/admin.vm.List:test-mgmt $adminvm allow,target=$adminvm

Should give me list only VM create by my test-mgmt, but I get all vms:

[user@test-mgmt qubes-core-admin-client]$ qvm-ls
NAME              STATE    CLASS       LABEL  TEMPLATE   NETVM
debian-9          NA       TemplateVM  -      -          -
dev               NA       TemplateVM  -      -          -
dom0              Running  AdminVM     -      -          -
fedora-25         NA       TemplateVM  -      -          -
fedora-25-dvm     NA       AppVM       -      -          -
managed-research  NA       AppVM       red    fedora-25  sys-firewall
managed-vpn       NA       AppVM       green  fedora-25  sys-firewall
managed-work      NA       AppVM       green  fedora-25  sys-firewall
personal          NA       AppVM       -      -          -
sys-firewall      NA       AppVM       -      -          -
sys-net           NA       AppVM       -      -          -
test-mgmt         NA       AppVM       -      -          -
test-mon          NA       AppVM       -      -          -
untrusted         NA       AppVM       -      -          -
vault             NA       AppVM       -      -          -
work              NA       AppVM       -      -          -

[marmarek]

This is how it works currently, as explained in the article:

Before we test things, we need to allow two more R-calls, which are needed due to a temporary limitation of the current implementation of the qvm-* tools, which always attempt to acquire the list of all the VMs in the system. So, we need to either grant access to all the global R-calls (note that we added the rule above to admin-local-rwx, so this time we would also have to add the rule to admin-global-ro), or we need to be more precise by selectively allowing only admin.vm.List and admin.label.List calls to $adminvm:

test-mgmt $adminvm allow,target=$adminvm

In upcoming releases (beyond 4.0-rc1) we plan to remove this limitation, allowing for the possibility of management VMs that cannot get a complete list of all the VMs in the system.

As you can see, you get only a list all VMs, but detailed info is limited to those managed VMs.
If you want to see also state of those managed VMs (running/halted etc), add this rule to admin.vm.List:

test-mgmt $tag:created-by-test-mgmt allow,target=$adminvm

[marmarek]

This is how it works currently, as explained in the article:

Before we test things, we need to allow two more R-calls, which are needed due to a temporary limitation of the current implementation of the qvm-* tools, which always attempt to acquire the list of all the VMs in the system. So, we need to either grant access to all the global R-calls (note that we added the rule above to admin-local-rwx, so this time we would also have to add the rule to admin-global-ro), or we need to be more precise by selectively allowing only admin.vm.List and admin.label.List calls to $adminvm:

test-mgmt $adminvm allow,target=$adminvm

In upcoming releases (beyond 4.0-rc1) we plan to remove this limitation, allowing for the possibility of management VMs that cannot get a complete list of all the VMs in the system.

As you can see, you get only a list all VMs, but detailed info is limited to those managed VMs.
If you want to see also state of those managed VMs (running/halted etc), add this rule to admin.vm.List:

test-mgmt $tag:created-by-test-mgmt allow,target=$adminvm

[pietrushnic]

@marmarek ok now got it. Thanks.

[pietrushnic]

@marmarek ok now got it. Thanks.

[qubesos-bot]

Automated announcement from builder-github

The package python2-qubesadmin-4.0.9-0.1.fc25 has been pushed to the r4.0 testing repository for dom0.
To test this update, please install it with the following command:

sudo qubes-dom0-update --enablerepo=qubes-dom0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-qubesadmin-4.0.9-0.1.fc25 has been pushed to the r4.0 testing repository for dom0.
To test this update, please install it with the following command:

sudo qubes-dom0-update --enablerepo=qubes-dom0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-admin-client_4.0.9-1+deb8u1 has been pushed to the r4.0 testing repository for the Debian jessie template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing jessie-testing, then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-admin-client_4.0.9-1+deb8u1 has been pushed to the r4.0 testing repository for the Debian jessie template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing jessie-testing, then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-admin-client_4.0.9-1+deb9u1 has been pushed to the r4.0 testing repository for the Debian stretch template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing stretch-testing, then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-admin-client_4.0.9-1+deb9u1 has been pushed to the r4.0 testing repository for the Debian stretch template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing stretch-testing, then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-qubesadmin-4.0.9-0.1.fc24 has been pushed to the r4.0 testing repository for the Fedora fc24 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r4.0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-qubesadmin-4.0.9-0.1.fc24 has been pushed to the r4.0 testing repository for the Fedora fc24 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r4.0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-qubesadmin-4.0.9-0.1.fc25 has been pushed to the r4.0 testing repository for the Fedora fc25 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r4.0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-qubesadmin-4.0.9-0.1.fc25 has been pushed to the r4.0 testing repository for the Fedora fc25 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r4.0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-qubesadmin-4.0.9-0.1.fc26 has been pushed to the r4.0 testing repository for the Fedora fc26 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r4.0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-qubesadmin-4.0.9-0.1.fc26 has been pushed to the r4.0 testing repository for the Fedora fc26 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r4.0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-admin-client_4.0.9-1+deb8u1 has been pushed to the r4.0 stable repository for the Debian jessie template.
To install this update, please use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-admin-client_4.0.9-1+deb8u1 has been pushed to the r4.0 stable repository for the Debian jessie template.
To install this update, please use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-admin-client_4.0.9-1+deb9u1 has been pushed to the r4.0 stable repository for the Debian stretch template.
To install this update, please use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-admin-client_4.0.9-1+deb9u1 has been pushed to the r4.0 stable repository for the Debian stretch template.
To install this update, please use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-qubesadmin-4.0.9-0.1.fc24 has been pushed to the r4.0 stable repository for the Fedora fc24 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-qubesadmin-4.0.9-0.1.fc24 has been pushed to the r4.0 stable repository for the Fedora fc24 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-qubesadmin-4.0.9-0.1.fc25 has been pushed to the r4.0 stable repository for the Fedora fc25 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-qubesadmin-4.0.9-0.1.fc25 has been pushed to the r4.0 stable repository for the Fedora fc25 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-qubesadmin-4.0.9-0.1.fc25 has been pushed to the r4.0 stable repository for dom0.
To install this update, please use the standard update command:

sudo qubes-dom0-update

Or update dom0 via Qubes Manager.

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-qubesadmin-4.0.9-0.1.fc25 has been pushed to the r4.0 stable repository for dom0.
To install this update, please use the standard update command:

sudo qubes-dom0-update

Or update dom0 via Qubes Manager.

Changes included in this update