qvm-remove is not working, since saving the qvm_collection fails, because getting the dispvm_netvm fails

[ColinRothgang]

Qubes OS R3.2

Affected Template VMs:

Unknown but probably any.

---

Expected behavior:

when calling qvm-remove (optionally with --just-db, this doesn't seem to make any difference except a warning) on a VM, whose files were deleted it should remove the VM from the system.

Actual behavior:

I tried to remove a VM (later called me just call it whonix-gw (name changed))
Thew command failed with the output:

--> Removing all the files on disk...
Warning: [Errno 2] No such file or directory: '/var/lib/qubes/servicevms/whonix-gw'
Traceback (most recent call last):
  File /usr/bin/qvm-remove, line 112, in <module>
    main()
  File /usr/bin/qvm-remove, line 109, in main
    qvm_collection.save()
  File /usr/lib64/python2.7/site-packages/qubes/qubes.py, line 687, in save
    element = vm.create_xml_element()
  File /usr/lib64/python2.7/site-packages/qubes/modules/000QubesVm.py, line 2147, in create_xml_element
    attrs = self.get_xml_attrs()
  File /usr/lib64/python2.7/site-packages/qubes/modules/000QubesVm.py, line 2136, in get_xml_attrs
    value = attr_config['save']()
  File /usr/lib64/python2.7/site-packages/qubes/modules/000QubesVm.py, line 235, in <lambda>
    if self.dispvm_netvm is not None   File /usr/lib64/python2.7/site-packages/qubes/modules/000QubesVm.py, line 655, in dispvm_netvm
    return self._collection[self._dispvm_netvm]
KeyError: 26

Steps to reproduce the behavior:

At that point I just checked out the specified source code file put in the name of the dispvm_netvm hardcoded and tried again (this time successfully). When trying to delete other VMs later on (with the original source code) the same error occurred, so I changed the dispvm_netvm to sys-firewall and put in the hardcoded sys-firewall again. Again the command ran without any problems. However, both qvm-prefs -l and Qubes VM Manager showed the correct settings for the dispvm_netvm, so I don't see why it should behave like this. I build a workaround for now, but I want to report this as other people might experience the same problem.

General notes:

I am running a Lenovo Thinkpad T440p without VT-x.

This occured the first time after upgrading from Qubes OS R3.1 to Qubes R3.2. However, it also occured later with a HVM and some Dispostable VMs who were not deleted correctly.
When testing now with a slightly different version of the code I get this error:

user@dom0:~$qvm-remove temp2; sudo rm -r /var/lib/qubes/appvms/temp2/; qvm-create -t debian-8 -l red temp2 && sudo rm -r /var/lib/qubes/appvms/temp2/ && qvm-remove temp2
A VM with the name 'temp2' does not exist in the system.
rm: das Entfernen von „/var/lib/qubes/appvms/temp2/“ ist nicht möglich: Datei oder Verzeichnis nicht gefunden
--> Using TemplateVM: debian-8
--> Creating directory: /var/lib/qubes/appvms/temp2
--> Copying the template's private image: /var/lib/qubes/vm-templates/debian-8/private.img
--> Creating volatile image: /var/lib/qubes/appvms/temp2/volatile.img...
--> Creating icon symlink: /var/lib/qubes/appvms/temp2/icon.png -> /usr/share/icons/hicolor/128x128/devices/appvm-red.png
--> Creating default whitelisted apps list: /var/lib/qubes/appvms/temp2/whitelisted-appmenus.list
--> Converting Appmenu Templates...
--> Adding Apps to the Menu...
kbuildsycoca5 running...
Traceback (most recent call last):
  File /usr/bin/qvm-create, line 223, in <module>
    main()
  File /usr/bin/qvm-create, line 219, in main
    qvm_collection.save()
  File /usr/lib64/python2.7/site-packages/qubes/qubes.py, line 687, in save
    element = vm.create_xml_element()
  File /usr/lib64/python2.7/site-packages/qubes/modules/000QubesVm.py, line 2146, in create_xml_element
    attrs = self.get_xml_attrs()
  File /usr/lib64/python2.7/site-packages/qubes/modules/000QubesVm.py, line 2135, in get_xml_attrs
    value = attr_config['save']()
  File /usr/lib64/python2.7/site-packages/qubes/modules/000QubesVm.py, line 235, in <lambda>
    lambda: str(self.dispvm_netvm.qid) if self.dispvm_netvm is not None else none #sys-firewall ## workaround, helped once when qid was out of range??
AttributeError: 'str' object has no attribute 'qid'

This error also appears with the original source code (in the line with if self.dispvm_netvm is not None).

---

Related issues:

While testing on this issue I also experienced an incomplete deletion of the created VMs (although they don't appear in qvm-ls anymore). The error description is:

libvirt.libvirtError: Operation schlug fehl: Domain 'temp2' ist bereits mit UUID f9e3e41c-cdc1-453e-9cd9-81df25ddd40e definiert.

[unman]

@ColinRothgang It isn't clear to me why you are doing this. but in any case I cant reproduce your problem.
The command you cite under "General Notes" completes without problem for me, other than noting that the files have already been deleted.

What version of qubes-core do you have installed?.

[unman]

@ColinRothgang It isn't clear to me why you are doing this. but in any case I cant reproduce your problem.
The command you cite under "General Notes" completes without problem for me, other than noting that the files have already been deleted.

What version of qubes-core do you have installed?.

[ColinRothgang]

I am running Qubes R 3.2. And for some reason the error is gone now (including the named cited command), however I had some other strange problems and while working around/trying stuff out, so I might have accidentally removed the problem. But you are right, it looks like there might have been something else wrong before, but I haven't touched any of these source files before the error (and now I reverted back). The code looks different than the one on Github however (I'm having only 2155 lines of code). unman:

…

@ColinRothgang It isn't clear to me why you are doing this. but in any case I cant reproduce your problem. The command you cite under "General Notes" completes without problem for me, other than noting that the files have already been deleted. What version of qubes-core do you have installed?.

[ColinRothgang]

I am running Qubes R 3.2. And for some reason the error is gone now (including the named cited command), however I had some other strange problems and while working around/trying stuff out, so I might have accidentally removed the problem. But you are right, it looks like there might have been something else wrong before, but I haven't touched any of these source files before the error (and now I reverted back). The code looks different than the one on Github however (I'm having only 2155 lines of code). unman:

…

@ColinRothgang It isn't clear to me why you are doing this. but in any case I cant reproduce your problem. The command you cite under "General Notes" completes without problem for me, other than noting that the files have already been deleted. What version of qubes-core do you have installed?.