Changing a named dispVM's template's template causes qubesd to crash

[3hhh]

Qubes OS version:

4.0rc4

Affected TemplateVMs:

all?
tested during debian-8 --> debian-9 migration

Steps to reproduce the behavior:

1. Use e.g. a named disposable sys-net VM with a template VM such as sys-net-dvm. Let's say sys-net-dvm uses a debian-8 template.

2. Whilst sys-net is running, switch the template of sys-net-dvm to e.g. debian-9 (yes, that works).

3. Reboot.

Expected behavior:

qubesd works or I cannot change the template's template whilst an instance is running.

Actual behavior:

qubesd crashes.

The root cause is an incorrect reference of the sys-net dvm instance to the old template in the qubes.xml file. It can be fixed by manually editing the file and rebooting.

[rustybird]

Looks like @3hhh described only the mildest case of this bug - the qubesd crash in that sequence is disruptive, but at least it fails closed. IMO the worst (and more frequent) case is that auto_cleanup=False DispVMs can be started with a snapshot of the wrong root volume, which is a potential privacy issue. /cc @andrewdavidwong

Starting a DispVM with wrong root volume:

$ qvm-clone -q fedora-26 fedora-test  # this is not specific to Fedora ...
$ qvm-clone -q debian-9  debian-test  # ... or Debian, they're just easy to tell apart
$ qvm-create -t fedora-test --prop template_for_dispvms=True -l red bug-dvm
$ qvm-create -t bug-dvm     -C DispVM                        -l red bug
$ qvm-run -p bug 'head -n 1 /etc/os-release'
NAME=Fedora
$ until qvm-shutdown --wait bug; do sleep 1; done
$ qvm-prefs -s bug-dvm template debian-test  # "journalctl -fu qubesd" prints "socket.send() raised exception." 11 times
$ echo $?
0
$ qvm-run -p bug 'head -n 1 /etc/os-release'  # should be Debian now:
NAME=Fedora

Then crashing qubesd:

$ until qvm-shutdown --wait bug; do sleep 1; done
$ qvm-remove -f fedora-test
$ qvm-start bug  # error message from volume's verify() - e.g. file-reflink:
Missing image file '/var/lib/qubes/vm-templates/fedora-test/root.img' for volume appvms/bug/root
$ systemctl restart qubesd
Job for qubesd.service failed because the control process exited with error code.

(using newest current-testing dom0 packages)

[rustybird]

Looks like @3hhh described only the mildest case of this bug - the qubesd crash in that sequence is disruptive, but at least it fails closed. IMO the worst (and more frequent) case is that auto_cleanup=False DispVMs can be started with a snapshot of the wrong root volume, which is a potential privacy issue. /cc @andrewdavidwong

Starting a DispVM with wrong root volume:

$ qvm-clone -q fedora-26 fedora-test  # this is not specific to Fedora ...
$ qvm-clone -q debian-9  debian-test  # ... or Debian, they're just easy to tell apart
$ qvm-create -t fedora-test --prop template_for_dispvms=True -l red bug-dvm
$ qvm-create -t bug-dvm     -C DispVM                        -l red bug
$ qvm-run -p bug 'head -n 1 /etc/os-release'
NAME=Fedora
$ until qvm-shutdown --wait bug; do sleep 1; done
$ qvm-prefs -s bug-dvm template debian-test  # "journalctl -fu qubesd" prints "socket.send() raised exception." 11 times
$ echo $?
0
$ qvm-run -p bug 'head -n 1 /etc/os-release'  # should be Debian now:
NAME=Fedora

Then crashing qubesd:

$ until qvm-shutdown --wait bug; do sleep 1; done
$ qvm-remove -f fedora-test
$ qvm-start bug  # error message from volume's verify() - e.g. file-reflink:
Missing image file '/var/lib/qubes/vm-templates/fedora-test/root.img' for volume appvms/bug/root
$ systemctl restart qubesd
Job for qubesd.service failed because the control process exited with error code.

(using newest current-testing dom0 packages)

[marmarek]

For now, as a quick fix, I'm going to forbid changing template of AppVM, when it have some DispVMs (running or not). Later a proper solution may be implemented.

[marmarek]

For now, as a quick fix, I'm going to forbid changing template of AppVM, when it have some DispVMs (running or not). Later a proper solution may be implemented.

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-dom0-4.0.22-1.fc25 has been pushed to the r4.0 testing repository for dom0.
To test this update, please install it with the following command:

sudo qubes-dom0-update --enablerepo=qubes-dom0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-dom0-4.0.22-1.fc25 has been pushed to the r4.0 testing repository for dom0.
To test this update, please install it with the following command:

sudo qubes-dom0-update --enablerepo=qubes-dom0-current-testing

Changes included in this update

[marmarek]

Reopening to remind about proper solution.

[marmarek]

Reopening to remind about proper solution.

[shunju]

Maybe, priority should be lowered to reflect the reduced urgency, e.g. as a user I don’t see why this should delay the release of 4.0.

[shunju]

Maybe, priority should be lowered to reflect the reduced urgency, e.g. as a user I don’t see why this should delay the release of 4.0.

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-dom0-4.0.24-1.fc25 has been pushed to the r4.0 stable repository for dom0.
To install this update, please use the standard update command:

sudo qubes-dom0-update

Or update dom0 via Qubes Manager.

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-dom0-4.0.24-1.fc25 has been pushed to the r4.0 stable repository for dom0.
To install this update, please use the standard update command:

sudo qubes-dom0-update

Or update dom0 via Qubes Manager.

Changes included in this update