Design for multiple disposable VMs, improved template VMs, snapshots, savestates

[qubesuser]

This is a suggestion for several independent features that are useful on their own, and allow to replace the current Disposable VM and template VM code with more flexible systems, and provide support for template trees, snapshot trees, saving running VMs and trees of running snapshots.

Building blocks

1. Linked cloning: linked cloning is like regular cloning, but instead of copying storage, it links to the old VM and only stores changes in the new VM (using DM snapshot). When a VM with linked cloning is deleted, changed are moved to the VMs linked to it. Replacement linked cloning is like cloning but also causes all VMs that were linked to the cloned VM to be changed to link to the clone instead.
2. Volatile VMs: when a VM to be deleted is running or has references to it (such as being the template instance for a savestate), it is instead marked as "volatile" and deleted when it becomes possible
3. Immutable VMs: an immutable VM has immutable storage. If the user attempts to start an immutable VM, a linked clone is performed and the cloned VM is started instead. Normally it's called "vname-inst#" name, but if started due to qvm-run or qvm-open-in-(d)vm it's called "vmname-disp#" and marked as volatile. The immutable flag cannot be removed by the user if the VM is referenced in ways that require an immutable VM (such as being used as a template for a running VM or a VM with a savestate).
4. Versioned VMs: when a versioned VM is first marked versioned it is renamed to "vmname-next" and at first and every time it is shut down (or manually, if configured for manual commit), a new "vmname-v#" (with # an ascending number) immutable replacement linked clone is made. Older "vmname-v#" clones are made internal and are deleted unless the versioned VM is set to preserve history. "vmname" becomes an alias for the most recent versioned clone.
5. VMs with automatic hot savestates: causes a DisposableVM-like savestate to be created immediately and recreated every time the machine is shut down or the template is updated.
6. Manual savestates: allow to manually save VM state. Requires the template VM changes detailed below to be usable with non-standalone VMs. The savestate is copied or linked on cloning just like the other storage.

Replacing the current disposable VMs

With the new features, instead of the current system, a VM can be turned into a disposable VM factory by configuring it as versioned and having hot savestates.

To launch a disposable VM, simply start the "vmname" VM alias referring to the latest version immutable clone of the disposable VM factory.

To modify the disposable VM factory, start the "vmname-next" versioned VM itself, change it and shut it down. Disposable VMs can still be started during this thanks to the VM versioning mechanism.

This allows to have multiple disposable VM factories.

Replacing the current template VMs

With the new features, instead of the current system, any standalone versioned VM can be used as a template.

When a templated AppVMs is started without a manual savestate, the "vmname-current" version is used. If it is started with a savestate, then the "vmname-v#" version recorded in the savestate is
used, which allows to support savestates properly.

Since template versions are immutable, there is no need for the root.img/root-cow.img mechanism (the linked cloning does that internally).

New use cases

1. Amnesiac AppVMs / create disposable VM from any AppVM: by marking an AppVM immutable or versioned and using the current alias, it's possible to have an AppVM without saved state, or in other words a disposable VM
2. Versioned history-preserving template VM/AppVMs: the user can make an AppVM or template VM versioned and history-preserved and be able to go back to its previous state after every VM shutdown or manual commit, with efficient storage and fast access to the latest version
3. Efficient cloning: AppVMs and templates can be cloned without requiring twice the disk space
4. Trees of snapshots: the user can use versioned VMs and manual linked cloning to easily create a complex tree of VM "snapshots" with efficient storage, and can do so with both shutdown VMs and saved running VMs and for all kinds of VMs
5. Clone and backup running VMs: it's possible to clone the running version of any VM by making a savestate, cloning and resuming both; by making a VM versioned, there is also the option of cloning the latest committed version; of course this can also be used for backup

[marmarek]

I don't think we need such advanced storage system - the current one level template is enough for most cases (and already quite hard to understand for "normal user"...). But we've discussed internally somehow related idea - rework of Disposable VMs to split the current concept into two separate properties:

1. Define Disposable VM by its stateless - VM which don't have any persistent storage, all the changes are lost after its termination; in fact it would be somehow similar to your Immutable VM
2. Add an option to have savefile with preloaded VM state (snapshot of just started VM) - to speedup VM startup

Anyway any such major change needs to be postponed until after R4.0, where we will introduce new core (Qubes*VM classes). The current core is getting too hacky, the new one is mostly rewrite from scratch. Some work is already done here:
https://github.com/marmarek/qubes-core-admin/tree/core3-wip

[marmarek]

I don't think we need such advanced storage system - the current one level template is enough for most cases (and already quite hard to understand for "normal user"...). But we've discussed internally somehow related idea - rework of Disposable VMs to split the current concept into two separate properties:

1. Define Disposable VM by its stateless - VM which don't have any persistent storage, all the changes are lost after its termination; in fact it would be somehow similar to your Immutable VM
2. Add an option to have savefile with preloaded VM state (snapshot of just started VM) - to speedup VM startup

Anyway any such major change needs to be postponed until after R4.0, where we will introduce new core (Qubes*VM classes). The current core is getting too hacky, the new one is mostly rewrite from scratch. Some work is already done here:
https://github.com/marmarek/qubes-core-admin/tree/core3-wip