DisposableVMs: support for in-RAM execution only (for anti-forensics)

[marmarek]

Reported by joanna on 25 Sep 2014 20:26 UTC
Currently volatile.img is being backed up on the fs. See: https://groups.google.com/forum/#!topic/qubes-devel/QwL5PjqPs-4/discussion

Migrated-From: https://wiki.qubes-os.org/ticket/904

[marmarek]

Two options are evaluated for it:

• use RAM only (including turning off swap)
• encrypt everything written to disk with disposable key

/cc @qubesuser

[marmarek]

Pasting @v6ak comment from linked discussion:

I've some scripts that I use for a temporary swapfile and temporary filesystem. They use a random key (from /dev/random). The tmp filesystem uses some configuration for better performance by disabling some features like journaling. (We don't need journaling for filesystem that are expected to be unreadable after reboot…)

My current usage:

1. The swapfile is attached automatically added in the background after 120 seconds in order not to block by reading from /dev/random during the system boot.
2. The largetmp is mounted only when needed, i.e. manually. (Yes, the usage of sudo suggests that…) I usually use tmpfs, but when I need something large, I mount the largetmp. I was thinking about automount of largetmp, but I was unsure about safety and some other potential issues. (Moreover, largetmp lies on rotational HDD, so using it instead of tmpfs could cause much more HDD usage and more power consumption.)

Some security considerations:

1. It is essential to handle safely situation when largetmp is not mounted. This is the reason why I use /tmp/large and not /large/tmp. If I forget to mount it, the worst thing that can happen is writing large amount of data to RAM. If it was in /large/tmp, accidental writing to a less protected partition (i.e. /large or /, which depends on the setup) may happen if the largetmp1 is not mounted. (Which is what I've once accidentally done. It was followed by several days of continuous wiping…)
2. The /dev/random is usually seeded from saved random seed. When some wear-levelling or relocation is used, the random seed might be available to local forensics, which could reduce the effective entropy of the key in the considered case. (Fortunatelly, the Qubes login screen requires some keystrokes, which adds some entrophy.)

There are the scripts and crypttab lines: https://gist.github.com/v6ak/3171313bc2c22efc263d

[rootkovska]

FWIW, I really like the idea of encrypting volatile.img with a one-time key and then throw it away after the DispVM shutdown. Can we ensure the dm will never write the key to dom0 fs anywhere? Do we need to disable swap in dom0 for that?

[marmarek]

No need to disable dom0 swap - VM memory is never written to disk (AFAIR
it isn't even supported).
The tricky part would be to integrate it with our DispVM implementation

• based on savefiles, to each DispVM have different key. But that's
  surely doable.

And it should be trivial for normal AppVMs when #1308 got implemented.

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[v6ak]

First, I see two levels of anti-forensic DVMs. Attacker is not able to get any data from the DVM, if:

1. …the whole computer is shut down and he cannot get any RAM dump, but attacker knows the encryption password. (Supposing that RAM does not persist. AFAIK this is sometimes not kept on older RAM types (see cold boot attacks), but it is practically kept on modern RAMs.)
2. …the DVM is shut down. (But the instance of Qubes might be still running) Attacker is able to get RAM dump (and thus e.g. extract encryption keys for full-disk-encryption), but this is not useful for extracting information about the DVM that is not running. Attacker also knows the encryption password.

The level 1 protection can be implemented IMHO relatively easily in dom0:

• All temporary files (e.g. volatile.img and COWs) are saved to some filesystem encrypted by a disposable key.
• Any dom0 swap should be encrypted by a disposable key. (I am not sure about default setup on Qubes.) If this is not done, it might be hard to ensure that no related sensitive data (e.g. encryption keys of the temporary filesystem above) might be swapped on HDD and thus potentially exposed to the attacker. This is likely doable, but much harder.

Both of them this might be also useful for standard VMs and also or performance reasons, as the disposable FS might be configured not to persist data reliably (e.g. data=writeback, disabled metadata journal and so on). But this is rather a nice side effect than primary goal of this issue.

By design, hibernation can't be supported.

The level 2 protection seems to be much harder:

• Even having some swap in dom0 is potentially an issue. Remember that encryption currently runs in dom0. Maybe this could be offloaded to some “stubvm” for these cases. (Note that enough entropy in such stubvm has to be ensured in some way.)
• When DVM shuts down, the related memory must be wiped immediately. I am not sure if Xen does that. It is likely that it does that lazily (in the same way that Linux kernel without proper GRSec config does) for performance reasons. But doing that lazily implies a potential problem.
• When the DVM releases some allocated memory (through Qmemman), it also must be wiped soon enough. (Theoretically, wiping might be delayed after VM shutdown, but not more.)
• Maybe a proper DVM shutdown indication has to be implemented.

Rather a side note at this point: I have to correct my statement about /dev/random and relocation. With my today's knowledge, this issue is true only for /dev/urandom, not for /dev/random, at least if standard systemd is used.

[marmarek]

I was thinking about implementing that volatile.img encryption inside of
VM, not in dom0. Dom0 would not know that encryption key. In this case
we can be sure that encryption key will not land in dom0 swap. Generally
this give us rather nice property - every data in DispVM is either in VM
RAM (not dom0 RAM), or written to volatile.img, encrypted. This doesn't
include some intentional leaks, cover channels etc, but this is offtopic
here.

As said earlier - VM memory is never written to dom0 swap (or any other
dom0 part). This, I think, fully solve the first case.

As for the second case, we "just" need to ensure that VM memory is wiped
when VM is shut down. In practice, I think this is the case, because all
the memory released by VM will be redistributed to other VMs. And before
VM get such memory, it is scrubbed by Xen. But in theory qmemman leave
some small part of memory (50MB) unassigned just for Xen internal use.
Theoretically it can happen that Xen will give some part from that pool
to the VM, instead of reusing just released memory - we do not control
which memory page is assigned where.
this pool and

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?