Shrinking disk images

[jpouellet]

Qubes OS version (e.g., R3.1):

R3.2

Affected TemplateVMs (e.g., fedora-23, if applicable):

fedora-23, likely others

---

Expected behavior:

According to marmarek:

Actually, we mount /rw with "discard" option, which do notify lower
layer(s) of freed blocks. It is supported by the whole path
ext4(VM)->xen-blkfront/back->loop->ext4(dom0).

Actual behavior:

Unlinking files does not appear to shrink the reported disk image size.

Perhaps I'm misunderstanding the intended behavior?

Steps to reproduce the behavior:

Fill up /:

[user@dom0 ~]$ qvm-ls -d | grep -E '(name|--)'
----------------------+----+---------+-------+-------+--------+-----------+----------+-----------+----------+----------+
                 name | on |   state | updbl |  type |  label | priv-curr | priv-max | root-curr | root-max |     disk |
----------------------+----+---------+-------+-------+--------+-----------+----------+-----------+----------+----------+
[user@dom0 ~]$ qvm-run disp3 -p 'dd if=/dev/urandom of=$HOME/$RANDOM.crap bs=1M count=256' && qvm-ls -d | grep disp && qvm-run -qp disp3 'df -h /'
256+0 records in
256+0 records out
268435456 bytes (268 MB) copied, 1.71515 s, 157 MB/s
              <disp3> |  * | Running |       |       |    red |     64 MB |  2048 MB |   7165 MB | 10240 MB |  1932 MB |
Filesystem          Size  Used Avail Use% Mounted on
/dev/mapper/dmroot  9.8G  7.6G  1.7G  83% /
...
[user@dom0 ~]$ qvm-run disp3 -p 'dd if=/dev/urandom of=$HOME/$RANDOM.crap bs=1M count=256' && qvm-ls -d | grep disp && qvm-run -qp disp3 'df -h /'
256+0 records in
256+0 records out
268435456 bytes (268 MB) copied, 1.67399 s, 160 MB/s
              <disp3> |  * | Running |       |       |    red |     64 MB |  2048 MB |   7165 MB | 10240 MB |  3162 MB |
Filesystem          Size  Used Avail Use% Mounted on
/dev/mapper/dmroot  9.8G  8.9G  394M  96% /
...
[user@dom0 ~]$ qvm-run disp3 -p 'dd if=/dev/urandom of=$HOME/$RANDOM.crap bs=1M count=256' && qvm-ls -d | grep disp && qvm-run -qp disp3 'df -h /'
256+0 records in
256+0 records out
268435456 bytes (268 MB) copied, 1.96327 s, 137 MB/s
              <disp3> |  * | Running |       |       |    red |     64 MB |  2048 MB |   7165 MB | 10240 MB |  3421 MB |
Filesystem          Size  Used Avail Use% Mounted on
/dev/mapper/dmroot  9.8G  9.1G  138M  99% /

[user@dom0 ~]$ qvm-run disp3 -p 'dd if=/dev/urandom of=$HOME/$RANDOM.crap bs=1M count=256' && qvm-ls -d | grep disp && qvm-run -qp disp3 'df -h'
dd: error writing ___/home/user/1932.crap___: No space left on device
138+0 records in
137+0 records out
144175104 bytes (144 MB) copied, 0.893459 s, 161 MB/s
[user@dom0 ~]$ qvm-run disp3 -p 'dd if=/dev/urandom of=$HOME/$RANDOM.crap bs=1M count=256'; qvm-ls -d | grep disp; qvm-run -qp disp3 'df -h /'
dd: error writing ___/home/user/770.crap___: No space left on device
1+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00570792 s, 0.0 kB/s
              <disp3> |  * | Running |       |       |    red |     64 MB |  2048 MB |   7165 MB | 10240 MB |  3814 MB |
Filesystem          Size  Used Avail Use% Mounted on
/dev/mapper/dmroot  9.8G  9.3G     0 100% /

Unlink the crap:

[user@dom0 ~]$ qvm-run disp3 -p 'echo *.crap'
10433.crap 10989.crap 13789.crap 15249.crap 15531.crap 1611.crap 16416.crap 1729.crap 17721.crap 17942.crap 18827.crap 19214.crap 1932.crap 19591.crap 19620.crap 19762.crap 19864.crap 20127.crap 21951.crap 22111.crap 22586.crap 23028.crap 23999.crap 25220.crap 262.crap 26378.crap 28006.crap 28266.crap 29868.crap 30638.crap 30712.crap 31616.crap 31792.crap 3705.crap 3868.crap 4395.crap 585.crap 6322.crap 6822.crap 7210.crap 770.crap 7779.crap 7968.crap 8813.crap 8863.crap 945.crap 9799.crap
[user@dom0 ~]$ qvm-run disp3 -p 'rm *.crap'
[user@dom0 ~]$ qvm-run disp3 -p 'dd if=/dev/urandom of=$HOME/$RANDOM.crap bs=1M count=256'; qvm-ls -d | grep disp; qvm-run -qp disp3 'df -h /'
256+0 records in
256+0 records out
268435456 bytes (268 MB) copied, 1.59558 s, 168 MB/s
              <disp3> |  * | Running |       |       |    red |     64 MB |  2048 MB |   7165 MB | 10240 MB |  3855 MB |
Filesystem          Size  Used Avail Use% Mounted on
/dev/mapper/dmroot  9.8G  6.2G  3.1G  67% /

Wait a while, still the same:

[user@dom0 ~]$ qvm-ls -d | grep disp; qvm-run -qp disp3 'df -h /'

              <disp3> |  * | Running |       |       |    red |     64 MB |  2048 MB |   7165 MB | 10240 MB |  3908 MB |
Filesystem          Size  Used Avail Use% Mounted on
/dev/mapper/dmroot  9.8G  6.4G  2.9G  70% /

Related issues

#2132 (comment) -- suggestion to add warning and suggestion to increase size in new qubes-manager when disk is approaching full

[jpouellet]

Tested above in a dispvm, but appears to be true for normal non-cow-backed vms too. Did not make isolated test specifically here, but observed that my VM which once held several install ISOs and no longer does still has a ~12gb private image.

[jpouellet]

Tested above in a dispvm, but appears to be true for normal non-cow-backed vms too. Did not make isolated test specifically here, but observed that my VM which once held several install ISOs and no longer does still has a ~12gb private image.

[jpouellet]

Ah, well my dispvm's filesystems are not mounted with discard:

[user@fedora-23-dvm]~$ sudo mount | grep discard
[user@fedora-23-dvm]~$ 

But some other VMs are:

$ sudo mount | grep discard
/dev/xvdb on /rw type ext4 (rw,relatime,discard,data=ordered)
/dev/xvdb on /home type ext4 (rw,relatime,discard,data=ordered)

[jpouellet]

Ah, well my dispvm's filesystems are not mounted with discard:

[user@fedora-23-dvm]~$ sudo mount | grep discard
[user@fedora-23-dvm]~$ 

But some other VMs are:

$ sudo mount | grep discard
/dev/xvdb on /rw type ext4 (rw,relatime,discard,data=ordered)
/dev/xvdb on /home type ext4 (rw,relatime,discard,data=ordered)

[jpouellet]

Yeah nevermind... it does work after all.

[user@dom0 ~]$ qvm-ls -d | grep -E '(name|--|discard)'
----------------------+----+---------+-------+-------+--------+-----------+----------+-----------+----------+----------+
                 name | on |   state | updbl |  type |  label | priv-curr | priv-max | root-curr | root-max |     disk |
----------------------+----+---------+-------+-------+--------+-----------+----------+-----------+----------+----------+
         discard-test |  * | Running |       |       |    red |     65 MB |  2048 MB |   7165 MB | 10240 MB |   132 MB |
[user@dom0 ~]$ qvm-run -p discard-test 'cat /dev/urandom >> garbage'
cat: write error: No space left on device
[user@dom0 ~]$ qvm-ls -d | grep -E '(name|--|discard)'
----------------------+----+---------+-------+-------+--------+-----------+----------+-----------+----------+----------+
                 name | on |   state | updbl |  type |  label | priv-curr | priv-max | root-curr | root-max |     disk |
----------------------+----+---------+-------+-------+--------+-----------+----------+-----------+----------+----------+
         discard-test |  * | Running |       |       |    red |   1997 MB |  2048 MB |   7165 MB | 10240 MB |  2008 MB |
[user@dom0 ~]$ qvm-run -p discard-test 'rm garbage'
[user@dom0 ~]$ qvm-ls -d | grep -E '(name|--|discard)'
----------------------+----+---------+-------+-------+--------+-----------+----------+-----------+----------+----------+
                 name | on |   state | updbl |  type |  label | priv-curr | priv-max | root-curr | root-max |     disk |
----------------------+----+---------+-------+-------+--------+-----------+----------+-----------+----------+----------+
         discard-test |  * | Running |       |       |    red |     65 MB |  2048 MB |   7165 MB | 10240 MB |   153 MB |

Just isn't enabled everywhere.

[jpouellet]

Yeah nevermind... it does work after all.

[user@dom0 ~]$ qvm-ls -d | grep -E '(name|--|discard)'
----------------------+----+---------+-------+-------+--------+-----------+----------+-----------+----------+----------+
                 name | on |   state | updbl |  type |  label | priv-curr | priv-max | root-curr | root-max |     disk |
----------------------+----+---------+-------+-------+--------+-----------+----------+-----------+----------+----------+
         discard-test |  * | Running |       |       |    red |     65 MB |  2048 MB |   7165 MB | 10240 MB |   132 MB |
[user@dom0 ~]$ qvm-run -p discard-test 'cat /dev/urandom >> garbage'
cat: write error: No space left on device
[user@dom0 ~]$ qvm-ls -d | grep -E '(name|--|discard)'
----------------------+----+---------+-------+-------+--------+-----------+----------+-----------+----------+----------+
                 name | on |   state | updbl |  type |  label | priv-curr | priv-max | root-curr | root-max |     disk |
----------------------+----+---------+-------+-------+--------+-----------+----------+-----------+----------+----------+
         discard-test |  * | Running |       |       |    red |   1997 MB |  2048 MB |   7165 MB | 10240 MB |  2008 MB |
[user@dom0 ~]$ qvm-run -p discard-test 'rm garbage'
[user@dom0 ~]$ qvm-ls -d | grep -E '(name|--|discard)'
----------------------+----+---------+-------+-------+--------+-----------+----------+-----------+----------+----------+
                 name | on |   state | updbl |  type |  label | priv-curr | priv-max | root-curr | root-max |     disk |
----------------------+----+---------+-------+-------+--------+-----------+----------+-----------+----------+----------+
         discard-test |  * | Running |       |       |    red |     65 MB |  2048 MB |   7165 MB | 10240 MB |   153 MB |

Just isn't enabled everywhere.