Mount / with 'discard'? #2479

jpouellet · Dec 3, 2016

I am wondering if there is a reason we do not enable discard for the root filesystem.

I just reclaimed ~10gb of real dom0 disk space by enabling discard on / in a particularly offending StandaloneVM (and dd-ing a bunch of garbage to a file & removing it).

I imagine doing so would afford similar savings for TemplateVMs which accumulate lots of update churn over time.

Relevant tickets:

#2478 (where I did some testing)

For StandaloneVM it indeed may help, but for AppVM unfortunately not - dm-snapshot do not pass DISCARD operations to lower level. But if dm-linear do, it still may worth enabling it for / too, to have it working in StandaloneVM and TemplateVM. Do I understand correctly that you claim it do work there?

marmarek · Dec 3, 2016

For StandaloneVM it indeed may help, but for AppVM unfortunately not - dm-snapshot do not pass DISCARD operations to lower level. But if dm-linear do, it still may worth enabling it for / too, to have it working in StandaloneVM and TemplateVM. Do I understand correctly that you claim it do work there?

Do I understand correctly that you claim it do work there?

Unless I am misinterpreting the below (root-curr & final disk field), yes:

[user@dom0 ~]$ qvm-ls -d standalone-discard-test
-------------------------+----+---------+-------+------+-------+-----------+----------+-----------+----------+---------+
                    name | on |   state | updbl | type | label | priv-curr | priv-max | root-curr | root-max |    disk |
-------------------------+----+---------+-------+------+-------+-----------+----------+-----------+----------+---------+
 standalone-discard-test |  * | Running |   Yes |      |   red |     65 MB |  2048 MB |   6170 MB | 10240 MB | 6648 MB |
[user@dom0 ~]$ qvm-run -p standalone-discard-test 'sudo dd if=/dev/urandom of=/var/garbage'
dd: writing to ___/var/garbage___: No space left on device
8045185+0 records in
8045184+0 records out
4119134208 bytes (4.1 GB) copied, 46.8408 s, 87.9 MB/s
[user@dom0 ~]$ qvm-ls -d standalone-discard-test
-------------------------+----+---------+-------+------+-------+-----------+----------+-----------+----------+----------+
                    name | on |   state | updbl | type | label | priv-curr | priv-max | root-curr | root-max |     disk |
-------------------------+----+---------+-------+------+-------+-----------+----------+-----------+----------+----------+
 standalone-discard-test |  * | Running |   Yes |      |   red |     65 MB |  2048 MB |  10097 MB | 10240 MB | 10575 MB |
[user@dom0 ~]$ qvm-run -p standalone-discard-test 'sudo rm /var/garbage'
[user@dom0 ~]$ qvm-ls -d standalone-discard-test
-------------------------+----+---------+-------+------+-------+-----------+----------+-----------+----------+---------+
                    name | on |   state | updbl | type | label | priv-curr | priv-max | root-curr | root-max |    disk |
-------------------------+----+---------+-------+------+-------+-----------+----------+-----------+----------+---------+
 standalone-discard-test |  * | Running |   Yes |      |   red |     65 MB |  2048 MB |   6170 MB | 10240 MB | 6647 MB |
[user@dom0 ~]$ qvm-ls -d standalone-discard-test

jpouellet · Dec 4, 2016

Do I understand correctly that you claim it do work there?

Unless I am misinterpreting the below (root-curr & final disk field), yes:

[user@dom0 ~]$ qvm-ls -d standalone-discard-test
-------------------------+----+---------+-------+------+-------+-----------+----------+-----------+----------+---------+
                    name | on |   state | updbl | type | label | priv-curr | priv-max | root-curr | root-max |    disk |
-------------------------+----+---------+-------+------+-------+-----------+----------+-----------+----------+---------+
 standalone-discard-test |  * | Running |   Yes |      |   red |     65 MB |  2048 MB |   6170 MB | 10240 MB | 6648 MB |
[user@dom0 ~]$ qvm-run -p standalone-discard-test 'sudo dd if=/dev/urandom of=/var/garbage'
dd: writing to ___/var/garbage___: No space left on device
8045185+0 records in
8045184+0 records out
4119134208 bytes (4.1 GB) copied, 46.8408 s, 87.9 MB/s
[user@dom0 ~]$ qvm-ls -d standalone-discard-test
-------------------------+----+---------+-------+------+-------+-----------+----------+-----------+----------+----------+
                    name | on |   state | updbl | type | label | priv-curr | priv-max | root-curr | root-max |     disk |
-------------------------+----+---------+-------+------+-------+-----------+----------+-----------+----------+----------+
 standalone-discard-test |  * | Running |   Yes |      |   red |     65 MB |  2048 MB |  10097 MB | 10240 MB | 10575 MB |
[user@dom0 ~]$ qvm-run -p standalone-discard-test 'sudo rm /var/garbage'
[user@dom0 ~]$ qvm-ls -d standalone-discard-test
-------------------------+----+---------+-------+------+-------+-----------+----------+-----------+----------+---------+
                    name | on |   state | updbl | type | label | priv-curr | priv-max | root-curr | root-max |    disk |
-------------------------+----+---------+-------+------+-------+-----------+----------+-----------+----------+---------+
 standalone-discard-test |  * | Running |   Yes |      |   red |     65 MB |  2048 MB |   6170 MB | 10240 MB | 6647 MB |
[user@dom0 ~]$ qvm-ls -d standalone-discard-test

Is it worth it to simply add discard for '/' to templates, or will that result in slightly higher CPU usage in appVMs where discard requests are simply generated and forgotten?

I think it may be the former: Adding discard to the '/' entry in the templates is very easy fix that cascades to newly created standalone VMs, and appVMs would be trying to write to '/' only rarely so probably no performance hit.

tasket · Dec 4, 2016

Is it worth it to simply add discard for '/' to templates, or will that result in slightly higher CPU usage in appVMs where discard requests are simply generated and forgotten?

I think it may be the former: Adding discard to the '/' entry in the templates is very easy fix that cascades to newly created standalone VMs, and appVMs would be trying to write to '/' only rarely so probably no performance hit.

If we do this, https://www.qubes-os.org/doc/template/debian/upgrade-8-to-9/#compacting-the-upgraded-template (and others) should be updated.

jpouellet · Dec 22, 2016

If we do this, https://www.qubes-os.org/doc/template/debian/upgrade-8-to-9/#compacting-the-upgraded-template (and others) should be updated.

AFAIK, templates still need qvm-trim-template, so that doc would be the same. What the proposed change actually affects is the root fs of certain standalone VMs (which inherit /etc/fstab from templates).

If we assume that some users will opt to make moderate-heavy use of standalone VMs (from PVM templates), then this change will be a real benefit to them.

tasket · Dec 22, 2016

AFAIK, templates still need qvm-trim-template, so that doc would be the same. What the proposed change actually affects is the root fs of certain standalone VMs (which inherit /etc/fstab from templates).

If we assume that some users will opt to make moderate-heavy use of standalone VMs (from PVM templates), then this change will be a real benefit to them.

Actually, the upgrade docs could have a separate word about using 'fstrim /' internally after upgrading standalone VMs. Otherwise, I think there are very few other docs affected by this. We could put a blurb in doc/templates about standalone VMs generated from Linux/PVM templates: How they do not use COW on root fs and can fstrim it internally.

tasket · Dec 22, 2016

Actually, the upgrade docs could have a separate word about using 'fstrim /' internally after upgrading standalone VMs. Otherwise, I think there are very few other docs affected by this. We could put a blurb in doc/templates about standalone VMs generated from Linux/PVM templates: How they do not use COW on root fs and can fstrim it internally.

@tasket: Added.

andrewdavidwong · Dec 22, 2016

@tasket: Added.

Oops, I added some also...
QubesOS/qubes-doc#254

tasket · Dec 22, 2016

Oops, I added some also...
QubesOS/qubes-doc#254

Since this is flagged as R4.0 I'll go ahead and state how relevant this is to the new release:

Templates now seem to easily trim their root volumes... Should be no special cases and no significant performance issues when adding discard to fstab.
dom0 root now has a problem where it expands too much, especially when backing up/restoring large amounts of data. Adding discard to root in the fstab should resolve this.

tasket · Nov 27, 2017

Since this is flagged as R4.0 I'll go ahead and state how relevant this is to the new release:

Templates now seem to easily trim their root volumes... Should be no special cases and no significant performance issues when adding discard to fstab.
dom0 root now has a problem where it expands too much, especially when backing up/restoring large amounts of data. Adding discard to root in the fstab should resolve this.

Thread:
https://groups.google.com/d/msgid/qubes-users/2a2ca8a8-a01b-5214-e954-96d4c92588dd%40posteo.net

tasket · Nov 27, 2017

Thread:
https://groups.google.com/d/msgid/qubes-users/2a2ca8a8-a01b-5214-e954-96d4c92588dd%40posteo.net

Automated announcement from builder-github

The package qubes-core-agent_4.0.19-1+deb8u1 has been pushed to the r4.0 testing repository for the Debian jessie template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing jessie-testing, then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

QubesOS/qubes-issues

Join GitHub today

Mount / with 'discard'? #2479

Comments

jpouellet commented Dec 3, 2016 • edited Edited 1 time jpouellet edited Dec 3, 2016 (most recent)

Relevant tickets:

jpouellet referenced this issue Dec 3, 2016

This comment has been minimized.

marmarek commented Dec 3, 2016

This comment has been minimized.

jpouellet commented Dec 4, 2016 • edited Edited 1 time jpouellet edited Dec 4, 2016 (most recent)

This comment has been minimized.

tasket commented Dec 4, 2016

andrewdavidwong added C: core enhancement labels Dec 4, 2016

andrewdavidwong added this to the Release 4.0 milestone Dec 4, 2016

This comment has been minimized.

jpouellet commented Dec 22, 2016

This comment has been minimized.

tasket commented Dec 22, 2016

This comment has been minimized.

tasket commented Dec 22, 2016

andrewdavidwong added a commit to QubesOS/qubes-doc that referenced this issue Dec 22, 2016

This comment has been minimized.

andrewdavidwong commented Dec 22, 2016

This comment has been minimized.

tasket commented Dec 22, 2016

jpouellet referenced this issue in QubesOS/qubes-doc Dec 22, 2016

This comment has been minimized.

tasket commented Nov 27, 2017

This comment has been minimized.

tasket commented Nov 27, 2017

marmarek closed this in marmarek/qubes-core-agent-linux@c5c4572 Jan 24, 2018

This comment has been minimized.

qubesos-bot commented Jan 24, 2018

qubesos-bot added the r4.0-jessie-cur-test label Jan 24, 2018

qubesos-bot referenced this issue in QubesOS/updates-status Jan 24, 2018

This comment has been minimized.

qubesos-bot commented Jan 24, 2018

qubesos-bot added the r4.0-stretch-cur-test label Jan 24, 2018

This comment has been minimized.

qubesos-bot commented Jan 24, 2018

qubesos-bot added the r4.0-fc24-cur-test label Jan 24, 2018

This comment has been minimized.

qubesos-bot commented Jan 24, 2018

qubesos-bot added the r4.0-fc25-cur-test label Jan 24, 2018

This comment has been minimized.

qubesos-bot commented Jan 24, 2018

qubesos-bot added the r4.0-fc26-cur-test label Jan 24, 2018

This comment has been minimized.

qubesos-bot commented Jan 24, 2018

qubesos-bot added the r4.0-centos7-cur-test label Jan 24, 2018

qubesos-bot added r4.0-jessie-stable and removed r4.0-jessie-cur-test labels Feb 6, 2018

This comment has been minimized.

qubesos-bot commented Feb 6, 2018

qubesos-bot added r4.0-stretch-stable r4.0-fc24-stable r4.0-fc25-stable and removed r4.0-stretch-cur-test r4.0-fc24-cur-test r4.0-fc25-cur-test labels Feb 6, 2018

This comment has been minimized.

qubesos-bot commented Feb 6, 2018

qubesos-bot added r4.0-fc26-stable and removed r4.0-fc26-cur-test labels Feb 6, 2018

This comment has been minimized.

qubesos-bot commented Feb 6, 2018

qubesos-bot added r4.0-centos7-stable and removed r4.0-centos7-cur-test labels Feb 6, 2018

This comment has been minimized.

tasket commented Mar 18, 2018

andrewdavidwong reopened this Mar 18, 2018

This comment has been minimized.

marmarek commented Mar 18, 2018

marmarek closed this Mar 18, 2018

jpouellet commented Dec 3, 2016 •

edited

Edited 1 time

jpouellet edited Dec 3, 2016 (most recent)

jpouellet commented Dec 4, 2016 •

edited

Edited 1 time

jpouellet edited Dec 4, 2016 (most recent)

marmarek closed this in `marmarek/qubes-core-agent-linux@c5c4572` Jan 24, 2018