template's partition layout make it hard to extend root volume

[marmarek]

Qubes OS version: R4.0

Affected TemplateVMs: all

Current partition layout of template in Qubes 4.0 is:

1. xvda1: root filesystem (almost all space)
2. xvda2: EFI system partition (empty, prepared for PVHv2 boot with VM-provided kernel)
3. xvda3: BIOS boot (grub2 installed, used when kernel='')

This makes resizing root volume hard, because one need to move xvda[23] data to the (new) end of the disk first. Also, having partitions at all (de facto required by grub2), makes online resize of root volume hard/impossible.

[edit by @woju] qubes-devel thread: https://groups.google.com/d/msg/qubes-devel/23FwvVd-pjU/P4Y-h91rAAAJ

[woju]

Online resizing of partitions is possible, but not with util-linux tools (blockdev --rereadpt). It is possible with fdisk + partprobe, the latter being part of parted.

Andrew Clausen, author of partprobe, commented on qubes-devel:

The BLKRRPART ioctl is no good because it can't accommodate busy block
devices at all, i.e. resizing partition 1 when partition 2 on the same disk
is mounted.

Instead, Parted primarily uses the BLKPG family of ioctl to inform the
kernel of partition table changes. (It also has "new" support for the
device mapper -- but that's probably not relevant here.) You can read
about the BLKPG ioctl in /usr/include/linux/blkpg.h. Since 2012, the linux
kernel supports a new BLKPG feature to do online partition resizing, i.e.
telling the kernel to modify a mounted partition. I think this is what is
being used here.

The relevant Parted code is in the function linux_disk_commit(), which
calls _disk_sync_part_table() and _blkpg_resize_partition() inside
http://git.savannah.gnu.org/cgit/parted.git/tree/libparted/arch/linux.c

If the BLKPG ioctl fails, then partprobe/parted will throw an exception and
tell you about it.

https://groups.google.com/d/msgid/qubes-devel/CAAXZBWL2p_84BdurZrYSG%2B_pZwv65qyUWRPKpG-p2OrP0zJ6uw%40mail.gmail.com

[woju]

Online resizing of partitions is possible, but not with util-linux tools (blockdev --rereadpt). It is possible with fdisk + partprobe, the latter being part of parted.

Andrew Clausen, author of partprobe, commented on qubes-devel:

The BLKRRPART ioctl is no good because it can't accommodate busy block
devices at all, i.e. resizing partition 1 when partition 2 on the same disk
is mounted.

Instead, Parted primarily uses the BLKPG family of ioctl to inform the
kernel of partition table changes. (It also has "new" support for the
device mapper -- but that's probably not relevant here.) You can read
about the BLKPG ioctl in /usr/include/linux/blkpg.h. Since 2012, the linux
kernel supports a new BLKPG feature to do online partition resizing, i.e.
telling the kernel to modify a mounted partition. I think this is what is
being used here.

The relevant Parted code is in the function linux_disk_commit(), which
calls _disk_sync_part_table() and _blkpg_resize_partition() inside
http://git.savannah.gnu.org/cgit/parted.git/tree/libparted/arch/linux.c

If the BLKPG ioctl fails, then partprobe/parted will throw an exception and
tell you about it.

https://groups.google.com/d/msgid/qubes-devel/CAAXZBWL2p_84BdurZrYSG%2B_pZwv65qyUWRPKpG-p2OrP0zJ6uw%40mail.gmail.com

[marmarek]

The outcome of the discussion: delay 4.0rc2 a week and change partition layout to:

1. xvda1: EFI system partition (empty, prepared for PVHv2 boot with VM-provided kernel)
2. xvda2: BIOS boot (grub2 installed, used when kernel='')
3. xvda3: root filesystem (almost all space)

[marmarek]

The outcome of the discussion: delay 4.0rc2 a week and change partition layout to:

1. xvda1: EFI system partition (empty, prepared for PVHv2 boot with VM-provided kernel)
2. xvda2: BIOS boot (grub2 installed, used when kernel='')
3. xvda3: root filesystem (almost all space)

[qubesos-bot]

Automated announcement from builder-github

The package linux-utils has been pushed to the r4.0 testing repository for the CentOS centos7 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r4.0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package linux-utils has been pushed to the r4.0 testing repository for the CentOS centos7 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r4.0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-utils_4.0.9+deb8u1 has been pushed to the r4.0 testing repository for the Debian jessie template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing jessie-testing, then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-utils_4.0.9+deb8u1 has been pushed to the r4.0 testing repository for the Debian jessie template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing jessie-testing, then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-utils_4.0.9+deb9u1 has been pushed to the r4.0 testing repository for the Debian stretch template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing stretch-testing, then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-utils_4.0.9+deb9u1 has been pushed to the r4.0 testing repository for the Debian stretch template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing stretch-testing, then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-qubesimgconverter-4.0.9-1.fc25 has been pushed to the r4.0 testing repository for dom0.
To test this update, please install it with the following command:

sudo qubes-dom0-update --enablerepo=qubes-dom0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-qubesimgconverter-4.0.9-1.fc25 has been pushed to the r4.0 testing repository for dom0.
To test this update, please install it with the following command:

sudo qubes-dom0-update --enablerepo=qubes-dom0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-qubesimgconverter-4.0.9-1.fc24 has been pushed to the r4.0 testing repository for the Fedora fc24 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r4.0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-qubesimgconverter-4.0.9-1.fc24 has been pushed to the r4.0 testing repository for the Fedora fc24 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r4.0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-qubesimgconverter-4.0.9-1.fc25 has been pushed to the r4.0 testing repository for the Fedora fc25 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r4.0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-qubesimgconverter-4.0.9-1.fc25 has been pushed to the r4.0 testing repository for the Fedora fc25 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r4.0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-qubesimgconverter-4.0.9-1.fc26 has been pushed to the r4.0 testing repository for the Fedora fc26 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r4.0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-qubesimgconverter-4.0.9-1.fc26 has been pushed to the r4.0 testing repository for the Fedora fc26 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r4.0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package core-agent-linux has been pushed to the r4.0 testing repository for the CentOS centos7 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r4.0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package core-agent-linux has been pushed to the r4.0 testing repository for the CentOS centos7 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r4.0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-agent_4.0.12-1+deb8u1 has been pushed to the r4.0 testing repository for the Debian jessie template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing jessie-testing, then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-agent_4.0.12-1+deb8u1 has been pushed to the r4.0 testing repository for the Debian jessie template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing jessie-testing, then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-dnf-plugins-qubes-hooks-4.0.12-1.fc24 has been pushed to the r4.0 testing repository for the Fedora fc24 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r4.0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-dnf-plugins-qubes-hooks-4.0.12-1.fc24 has been pushed to the r4.0 testing repository for the Fedora fc24 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r4.0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package linux-utils has been pushed to the r4.0 stable repository for the Fedora centos7 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package linux-utils has been pushed to the r4.0 stable repository for the Fedora centos7 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-qubesimgconverter-4.0.9-1.fc24 has been pushed to the r4.0 stable repository for the Fedora fc24 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-qubesimgconverter-4.0.9-1.fc24 has been pushed to the r4.0 stable repository for the Fedora fc24 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-qubesimgconverter-4.0.9-1.fc25 has been pushed to the r4.0 stable repository for the Fedora fc25 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-qubesimgconverter-4.0.9-1.fc25 has been pushed to the r4.0 stable repository for the Fedora fc25 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-qubesimgconverter-4.0.9-1.fc26 has been pushed to the r4.0 stable repository for the Fedora fc26 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-qubesimgconverter-4.0.9-1.fc26 has been pushed to the r4.0 stable repository for the Fedora fc26 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package core-agent-linux has been pushed to the r4.0 stable repository for the Fedora centos7 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package core-agent-linux has been pushed to the r4.0 stable repository for the Fedora centos7 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-dnf-plugins-qubes-hooks-4.0.12-1.fc24 has been pushed to the r4.0 stable repository for the Fedora fc24 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-dnf-plugins-qubes-hooks-4.0.12-1.fc24 has been pushed to the r4.0 stable repository for the Fedora fc24 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-dnf-plugins-qubes-hooks-4.0.12-1.fc25 has been pushed to the r4.0 stable repository for the Fedora fc25 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-dnf-plugins-qubes-hooks-4.0.12-1.fc25 has been pushed to the r4.0 stable repository for the Fedora fc25 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-dnf-plugins-qubes-hooks-4.0.12-1.fc26 has been pushed to the r4.0 stable repository for the Fedora fc26 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-dnf-plugins-qubes-hooks-4.0.12-1.fc26 has been pushed to the r4.0 stable repository for the Fedora fc26 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-agent_4.0.12-1+deb8u1 has been pushed to the r4.0 stable repository for the Debian jessie template.
To install this update, please use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-agent_4.0.12-1+deb8u1 has been pushed to the r4.0 stable repository for the Debian jessie template.
To install this update, please use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-agent_4.0.12-1+deb9u1 has been pushed to the r4.0 stable repository for the Debian stretch template.
To install this update, please use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-agent_4.0.12-1+deb9u1 has been pushed to the r4.0 stable repository for the Debian stretch template.
To install this update, please use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-utils_4.0.9+deb8u1 has been pushed to the r4.0 stable repository for the Debian jessie template.
To install this update, please use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-utils_4.0.9+deb8u1 has been pushed to the r4.0 stable repository for the Debian jessie template.
To install this update, please use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-utils_4.0.9+deb9u1 has been pushed to the r4.0 stable repository for the Debian stretch template.
To install this update, please use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-utils_4.0.9+deb9u1 has been pushed to the r4.0 stable repository for the Debian stretch template.
To install this update, please use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-qubesimgconverter-4.0.9-1.fc25 has been pushed to the r4.0 stable repository for dom0.
To install this update, please use the standard update command:

sudo qubes-dom0-update

Or update dom0 via Qubes Manager.

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package python2-qubesimgconverter-4.0.9-1.fc25 has been pushed to the r4.0 stable repository for dom0.
To install this update, please use the standard update command:

sudo qubes-dom0-update

Or update dom0 via Qubes Manager.

Changes included in this update