FS#2951 - x86_64 ext4 generic: online resizing of root partition fails

[openwrt-bot]

jneuhauser:

Used HW/SW:

• PCEngines APU2C4
• OpenWrt SNAPSHOT, r12776-437eb41f23

Steps to reproduce:

• Install needed packages:
  root@OpenWrt:~# opkg update && opkg install fdisk resize2fs
• Extend root partition from 104 MB to 256 MB with fdisk:
  root@OpenWrt:~# fdisk /dev/mmcblk0 << EOF
p
d
2
n
p
2
33792
+256M

w
EOF

• Extend ext4 filesystem:
  root@OpenWrt:~# resize2fs /dev/mmcblk0p2

Here are the output of resize2fs:
root@OpenWrt:~# resize2fs /dev/mmcblk0p2
resize2fs 1.45.4 (23-Sep-2019)
Filesystem at /dev/mmcblk0p2 is mounted on /; on-line resizing required
old_desc_blocks = 1, new_desc_blocks = 1
Performing an on-line resize of /dev/mmcblk0p2 to 65664 (4k) blocks.
resize2fs: Invalid argument While trying to add group #1

Here are the dmesg output:
root@OpenWrt:~# dmesg | tail -n 6
[50698.949591] EXT4-fs (mmcblk0p2): resizing filesystem from 32768 to 65664 blocks
[50698.957030] EXT4-fs (mmcblk0p2): resizing filesystem from 32768 to 65536 blocks
[50698.969299] EXT4-fs warning (device mmcblk0p2): reserve_backup_gdb:1033: reserved block 8 not at offset 7
[50698.978959] EXT4-fs warning (device mmcblk0p2): ext4_resize_fs:2120: error (-22) occurred during file system resize
[50698.989429] EXT4-fs (mmcblk0p2): resized filesystem to 32768
[50698.999227] EXT4-fs warning (device mmcblk0p2): reserve_backup_gdb:1033: reserved block 8 not at offset 7

Offline resizing does work as expected.

[openwrt-bot]

jneuhauser:

Some time ago i´m able to install all needed packages into my rootfs partition without the need of resizing.

The following commits changed the old size of 256 MB first to 128 MB and later to 104 MB.
469ba33
61c57af

This really sucks as i do not understand the reason for such small and not resizeable rootfs partiitons on the x86_64 target.

[openwrt-bot]

cronix:

I can confirm this issue,
having exactly the same outputs with todays trunk builds on my APU2 hardware as reported by the user above.

root@OpenWrt:~# resize2fs /dev/sda2
resize2fs 1.45.6 (20-Mar-2020)
Filesystem at /dev/sda2 is mounted on /; on-line resizing required
old_desc_blocks = 1, new_desc_blocks = 1
resize2fs: Invalid argument While checking for on-line resizing support


[ 275.145363] EXT4-fs (sda2): resizing filesystem from 26624 to 3905430 blocks
[ 275.153734] EXT4-fs warning (device sda2): reserve_backup_gdb:1033: reserved block 8 not at offset 7
[ 275.162926] EXT4-fs warning (device sda2): ext4_resize_fs:2120: error (-22) occurred during file system resize
[ 275.172949] EXT4-fs (sda2): resized filesystem to 32768
[ 275.178220] EXT4-fs warning (device sda2): ext4_group_extend:1804: can't shrink FS - resize aborted


[openwrt-bot]

MarioT:

Issue was already present in the x86_64 image on 19.07.7
(Using CompuLab fit-PC4, not far from a PCEngines APU2C4)

Could be related to the partition not aligned to 4k boundary.
Following workaround has allowed me to resize the ext4 "root":

mount -o remount,ro / #Remount root as ReadOnly
tune2fs -O^resize_inode /dev/mmcblk0p2 #Remove reserved GDT blocks
fsck.ext4 /dev/mmcblk0p2 #Fix part, answer yes to remove GDT blocks remnants

#reboot
resize2fs /dev/mmcblk0p2

[openwrt-bot]

orev:

I'm seeing the same issue on the 21.02.0 release and Mario's workaround did seem to work. However, I had better success resizing it when booted to a live USB (Fedora in my case). Using tune2fs/fsck causes the filesystem to need repairs, which just doesn't seem like the "right" way to approach it.

[openwrt-bot]

SiNONiMiTY:

This is working. Must be added as a reference for those who needs this.

[JOduMonT]

mount -o remount,ro / #Remount root as ReadOnly tune2fs -O^resize_inode /dev/mmcblk0p2 #Remove reserved GDT blocks fsck.ext4 /dev/mmcblk0p2 #Fix part, answer yes to remove GDT blocks remnants

#reboot resize2fs /dev/mmcblk0p2

in my case; to resized my root (/) only these commands has been required

# Remount root as ReadOnly 
mount -o remount,ro /
# Check and Repair blocks
e2fsck -f /dev/sda2

then after a reboot resize /dev/sda2 worked perfectly

[PupilTong]

for x86 openwrt 21.02:
The most direct way:
opkg update && opkg install parted

parted
p
resizepart 2
q
``
# reboot

resize2fs /dev/sda2

[nicsure]

@PupilTong

resize2fs /dev/sda2

resize2fs: Invalid argument While trying to add group #1

[ztatw]

add a comment for caution: I modified the partition by fdisk, just delete partition and create it again, because the sda2 is the last partition, when I delete then create, the default partition size is use the remaining space(that could be a trick). But I forget to update grub.cfg, the PARTUUID was changed, so I have to boot via Ubuntu live CD, mount sda1, update the PARTUUID

[jck112]

This looks to be a bug in make_ext4fs (which Google abandoned several years ago for Android). Fortunately mkfs.ext4 is more or less a direct replacement.

If using imagebuilder you can apply this patch:

--- include/image.mk	2023-04-27 13:28:15
+++ include/image.mk	2023-05-17 09:56:18
@@ -262,12 +262,12 @@
 endef
 
 define Image/mkfs/ext4
-	$(STAGING_DIR_HOST)/bin/make_ext4fs -L rootfs \
-		-l $(ROOTFS_PARTSIZE) -b $(CONFIG_TARGET_EXT4_BLOCKSIZE) \
+	$(STAGING_DIR_HOST)/bin/mkfs.ext4 -T default -L rootfs \
+		-b $(CONFIG_TARGET_EXT4_BLOCKSIZE) \
+		-d $(call mkfs_target_dir,$(1))/ \
 		$(if $(CONFIG_TARGET_EXT4_RESERVED_PCT),-m $(CONFIG_TARGET_EXT4_RESERVED_PCT)) \
-		$(if $(CONFIG_TARGET_EXT4_JOURNAL),,-J) \
-		$(if $(SOURCE_DATE_EPOCH),-T $(SOURCE_DATE_EPOCH)) \
-		$@ $(call mkfs_target_dir,$(1))/
+		$(if $(CONFIG_TARGET_EXT4_JOURNAL),,-O ^has_journal) \
+		$@ $$(($(ROOTFS_PARTSIZE)/$(CONFIG_TARGET_EXT4_BLOCKSIZE)))
 endef
 
 define Image/Manifest

Then you can simply live resize with:

parted -s /dev/mmcblk1 resizepart 2 100%
resize2fs /dev/mmcblk1p2

[wtdcode]

https://openwrt.org/docs/guide-user/installation/openwrt_x86

Documents here using losetup /dev/loop1 /dev/sda2 does the trick.

[zokier]

I can confirm that the change to mkfs.ext4 as suggested by @jck112 above does indeed produce filesystems that are resizeable without any workarounds. Does anyone know if this is a change that would be acceptable as PR, or are there some reasons why make_ext4fs would be preferred by openwrt? As far as I can tell, e2fsprogs is already included in the build environment so this doesn't add any new dependency and maybe could even get rid of make_ext4fs completely.

If I understand the loopback workaround correctly, it is basically tricking resize2fs to do offline resize despite the partition being mounted. That feels wildly unsafe, I assume the offline/online resize distinction is there for a good reason.

[zokier]

I just realized a major problem using mke2fs: it does not change the uid/gid of files, so all the files in the created FS end up being owned by the uid/gid of builder! There is a PR for e2fsprogs that allows forcing a uid/gid, but idk if it'll ever get merged: tytso/e2fsprogs#126

Android seems to be doing something clever here, so maybe we could borrow from them

[zokier]

In case anyone is following along, I did start working on a patchset to make mkfs.ext4 happen: https://github.com/openwrt/openwrt/compare/main...zokier:openwrt:mkfs.ext4?expand=1

It is still very much WIP and not much tested, but I did build one image with it already successfully :) I'll try to get that cleaned up and posted as PR when I'm ready

[seoeaa]

It works for me
opkg update
opkg install losetup resize2fs parted
parted /dev/nvme0n1 resizepart 2 1000M
losetup /dev/loop0 /dev/nvme0n1p2
resize2fs -f /dev/loop0
reboot