Kernel 4.14: SQUASHFS error: unable to read xattr id index table

[pintuk]

Hi,

This is regarding the squashfs mount failure that I am getting on my
device during boot time.
I just wanted to know if someone else has come across this issue, or
this issue is already fixed, or this is altogether a different issue?

Here are more details:
Kernel: 4.14.170 ; Qualcomm chipset (arm32 bit)
Platform: busybox
Storage: NAND 512MB
Filesystem: ubifs + squashfs
ubi0 : with 5 volumes (rootfs, usrfs, others)
Kernel command line: ro rootwait console=ttyMSM0,115200,n8
rootfstype=squashfs root=/dev/mtdblock34 ubi.mtd=30,0,30 ....

Background:
We are using ubifs filesystem with squashfs for rootfs (as ready only).
First we tried to flash "usrfs" (data) volume (ubi0_1) and it worked
fine (with device booting successfully).

Next we are trying to flash "rootfs" volume (ubi0_0) now. The volume
flashing is successful but after that when we reboot the system we are
getting below errors.

Logs:
[....]
[ 4.589340] vreg_conn_pa: dis▒[ 4.602779] squashfs: SQUASHFS
error: unable to read xattr id index table
[...]
[ 4.964083] No filesystem could mount root, tried:
[ 4.964087] squashfs
[ 4.966255]
[ 4.973443] Kernel panic - not syncing: VFS: Unable to mount root
fs on unknown-block(31,34)

---

[ 4.246861] ubi0: attaching mtd30
[ 4.453241] ubi0: scanning is finished
[ 4.460655] ubi0: attached mtd30 (name "system", size 216 MiB)
[ 4.460704] ubi0: PEB size: 262144 bytes (256 KiB), LEB size: 253952 bytes
[ 4.465562] ubi0: min./max. I/O unit sizes: 4096/4096, sub-page size 4096
[ 4.472483] ubi0: VID header offset: 4096 (aligned 4096), data offset: 8192
[ 4.479295] ubi0: good PEBs: 864, bad PEBs: 0, corrupted PEBs: 0
[ 4.486067] ubi0: user volume: 5, internal volumes: 1, max. volumes
count: 128
[ 4.492311] ubi0: max/mean erase counter: 4/0, WL threshold: 4096,
image sequence number: 1
[ 4.499333] ubi0: available PEBs: 0, total reserved PEBs: 864, PEBs
reserved for bad PEB handling: 60

So, we just wanted to know if this issue is related to squashfs or if
there is some issue with our volume flashing.
Note: We are using fastboot mechanism to support UBI volume flashing.

Observation:
Recently I have seen some squashfs changes related to similar issues
(xattr) so I wanted to understand if these changes are relevant to our
issue or not ?

Age Commit message(Expand) Author
2021-03-30 squashfs: fix xattr id and id lookup sanity checks
Phillip Lougher
2021-03-30 squashfs: fix inode lookup sanity checks
Sean Nyekjaer
2021-02-23 squashfs: add more sanity checks in xattr id lookup
Phillip Lougher
2021-02-23 squashfs: add more sanity checks in inode lookup
Phillip Lougher
2021-02-23 squashfs: add more sanity checks in id lookup
Phillip Lougher

Please let us know your opinion about this issue...
It will help us to decide whether the issue is related to squashfs or not.

Thanks,
Pintu

[pintuk]

I have posted this query even in mainline and awaiting response.

[plougher]

Your kernel (4.14.170) was released on 5 Feb 2020, and so it won't contain any of the above commits. The xattr -id code in 4.14.170, was last updated in May 2011, and so it is much more likely the problem is elsewhere.

The xattr id index table is written to the end of the Squashfs filesystem, and it is the first table read on mounting.

As such this is the error you will receive if the Squashfs filesystem has been truncated in some way. This is by far the most likely reason for the error.

So you need to check if the Squashfs filesystem image is truncated or corrupted in some way. This could obviously have happened before writing to the flash, during writing or afterwards. It could also be being truncated at read time. The cause could be faulty hardware or software at any point in the I/O path, at any point in the processs.

So, you need to double check everything at each of the above stages.

1. Check the Squashfs filesystem for correctness before writing it to the flash. You can run Unsquashfs on the image and see if it reports any errors.

2. You need to check the filesystem for integrity after writing it to the flash. Compute a checksum, and compare it with the original checksum.

In that way you can pinpoint the cause of the truncation/corruption. But, this is unlikely to be a Squashfs issue, and more likely truncation/corruption caused by something else.

[plougher]

This was followed up on LKML (Linux kernel mailing list), and the cause of the issue was found.