Fio test trigger ext4 kernel panic

[heyg]

Hi Guys,

When I use fio to read/write 4 partitions (ufs), this triggers an ext4 kernel panic. This needs to be tested for more than 24 hours for this to happen, does anyone have any good suggestions?

panic log,

[64816.118051] ------------[ cut here ]------------
[64816.118057] kernel BUG at fs/ext4/inode.c:547!
[64817.122963] Internal error: Oops - BUG: 0 [#1] PREEMPT_RT SMP
[64817.125331] Modules linked in: xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xt_addrtype iptable_filter iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c br_netfilter can_raw can 8021q garp mrp aes_ce_blk crypto_simd cryptd...
[64817.136572] CPU: 4 PID: 4174918 Comm: kworker/u24:6 Tainted: G O 5.10.120-rt70-tegra #1
[64817.137962] Hardware name: p3663-0001 (DT)
[64817.138576] Workqueue: writeback wb_workfn (flush-253:1408)
[64817.139399] pstate: 40c00005 (nZcv daif +PAN +UAO -TCO BTYPE=--)
[64817.140281] pc : ext4_map_blocks+0x5d4/0x5e0
[64817.140909] lr : ext4_map_blocks+0x7c/0x5e0
[64817.141530] sp : ffff80003628b800
[64817.142041] x29: ffff80003628b800 x28: 0000000000000009
[64817.142833] x27: ffff000097356630 x26: 0000000000000001
[64817.143627] x25: ffff0000b0170000 x24: ffff0000df9cfb28
[64817.144402] x23: ffff0000b0170000 x22: 000000000000042b
[64817.145190] x21: ffff0000b9beb408 x20: ffff0000b9beb408
[64817.145978] x19: ffff80003628b9a8 x18: 0000000000000002
[64817.146768] x17: 0000000000000000 x16: ffff800010615e80
[64817.147570] x15: ffff80001185a000 x14: 000000000000004e
[64817.148370] x13: ffff00003b813fc0 x12: 0000000000000000
[64817.149157] x11: ffffffffffffffc0 x10: 0000000000000040
[64817.149990] x9 : ffff80001126c9a4 x8 : ffff00009fa92720
[64817.150829] x7 : 0000000000000000 x6 : ffff0000b0170000
[64817.151630] x5 : ffff8002f7b4c000 x4 : ffff0000b0170000
[64817.152420] x3 : 0000000100000000 x2 : 0000000080000001
[64817.153246] x1 : 0000000000000000 x0 : 0000000000000000
[64817.154018] Call trace:
[64817.154379] ext4_map_blocks+0x5d4/0x5e0
[64817.155016] ext4_writepages+0x8d0/0xeb0
[64817.155635] do_writepages+0x5c/0x130
[64817.156196] __writeback_single_inode+0x4c/0x5b0
[64817.156873] writeback_sb_inodes+0x21c/0x4f0
[64817.157492] __writeback_inodes_wb+0x54/0x110
[64817.158145] wb_writeback+0x36c/0x480
[64817.158678] wb_workfn+0x324/0x6b0
[64817.159190] process_one_work+0x1c4/0x4f0
[64817.159829] worker_thread+0x54/0x430
[64817.160410] kthread+0x180/0x1c0
[64817.160879] ret_from_fork+0x10/0x34
[64817.161444] Code: a9046bf9 a90573fb 9439f905 a90573fb (d4210000)
[64817.162398] ---[ end trace 0000000000000002 ]---
[64818.167344] printk: enabled sync mode
[64818.174931] tegra_hv_vblk_oops tegra_virt_storage31: vblk_oops_write invoked in atomic context..aborting
[64818.176438] Kernel panic - not syncing:
[64818.177051] Oops - BUG: Fatal exception
[64818.177100] SMP: stopping secondary CPUs
[64818.178298] Kernel Offset: disabled
[64818.179042] CPU features: 0x081c0146,4800a238
[64818.179959] Memory Limit: none
[64818.186877] ---[ end Kernel panic - not syncing: Oops - BUG: Fatal exception ]---

[nathanchance]

Few questions to see if they help isolate things.

1. Does this happen with GCC?
2. I notice the kernel version is kind of old, 5.10.120 is over a year and a half old. Is it possible to update to the latest 5.10 release to see if there was anything that potentially fixed this?

[heyg]

Hi Nathan,

1 Does this happen with GCC?
I use aarch64-buildroot-linux-gnu, gcc version is 9.3.0

2 I notice the kernel version is kind of old, 5.10.120 is over a year and a half old. Is it possible to update to the latest 5.10 release to see if there was anything that potentially fixed this?

5.15 version also can repro this issue.

[nathanchance]

If this happens with GCC, it is probably worth reporting this upstream, as it is likely not a compiler issue (or at least one that does not just affect LLVM/clang).