Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

qemu boards: move away of tsc clock source to pit for qemu TCG #1683

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

tlaurion
Copy link
Collaborator

This PR overrides usage of tsc clocksource in qemu boards to use pit instead, which should be compatible with qemu TCG/KVM instead of breaking Qemu TCG (my usage for testing).

Otherwise Qemu TCG used with default OS'es tsc clock source results in:

[   30.260883] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 3703355526 wd_nsec: 3703355450
[   58.769175] watchdog: BUG: soft lockup - CPU#0 stuck for 46s! [(udev-worker):120]
[   58.770863] Modules linked in: crypto_simd cryptd
[   58.772911] CPU: 0 PID: 120 Comm: (udev-worker) Not tainted 6.8.0-31-generic #31-Ubuntu
[   58.773335] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS Heads-v0.2.0-2178-gcf08056 01/01/1970
[   58.774045] RIP: 0010:clear_page_rep+0x7/0x10
[   58.774393] Code: 5b 41 5c 5d 31 ff c3 cc cc cc cc cc cc cc cc cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab c3 cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90
[   58.775262] RSP: 0018:ffffaf6cc027fa98 EFLAGS: 00000246
[   58.775501] RAX: 0000000000000000 RBX: 0000000000000100 RCX: 00000000000001de
[   58.775958] RDX: fffff7aec414c000 RSI: fffff7aec4148b00 RDI: ffff932bc522c110
[   58.776205] RBP: ffffaf6cc027fac8 R08: 0000000000000000 R09: 0000000000000000
[   58.776442] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000004000
[   58.776680] R13: 0000000000000008 R14: fffff7aec4148000 R15: 0000000000000001
[   58.776958] FS:  00007c9dc13958c0(0000) GS:ffff932d3fc00000(0000) knlGS:0000000000000000
[   58.777274] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   58.777481] CR2: 00007c9dc13495de CR3: 00000001050f6000 CR4: 00000000000006f0
[   58.777982] Call Trace:
[   58.778786]  <IRQ>
[   58.779118]  ? show_regs+0x6d/0x80
[   58.779301]  ? watchdog_timer_fn+0x206/0x290
[   58.779414]  ? __pfx_watchdog_timer_fn+0x10/0x10
[   58.779528]  ? __hrtimer_run_queues+0x112/0x2a0
[   58.779682]  ? hrtimer_interrupt+0xf6/0x250
[   58.779818]  ? __sysvec_apic_timer_interrupt+0x51/0x150
[   58.779957]  ? sysvec_apic_timer_interrupt+0x8d/0xd0
[   58.780110]  </IRQ>
[   58.780189]  <TASK>
[   58.780254]  ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
[   58.780429]  ? clear_page_rep+0x7/0x10
[   58.780534]  ? post_alloc_hook+0xcd/0x120
[   58.780688]  get_page_from_freelist+0x452/0x6d0
[   58.780943]  ? _copy_to_iter+0x34d/0x590
[   58.781122]  __alloc_pages+0x1e9/0x350
[   58.781342]  __kmalloc_large_node+0x75/0x130
[   58.781503]  __kmalloc_node+0x3ed/0x500
[   58.781677]  ? kvmalloc_node+0x5d/0x100
[   58.782303]  kvmalloc_node+0x5d/0x100
[   58.782770]  ? kvmalloc_node+0x5d/0x100
[   58.782931]  module_zstd_decompress+0xe1/0x2a0
[   58.783105]  module_decompress+0x37/0xc0
[   58.783236]  init_module_from_file+0xd0/0x100
[   58.783389]  idempotent_init_module+0x11c/0x2b0
[   58.783639]  __x64_sys_finit_module+0x64/0xd0
[   58.783933]  x64_sys_call+0x1d6e/0x25c0
[   58.784097]  do_syscall_64+0x7f/0x180
[   58.784229]  ? irqentry_exit_to_user_mode+0x7b/0x260
[   58.784400]  ? irqentry_exit+0x43/0x50
[   58.784543]  entry_SYSCALL_64_after_hwframe+0x73/0x7b
[   58.784887] RIP: 0033:0x7c9dc1b5a25d
[   58.785368] Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 8b bb 0d 00 f7 d8 64 89 01 48
[   58.786678] RSP: 002b:00007fff26524238 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[   58.787122] RAX: ffffffffffffffda RBX: 000058d8cd8cad50 RCX: 00007c9dc1b5a25d
[   58.787346] RDX: 0000000000000004 RSI: 00007c9dc1c9707d RDI: 000000000000001e
[   58.787556] RBP: 00007fff265242f0 R08: 0000000000000040 R09: 00007fff26524280
[   58.787777] R10: 00007c9dc1c36b20 R11: 0000000000000246 R12: 00007c9dc1c9707d
[   58.788043] R13: 0000000000020000 R14: 000058d8cd8de7b0 R15: 000058d8cd8df5d0
[   58.788427]  </TASK>
[   73.194850] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[   73.195398] rcu: 	(detected by 0, t=60952 jiffies, g=7897, q=1 ncpus=1)
[   73.195631] rcu: All QSes seen, last rcu_preempt kthread activity 60952 (4294739946-4294678994), jiffies_till_next_fqs=3, root ->qsmask 0x0
[   73.196080] rcu: rcu_preempt kthread starved for 60952 jiffies! g7897 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0
[   73.196327] rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
[   73.196524] rcu: RCU grace-period kthread stack dump:
[   73.196756] task:rcu_preempt     state:R  running task     stack:0     pid:17    tgid:17    ppid:2      flags:0x00004000
[   73.197272] Call Trace:
[   73.197400]  <TASK>
[   73.197493]  __schedule+0x27c/0x6b0
[   73.197617]  ? __pfx_rcu_gp_kthread+0x10/0x10
[   73.198112]  schedule+0x33/0x110
[   73.198444]  schedule_timeout+0x95/0x170
[   73.198901]  ? __pfx_process_timeout+0x10/0x10
[   73.199289]  rcu_gp_fqs_loop+0x105/0x580
[   73.199789]  rcu_gp_kthread+0xea/0x180
[   73.200291]  kthread+0xf2/0x120
[   73.200661]  ? __pfx_kthread+0x10/0x10
[   73.201057]  ret_from_fork+0x47/0x70
[   73.201462]  ? __pfx_kthread+0x10/0x10
[   73.201922]  ret_from_fork_asm+0x1b/0x30
[   73.202476]  </TASK>
[   73.202945] rcu: Stack dump where RCU GP kthread last ran:
[   73.203783] CPU: 0 PID: 120 Comm: (udev-worker) Tainted: G             L     6.8.0-31-generic #31-Ubuntu
[   73.204164] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS Heads-v0.2.0-2178-gcf08056 01/01/1970
[   73.204430] RIP: 0010:clear_page_rep+0x7/0x10
[   73.204632] Code: 5b 41 5c 5d 31 ff c3 cc cc cc cc cc cc cc cc cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab c3 cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90
[   73.205404] RSP: 0018:ffffaf6cc027fa98 EFLAGS: 00000246
[   73.205587] RAX: 0000000000000000 RBX: 0000000000000100 RCX: 00000000000001d3
[   73.205792] RDX: fffff7aec414c000 RSI: fffff7aec4148b00 RDI: ffff932bc522c168
[   73.205961] RBP: ffffaf6cc027fac8 R08: 0000000000000000 R09: 0000000000000000
[   73.206138] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000004000
[   73.206331] R13: 0000000000000008 R14: fffff7aec4148000 R15: 0000000000000001
[   73.206531] FS:  00007c9dc13958c0(0000) GS:ffff932d3fc00000(0000) knlGS:0000000000000000
[   73.206757] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   73.206901] CR2: 00007c9dc13495de CR3: 00000001050f6000 CR4: 00000000000006f0
[   73.207074] Call Trace:
[   73.207150]  <IRQ>
[   73.207220]  ? show_regs+0x6d/0x80
[   73.207328]  ? dump_cpu_task+0x77/0x90
[   73.207457]  ? rcu_check_gp_kthread_starvation+0x1ce/0x280
[   73.207730]  ? print_other_cpu_stall+0x2a3/0x580
[   73.207954]  ? check_cpu_stall+0x1ca/0x230
[   73.208129]  ? rcu_pending+0x32/0x1f0
[   73.208310]  ? account_system_index_time+0x9a/0xd0
[   73.208519]  ? rcu_sched_clock_irq+0xd5/0x3c0
[   73.208756]  ? update_process_times+0x76/0xb0
[   73.209042]  ? tick_sched_handle+0x28/0x70
[   73.209190]  ? tick_nohz_highres_handler+0x78/0xa0
[   73.209349]  ? __pfx_tick_nohz_highres_handler+0x10/0x10
[   73.209528]  ? __hrtimer_run_queues+0x112/0x2a0
[   73.209770]  ? hrtimer_interrupt+0xf6/0x250
[   73.210281]  ? __sysvec_apic_timer_interrupt+0x51/0x150
[   73.210943]  ? sysvec_apic_timer_interrupt+0x8d/0xd0
[   73.211495]  </IRQ>
[   73.211703]  <TASK>
[   73.211791]  ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
[   73.211977]  ? clear_page_rep+0x7/0x10
[   73.212109]  ? post_alloc_hook+0xcd/0x120
[   73.212243]  get_page_from_freelist+0x452/0x6d0
[   73.212374]  ? _copy_to_iter+0x34d/0x590
[   73.212502]  __alloc_pages+0x1e9/0x350
[   73.212626]  __kmalloc_large_node+0x75/0x130
[   73.212826]  __kmalloc_node+0x3ed/0x500
[   73.212999]  ? kvmalloc_node+0x5d/0x100
[   73.213181]  kvmalloc_node+0x5d/0x100
[   73.213378]  ? kvmalloc_node+0x5d/0x100
[   73.213575]  module_zstd_decompress+0xe1/0x2a0
[   73.213765]  module_decompress+0x37/0xc0
[   73.213905]  init_module_from_file+0xd0/0x100
[   73.214063]  idempotent_init_module+0x11c/0x2b0
[   73.214224]  __x64_sys_finit_module+0x64/0xd0
[   73.214444]  x64_sys_call+0x1d6e/0x25c0
[   73.214604]  do_syscall_64+0x7f/0x180
[   73.214978]  ? irqentry_exit_to_user_mode+0x7b/0x260
[   73.215176]  ? irqentry_exit+0x43/0x50
[   73.215322]  entry_SYSCALL_64_after_hwframe+0x73/0x7b
[   73.215493] RIP: 0033:0x7c9dc1b5a25d
[   73.215673] Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 8b bb 0d 00 f7 d8 64 89 01 48
[   73.216258] RSP: 002b:00007fff26524238 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[   73.216505] RAX: ffffffffffffffda RBX: 000058d8cd8cad50 RCX: 00007c9dc1b5a25d
[   73.216717] RDX: 0000000000000004 RSI: 00007c9dc1c9707d RDI: 000000000000001e
[   73.217042] RBP: 00007fff265242f0 R08: 0000000000000040 R09: 00007fff26524280
[   73.217410] R10: 00007c9dc1c36b20 R11: 0000000000000246 R12: 00007c9dc1c9707d
[   73.217696] R13: 0000000000020000 R14: 000058d8cd8de7b0 R15: 000058d8cd8df5d0
[   73.217966]  </TASK>
[   98.119999] watchdog: BUG: soft lockup - CPU#0 stuck for 83s! [(udev-worker):120]
[   98.120326] Modules linked in: crypto_simd cryptd
[   98.120494] CPU: 0 PID: 120 Comm: (udev-worker) Tainted: G             L     6.8.0-31-generic #31-Ubuntu
[   98.120796] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS Heads-v0.2.0-2178-gcf08056 01/01/1970
[   98.121190] RIP: 0010:clear_page_rep+0x7/0x10
[   98.121387] Code: 5b 41 5c 5d 31 ff c3 cc cc cc cc cc cc cc cc cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab c3 cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90
[   98.122079] RSP: 0018:ffffaf6cc027fa98 EFLAGS: 00000246
[   98.122280] RAX: 0000000000000000 RBX: 0000000000000100 RCX: 000000000000019d
[   98.122517] RDX: fffff7aec414c000 RSI: fffff7aec4148b00 RDI: ffff932bc522c318
[   98.122844] RBP: ffffaf6cc027fac8 R08: 0000000000000000 R09: 0000000000000000
[   98.123143] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000004000
[   98.123373] R13: 0000000000000008 R14: fffff7aec4148000 R15: 0000000000000001
[   98.123629] FS:  00007c9dc13958c0(0000) GS:ffff932d3fc00000(0000) knlGS:0000000000000000
[   98.124009] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   98.124366] CR2: 00007c9dc13495de CR3: 00000001050f6000 CR4: 00000000000006f0
[   98.124616] Call Trace:
[   98.124744]  <IRQ>
[   98.124848]  ? show_regs+0x6d/0x80
[   98.125027]  ? watchdog_timer_fn+0x206/0x290
[   98.125351]  ? __pfx_watchdog_timer_fn+0x10/0x10
[   98.125549]  ? __hrtimer_run_queues+0x112/0x2a0
[   98.125738]  ? hrtimer_interrupt+0xf6/0x250
[   98.125931]  ? __sysvec_apic_timer_interrupt+0x51/0x150
[   98.126227]  ? sysvec_apic_timer_interrupt+0x8d/0xd0
[   98.126406]  </IRQ>
[   98.126490]  <TASK>
[   98.126574]  ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
[   98.127177]  ? clear_page_rep+0x7/0x10
[   98.127842]  ? post_alloc_hook+0xcd/0x120
[   98.128038]  get_page_from_freelist+0x452/0x6d0
[   98.128196]  ? _copy_to_iter+0x34d/0x590
[   98.128337]  __alloc_pages+0x1e9/0x350
[   98.128467]  __kmalloc_large_node+0x75/0x130
[   98.128717]  __kmalloc_node+0x3ed/0x500
[   98.128882]  ? kvmalloc_node+0x5d/0x100
[   98.129044]  kvmalloc_node+0x5d/0x100
[   98.129175]  ? kvmalloc_node+0x5d/0x100
[   98.129292]  module_zstd_decompress+0xe1/0x2a0
[   98.129431]  module_decompress+0x37/0xc0
[   98.129549]  init_module_from_file+0xd0/0x100
[   98.129771]  idempotent_init_module+0x11c/0x2b0
[   98.130078]  __x64_sys_finit_module+0x64/0xd0
[   98.130252]  x64_sys_call+0x1d6e/0x25c0
[   98.130404]  do_syscall_64+0x7f/0x180
[   98.130541]  ? irqentry_exit_to_user_mode+0x7b/0x260
[   98.130840]  ? irqentry_exit+0x43/0x50
[   98.131054]  entry_SYSCALL_64_after_hwframe+0x73/0x7b
[   98.131294] RIP: 0033:0x7c9dc1b5a25d
[   98.131574] Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 8b bb 0d 00 f7 d8 64 89 01 48
[   98.132318] RSP: 002b:00007fff26524238 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[   98.132978] RAX: ffffffffffffffda RBX: 000058d8cd8cad50 RCX: 00007c9dc1b5a25d
[   98.133312] RDX: 0000000000000004 RSI: 00007c9dc1c9707d RDI: 000000000000001e
[   98.133603] RBP: 00007fff265242f0 R08: 0000000000000040 R09: 00007fff26524280
[   98.134005] R10: 00007c9dc1c36b20 R11: 0000000000000246 R12: 00007c9dc1c9707d
[   98.134323] R13: 0000000000020000 R14: 000058d8cd8de7b0 R15: 000058d8cd8df5d0
[   98.134846]  </TASK>
[  125.509373] watchdog: BUG: soft lockup - CPU#0 stuck for 108s! [(udev-worker):120]
[  125.509705] Modules linked in: crypto_simd cryptd
[  125.509908] CPU: 0 PID: 120 Comm: (udev-worker) Tainted: G             L     6.8.0-31-generic #31-Ubuntu
[  125.510225] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS Heads-v0.2.0-2178-gcf08056 01/01/1970
[  125.510678] RIP: 0010:clear_page_rep+0x7/0x10
[  125.510921] Code: 5b 41 5c 5d 31 ff c3 cc cc cc cc cc cc cc cc cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab c3 cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90
[  125.511613] RSP: 0018:ffffaf6cc027fa98 EFLAGS: 00000246
[  125.511827] RAX: 0000000000000000 RBX: 0000000000000100 RCX: 0000000000000044
[  125.512017] RDX: fffff7aec414c000 RSI: fffff7aec4148b00 RDI: ffff932bc522cde0
[  125.512247] RBP: ffffaf6cc027fac8 R08: 0000000000000000 R09: 0000000000000000
[  125.512460] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000004000
[  125.512733] R13: 0000000000000008 R14: fffff7aec4148000 R15: 0000000000000001
[  125.513557] FS:  00007c9dc13958c0(0000) GS:ffff932d3fc00000(0000) knlGS:0000000000000000
[  125.514233] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  125.514425] CR2: 00007c9dc13495de CR3: 00000001050f6000 CR4: 00000000000006f0
[  125.514628] Call Trace:
[  125.514732]  <IRQ>
[  125.514867]  ? show_regs+0x6d/0x80
[  125.515055]  ? watchdog_timer_fn+0x206/0x290
[  125.515221]  ? __pfx_watchdog_timer_fn+0x10/0x10
[  125.515381]  ? __hrtimer_run_queues+0x112/0x2a0
[  125.515551]  ? hrtimer_interrupt+0xf6/0x250
[  125.515738]  ? __sysvec_apic_timer_interrupt+0x51/0x150
[  125.515967]  ? sysvec_apic_timer_interrupt+0x8d/0xd0
[  125.516144]  </IRQ>
[  125.516223]  <TASK>
[  125.516300]  ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
[  125.516474]  ? clear_page_rep+0x7/0x10
[  125.516701]  ? post_alloc_hook+0xcd/0x120
[  125.516929]  get_page_from_freelist+0x452/0x6d0
[  125.517146]  ? _copy_to_iter+0x34d/0x590
[  125.517345]  __alloc_pages+0x1e9/0x350
[  125.517540]  __kmalloc_large_node+0x75/0x130
[  125.517739]  __kmalloc_node+0x3ed/0x500
[  125.517907]  ? kvmalloc_node+0x5d/0x100
[  125.518060]  kvmalloc_node+0x5d/0x100
[  125.518195]  ? kvmalloc_node+0x5d/0x100
[  125.518317]  module_zstd_decompress+0xe1/0x2a0
[  125.518460]  module_decompress+0x37/0xc0
[  125.518587]  init_module_from_file+0xd0/0x100
[  125.518746]  idempotent_init_module+0x11c/0x2b0
[  125.518896]  __x64_sys_finit_module+0x64/0xd0
[  125.519026]  x64_sys_call+0x1d6e/0x25c0
[  125.519133]  do_syscall_64+0x7f/0x180
[  125.519238]  ? irqentry_exit_to_user_mode+0x7b/0x260
[  125.519370]  ? irqentry_exit+0x43/0x50
[  125.519557]  entry_SYSCALL_64_after_hwframe+0x73/0x7b
[  125.519798] RIP: 0033:0x7c9dc1b5a25d
[  125.519956] Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 8b bb 0d 00 f7 d8 64 89 01 48
[  125.520425] RSP: 002b:00007fff26524238 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[  125.520651] RAX: ffffffffffffffda RBX: 000058d8cd8cad50 RCX: 00007c9dc1b5a25d
[  125.520842] RDX: 0000000000000004 RSI: 00007c9dc1c9707d RDI: 000000000000001e
[  125.521029] RBP: 00007fff265242f0 R08: 0000000000000040 R09: 00007fff26524280
[  125.521207] R10: 00007c9dc1c36b20 R11: 0000000000000246 R12: 00007c9dc1c9707d
[  125.521383] R13: 0000000000020000 R14: 000058d8cd8de7b0 R15: 000058d8cd8df5d0
[  125.521571]  </TASK>
[  153.925148] watchdog: BUG: soft lockup - CPU#0 stuck for 135s! [(udev-worker):120]
[  153.925494] Modules linked in: crypto_simd cryptd
[  153.926140] CPU: 0 PID: 120 Comm: (udev-worker) Tainted: G             L     6.8.0-31-generic #31-Ubuntu
[  153.926817] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS Heads-v0.2.0-2178-gcf08056 01/01/1970
[  153.927150] RIP: 0010:clear_page_rep+0x7/0x10
[  153.927386] Code: 5b 41 5c 5d 31 ff c3 cc cc cc cc cc cc cc cc cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab c3 cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90
[  153.928196] RSP: 0018:ffffaf6cc027fa98 EFLAGS: 00000246
[  153.928445] RAX: 0000000000000000 RBX: 0000000000000100 RCX: 00000000000001c9
[  153.928812] RDX: fffff7aec414c000 RSI: fffff7aec4148b40 RDI: ffff932bc522d1b8
[  153.929065] RBP: ffffaf6cc027fac8 R08: 0000000000000000 R09: 0000000000000000
[  153.929296] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000004000
[  153.929559] R13: 0000000000000008 R14: fffff7aec4148000 R15: 0000000000000001
[  153.929793] FS:  00007c9dc13958c0(0000) GS:ffff932d3fc00000(0000) knlGS:0000000000000000
[  153.930085] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  153.930279] CR2: 00007c9dc13495de CR3: 00000001050f6000 CR4: 00000000000006f0
[  153.930566] Call Trace:
[  153.930749]  <IRQ>
[  153.930938]  ? show_regs+0x6d/0x80
[  153.931104]  ? watchdog_timer_fn+0x206/0x290
[  153.931256]  ? __pfx_watchdog_timer_fn+0x10/0x10
[  153.931436]  ? __hrtimer_run_queues+0x112/0x2a0
[  153.931801]  ? hrtimer_interrupt+0xf6/0x250
[  153.932033]  ? __sysvec_apic_timer_interrupt+0x51/0x150
[  153.932229]  ? sysvec_apic_timer_interrupt+0x8d/0xd0
[  153.932404]  </IRQ>
[  153.932501]  <TASK>
[  153.932628]  ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
[  153.933064]  ? clear_page_rep+0x7/0x10
[  153.933216]  ? post_alloc_hook+0xcd/0x120
[  153.933446]  get_page_from_freelist+0x452/0x6d0
[  153.933726]  ? _copy_to_iter+0x34d/0x590
[  153.933889]  __alloc_pages+0x1e9/0x350
[  153.934027]  __kmalloc_large_node+0x75/0x130
[  153.934163]  __kmalloc_node+0x3ed/0x500
[  153.934283]  ? kvmalloc_node+0x5d/0x100
[  153.934415]  kvmalloc_node+0x5d/0x100
[  153.934535]  ? kvmalloc_node+0x5d/0x100
[  153.934769]  module_zstd_decompress+0xe1/0x2a0
[  153.935014]  module_decompress+0x37/0xc0
[  153.935237]  init_module_from_file+0xd0/0x100
[  153.935481]  idempotent_init_module+0x11c/0x2b0
[  153.935714]  __x64_sys_finit_module+0x64/0xd0
[  153.935890]  x64_sys_call+0x1d6e/0x25c0
[  153.936039]  do_syscall_64+0x7f/0x180
[  153.936177]  ? irqentry_exit_to_user_mode+0x7b/0x260
[  153.936357]  ? irqentry_exit+0x43/0x50
[  153.936505]  entry_SYSCALL_64_after_hwframe+0x73/0x7b
[  153.936953] RIP: 0033:0x7c9dc1b5a25d
[  153.937358] Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 8b bb 0d 00 f7 d8 64 89 01 48
[  153.939607] RSP: 002b:00007fff26524238 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[  153.940385] RAX: ffffffffffffffda RBX: 000058d8cd8cad50 RCX: 00007c9dc1b5a25d
[  153.940693] RDX: 0000000000000004 RSI: 00007c9dc1c9707d RDI: 000000000000001e
[  153.940909] RBP: 00007fff265242f0 R08: 0000000000000040 R09: 00007fff26524280
[  153.941105] R10: 00007c9dc1c36b20 R11: 0000000000000246 R12: 00007c9dc1c9707d
[  153.941298] R13: 0000000000020000 R14: 000058d8cd8de7b0 R15: 000058d8cd8df5d0
[  153.941555]  </TASK>
[  181.553279] watchdog: BUG: soft lockup - CPU#0 stuck for 161s! [(udev-worker):120]
[  181.554138] Modules linked in: crypto_simd cryptd
[  181.554739] CPU: 0 PID: 120 Comm: (udev-worker) Tainted: G             L     6.8.0-31-generic #31-Ubuntu
[  181.555940] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS Heads-v0.2.0-2178-gcf08056 01/01/1970
[  181.556396] RIP: 0010:clear_page_rep+0x7/0x10
[  181.556580] Code: 5b 41 5c 5d 31 ff c3 cc cc cc cc cc cc cc cc cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab c3 cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90
[  181.557167] RSP: 0018:ffffaf6cc027fa98 EFLAGS: 00000246
[  181.557350] RAX: 0000000000000000 RBX: 0000000000000100 RCX: 00000000000001fc
[  181.557558] RDX: fffff7aec414c000 RSI: fffff7aec4148b80 RDI: ffff932bc522e020
[  181.558219] RBP: ffffaf6cc027fac8 R08: 0000000000000000 R09: 0000000000000000
[  181.559037] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000004000
[  181.559249] R13: 0000000000000008 R14: fffff7aec4148000 R15: 0000000000000001
[  181.559448] FS:  00007c9dc13958c0(0000) GS:ffff932d3fc00000(0000) knlGS:0000000000000000
[  181.559753] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  181.559968] CR2: 00007c9dc13495de CR3: 00000001050f6000 CR4: 00000000000006f0
[  181.560208] Call Trace:
[  181.560310]  <IRQ>
[  181.560407]  ? show_regs+0x6d/0x80
[  181.560530]  ? watchdog_timer_fn+0x206/0x290
[  181.560677]  ? __pfx_watchdog_timer_fn+0x10/0x10
[  181.560852]  ? __hrtimer_run_queues+0x112/0x2a0
[  181.561074]  ? hrtimer_interrupt+0xf6/0x250
[  181.561304]  ? __sysvec_apic_timer_interrupt+0x51/0x150
[  181.561514]  ? sysvec_apic_timer_interrupt+0x8d/0xd0
[  181.561693]  </IRQ>
[  181.561769]  <TASK>
[  181.561840]  ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
[  181.561995]  ? clear_page_rep+0x7/0x10
[  181.562095]  ? post_alloc_hook+0xcd/0x120
[  181.562204]  get_page_from_freelist+0x452/0x6d0
[  181.562316]  ? _copy_to_iter+0x34d/0x590
[  181.562426]  __alloc_pages+0x1e9/0x350
[  181.562530]  __kmalloc_large_node+0x75/0x130
[  181.562656]  __kmalloc_node+0x3ed/0x500
[  181.562811]  ? kvmalloc_node+0x5d/0x100
[  181.563042]  kvmalloc_node+0x5d/0x100
[  181.563201]  ? kvmalloc_node+0x5d/0x100
[  181.563416]  module_zstd_decompress+0xe1/0x2a0
[  181.563606]  module_decompress+0x37/0xc0
[  181.563782]  init_module_from_file+0xd0/0x100
[  181.563951]  idempotent_init_module+0x11c/0x2b0
[  181.564098]  __x64_sys_finit_module+0x64/0xd0
[  181.564226]  x64_sys_call+0x1d6e/0x25c0
[  181.564340]  do_syscall_64+0x7f/0x180
[  181.564451]  ? irqentry_exit_to_user_mode+0x7b/0x260
[  181.564594]  ? irqentry_exit+0x43/0x50
[  181.565072]  entry_SYSCALL_64_after_hwframe+0x73/0x7b
[  181.565796] RIP: 0033:0x7c9dc1b5a25d
[  181.566259] Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 8b bb 0d 00 f7 d8 64 89 01 48
[  181.567956] RSP: 002b:00007fff26524238 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[  181.568189] RAX: ffffffffffffffda RBX: 000058d8cd8cad50 RCX: 00007c9dc1b5a25d
[  181.568358] RDX: 0000000000000004 RSI: 00007c9dc1c9707d RDI: 000000000000001e
[  181.568510] RBP: 00007fff265242f0 R08: 0000000000000040 R09: 00007fff26524280
[  181.568687] R10: 00007c9dc1c36b20 R11: 0000000000000246 R12: 00007c9dc1c9707d
[  181.568936] R13: 0000000000020000 R14: 000058d8cd8de7b0 R15: 000058d8cd8df5d0
[  181.569176]  </TASK>
[  214.072068] watchdog: BUG: soft lockup - CPU#0 stuck for 191s! [(udev-worker):120]
[  214.072405] Modules linked in: crypto_simd cryptd
[  214.072671] CPU: 0 PID: 120 Comm: (udev-worker) Tainted: G             L     6.8.0-31-generic #31-Ubuntu
[  214.073211] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS Heads-v0.2.0-2178-gcf08056 01/01/1970
[  214.073504] RIP: 0010:clear_page_rep+0x7/0x10
[  214.073697] Code: 5b 41 5c 5d 31 ff c3 cc cc cc cc cc cc cc cc cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab c3 cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90
[  214.074245] RSP: 0018:ffffaf6cc027fa98 EFLAGS: 00000246
[  214.074459] RAX: 0000000000000000 RBX: 0000000000000100 RCX: 00000000000001d8
[  214.074774] RDX: fffff7aec414c000 RSI: fffff7aec4148b80 RDI: ffff932bc522e140
[  214.075107] RBP: ffffaf6cc027fac8 R08: 0000000000000000 R09: 0000000000000000
[  214.075298] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000004000
[  214.075481] R13: 0000000000000008 R14: fffff7aec4148000 R15: 0000000000000001
[  214.075747] FS:  00007c9dc13958c0(0000) GS:ffff932d3fc00000(0000) knlGS:0000000000000000
[  214.076045] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  214.076215] CR2: 00007c9dc13495de CR3: 00000001050f6000 CR4: 00000000000006f0
[  214.076403] Call Trace:
[  214.076550]  <IRQ>
[  214.076692]  ? show_regs+0x6d/0x80
[  214.076867]  ? watchdog_timer_fn+0x206/0x290
[  214.077067]  ? __pfx_watchdog_timer_fn+0x10/0x10
[  214.077245]  ? __hrtimer_run_queues+0x112/0x2a0
[  214.077390]  ? clockevents_program_event+0xc1/0x150
[  214.077550]  ? hrtimer_interrupt+0xf6/0x250
[  214.077720]  ? __sysvec_apic_timer_interrupt+0x51/0x150
[  214.078002]  ? sysvec_apic_timer_interrupt+0x8d/0xd0
[  214.078233]  </IRQ>
[  214.078345]  <TASK>
[  214.078442]  ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
[  214.078697]  ? clear_page_rep+0x7/0x10
[  214.078849]  ? post_alloc_hook+0xcd/0x120
[  214.079012]  get_page_from_freelist+0x452/0x6d0
[  214.079153]  ? _copy_to_iter+0x34d/0x590
[  214.079275]  __alloc_pages+0x1e9/0x350
[  214.079381]  __kmalloc_large_node+0x75/0x130
[  214.079491]  __kmalloc_node+0x3ed/0x500
[  214.079655]  ? kvmalloc_node+0x5d/0x100
[  214.079823]  kvmalloc_node+0x5d/0x100
[  214.079967]  ? kvmalloc_node+0x5d/0x100
[  214.080106]  module_zstd_decompress+0xe1/0x2a0
[  214.080248]  module_decompress+0x37/0xc0
[  214.080430]  init_module_from_file+0xd0/0x100
[  214.080692]  idempotent_init_module+0x11c/0x2b0
[  214.080886]  __x64_sys_finit_module+0x64/0xd0
[  214.081088]  x64_sys_call+0x1d6e/0x25c0
[  214.081230]  do_syscall_64+0x7f/0x180
[  214.081388]  ? irqentry_exit_to_user_mode+0x7b/0x260
[  214.081647]  ? irqentry_exit+0x43/0x50
[  214.081852]  entry_SYSCALL_64_after_hwframe+0x73/0x7b
[  214.082090] RIP: 0033:0x7c9dc1b5a25d
[  214.082241] Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 8b bb 0d 00 f7 d8 64 89 01 48
[  214.083087] RSP: 002b:00007fff26524238 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[  214.083690] RAX: ffffffffffffffda RBX: 000058d8cd8cad50 RCX: 00007c9dc1b5a25d
[  214.084173] RDX: 0000000000000004 RSI: 00007c9dc1c9707d RDI: 000000000000001e
[  214.084711] RBP: 00007fff265242f0 R08: 0000000000000040 R09: 00007fff26524280
[  214.085111] R10: 00007c9dc1c36b20 R11: 0000000000000246 R12: 00007c9dc1c9707d
[  214.085312] R13: 0000000000020000 R14: 000058d8cd8de7b0 R15: 000058d8cd8df5d0
[  214.085524]  </TASK>
[  242.577519] watchdog: BUG: soft lockup - CPU#0 stuck for 217s! [(udev-worker):120]
[  242.578518] Modules linked in: crypto_simd cryptd
[  242.579070] CPU: 0 PID: 120 Comm: (udev-worker) Tainted: G             L     6.8.0-31-generic #31-Ubuntu
[  242.580110] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS Heads-v0.2.0-2178-gcf08056 01/01/1970
[  242.580672] RIP: 0010:clear_page_rep+0x7/0x10
[  242.580853] Code: 5b 41 5c 5d 31 ff c3 cc cc cc cc cc cc cc cc cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab c3 cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90
[  242.581453] RSP: 0018:ffffaf6cc027fa98 EFLAGS: 00000246
[  242.581659] RAX: 0000000000000000 RBX: 0000000000000100 RCX: 00000000000001af
[  242.581935] RDX: fffff7aec414c000 RSI: fffff7aec4148b80 RDI: ffff932bc522e288
[  242.582163] RBP: ffffaf6cc027fac8 R08: 0000000000000000 R09: 0000000000000000
[  242.582401] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000004000
[  242.582709] R13: 0000000000000008 R14: fffff7aec4148000 R15: 0000000000000001
[  242.582944] FS:  00007c9dc13958c0(0000) GS:ffff932d3fc00000(0000) knlGS:0000000000000000
[  242.583291] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  242.583484] CR2: 00007c9dc13495de CR3: 00000001050f6000 CR4: 00000000000006f0
[  242.583702] Call Trace:
[  242.583796]  <IRQ>
[  242.583911]  ? show_regs+0x6d/0x80
[  242.584076]  ? watchdog_timer_fn+0x206/0x290
[  242.584225]  ? __pfx_watchdog_timer_fn+0x10/0x10
[  242.584384]  ? __hrtimer_run_queues+0x112/0x2a0
[  242.584583]  ? hrtimer_interrupt+0xf6/0x250
[  242.584761]  ? __sysvec_apic_timer_interrupt+0x51/0x150
[  242.584944]  ? sysvec_apic_timer_interrupt+0x8d/0xd0
[  242.585099]  </IRQ>
[  242.585170]  <TASK>
[  242.585247]  ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
[  242.585410]  ? clear_page_rep+0x7/0x10
[  242.585545]  ? post_alloc_hook+0xcd/0x120
[  242.585695]  get_page_from_freelist+0x452/0x6d0
[  242.585834]  ? _copy_to_iter+0x34d/0x590
[  242.585965]  __alloc_pages+0x1e9/0x350
[  242.586099]  __kmalloc_large_node+0x75/0x130
[  242.586309]  __kmalloc_node+0x3ed/0x500
[  242.586499]  ? kvmalloc_node+0x5d/0x100
[  242.586699]  kvmalloc_node+0x5d/0x100
[  242.586829]  ? kvmalloc_node+0x5d/0x100
[  242.586955]  module_zstd_decompress+0xe1/0x2a0
[  242.587102]  module_decompress+0x37/0xc0
[  242.587205]  init_module_from_file+0xd0/0x100
[  242.587333]  idempotent_init_module+0x11c/0x2b0
[  242.587501]  __x64_sys_finit_module+0x64/0xd0
[  242.587685]  x64_sys_call+0x1d6e/0x25c0
[  242.587974]  do_syscall_64+0x7f/0x180
[  242.588098]  ? irqentry_exit_to_user_mode+0x7b/0x260
[  242.588238]  ? irqentry_exit+0x43/0x50
[  242.588358]  entry_SYSCALL_64_after_hwframe+0x73/0x7b
[  242.588504] RIP: 0033:0x7c9dc1b5a25d
[  242.588617] Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 8b bb 0d 00 f7 d8 64 89 01 48
[  242.589047] RSP: 002b:00007fff26524238 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[  242.589230] RAX: ffffffffffffffda RBX: 000058d8cd8cad50 RCX: 00007c9dc1b5a25d
[  242.589395] RDX: 0000000000000004 RSI: 00007c9dc1c9707d RDI: 000000000000001e
[  242.589612] RBP: 00007fff265242f0 R08: 0000000000000040 R09: 00007fff26524280
[  242.589973] R10: 00007c9dc1c36b20 R11: 0000000000000246 R12: 00007c9dc1c9707d
[  242.590316] R13: 0000000000020000 R14: 000058d8cd8de7b0 R15: 000058d8cd8df5d0
[  242.590553]  </TASK>
[  253.341960] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[  253.342313] rcu: 	(detected by 0, t=241099 jiffies, g=7897, q=1 ncpus=1)
[  253.342526] rcu: All QSes seen, last rcu_preempt kthread activity 241099 (4294920093-4294678994), jiffies_till_next_fqs=3, root ->qsmask 0x0
[  253.342971] rcu: rcu_preempt kthread starved for 241099 jiffies! g7897 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0
[  253.343307] rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
[  253.343542] rcu: RCU grace-period kthread stack dump:
[  253.343754] task:rcu_preempt     state:R  running task     stack:0     pid:17    tgid:17    ppid:2      flags:0x00004000
[  253.344138] Call Trace:
[  253.344241]  <TASK>
[  253.344341]  __schedule+0x27c/0x6b0
[  253.344551]  ? __pfx_rcu_gp_kthread+0x10/0x10
[  253.344783]  schedule+0x33/0x110
[  253.344967]  schedule_timeout+0x95/0x170
[  253.345110]  ? __pfx_process_timeout+0x10/0x10
[  253.345249]  rcu_gp_fqs_loop+0x105/0x580
[  253.345376]  rcu_gp_kthread+0xea/0x180
[  253.345558]  kthread+0xf2/0x120
[  253.345700]  ? __pfx_kthread+0x10/0x10
[  253.345820]  ret_from_fork+0x47/0x70
[  253.345927]  ? __pfx_kthread+0x10/0x10
[  253.346033]  ret_from_fork_asm+0x1b/0x30
[  253.346190]  </TASK>
[  253.346310] rcu: Stack dump where RCU GP kthread last ran:
[  253.346571] CPU: 0 PID: 120 Comm: (udev-worker) Tainted: G             L     6.8.0-31-generic #31-Ubuntu
[  253.346979] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS Heads-v0.2.0-2178-gcf08056 01/01/1970
[  253.347306] RIP: 0010:clear_page_rep+0x7/0x10
[  253.347481] Code: 5b 41 5c 5d 31 ff c3 cc cc cc cc cc cc cc cc cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab c3 cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90
[  253.348260] RSP: 0018:ffffaf6cc027fa98 EFLAGS: 00000246
[  253.348657] RAX: 0000000000000000 RBX: 0000000000000100 RCX: 0000000000000198
[  253.349312] RDX: fffff7aec414c000 RSI: fffff7aec4148b80 RDI: ffff932bc522e340
[  253.349533] RBP: ffffaf6cc027fac8 R08: 0000000000000000 R09: 0000000000000000
[  253.349725] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000004000
[  253.349995] R13: 0000000000000008 R14: fffff7aec4148000 R15: 0000000000000001
[  253.350265] FS:  00007c9dc13958c0(0000) GS:ffff932d3fc00000(0000) knlGS:0000000000000000
[  253.350510] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  253.350681] CR2: 00007c9dc13495de CR3: 00000001050f6000 CR4: 00000000000006f0
[  253.350977] Call Trace:
[  253.351078]  <IRQ>
[  253.351158]  ? show_regs+0x6d/0x80
[  253.351276]  ? dump_cpu_task+0x77/0x90
[  253.351407]  ? rcu_check_gp_kthread_starvation+0x1ce/0x280
[  253.351581]  ? print_other_cpu_stall+0x2a3/0x580
[  253.352116]  ? check_cpu_stall+0x1ca/0x230
[  253.352597]  ? rcu_pending+0x32/0x1f0
[  253.352903]  ? account_system_index_time+0x9a/0xd0
[  253.353219]  ? rcu_sched_clock_irq+0xd5/0x3c0
[  253.353500]  ? update_process_times+0x76/0xb0
[  253.353847]  ? tick_sched_handle+0x28/0x70
[  253.354179]  ? tick_nohz_highres_handler+0x78/0xa0
[  253.354334]  ? __pfx_tick_nohz_highres_handler+0x10/0x10
[  253.354489]  ? __hrtimer_run_queues+0x112/0x2a0
[  253.354621]  ? clockevents_program_event+0xc1/0x150
[  253.354915]  ? hrtimer_interrupt+0xf6/0x250
[  253.355175]  ? __sysvec_apic_timer_interrupt+0x51/0x150
[  253.355479]  ? sysvec_apic_timer_interrupt+0x8d/0xd0
[  253.355704]  </IRQ>
[  253.355797]  <TASK>
[  253.355882]  ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
[  253.356071]  ? clear_page_rep+0x7/0x10
[  253.356194]  ? post_alloc_hook+0xcd/0x120
[  253.356331]  get_page_from_freelist+0x452/0x6d0
[  253.356475]  ? _copy_to_iter+0x34d/0x590
[  253.356612]  __alloc_pages+0x1e9/0x350
[  253.357109]  __kmalloc_large_node+0x75/0x130
[  253.357612]  __kmalloc_node+0x3ed/0x500
[  253.358092]  ? kvmalloc_node+0x5d/0x100
[  253.358662]  kvmalloc_node+0x5d/0x100
[  253.359345]  ? kvmalloc_node+0x5d/0x100
[  253.359962]  module_zstd_decompress+0xe1/0x2a0
[  253.360404]  module_decompress+0x37/0xc0
[  253.360564]  init_module_from_file+0xd0/0x100
[  253.360747]  idempotent_init_module+0x11c/0x2b0
[  253.360930]  __x64_sys_finit_module+0x64/0xd0
[  253.361148]  x64_sys_call+0x1d6e/0x25c0
[  253.361423]  do_syscall_64+0x7f/0x180
[  253.361616]  ? irqentry_exit_to_user_mode+0x7b/0x260
[  253.361991]  ? irqentry_exit+0x43/0x50
[  253.362167]  entry_SYSCALL_64_after_hwframe+0x73/0x7b
[  253.362358] RIP: 0033:0x7c9dc1b5a25d
[  253.362568] Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 8b bb 0d 00 f7 d8 64 89 01 48
[  253.363575] RSP: 002b:00007fff26524238 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[  253.363937] RAX: ffffffffffffffda RBX: 000058d8cd8cad50 RCX: 00007c9dc1b5a25d
[  253.364191] RDX: 0000000000000004 RSI: 00007c9dc1c9707d RDI: 000000000000001e
[  253.364427] RBP: 00007fff265242f0 R08: 0000000000000040 R09: 00007fff26524280
[  253.364682] R10: 00007c9dc1c36b20 R11: 0000000000000246 R12: 00007c9dc1c9707d
[  253.364949] R13: 0000000000020000 R14: 000058d8cd8de7b0 R15: 000058d8cd8df5d0
[  253.365195]  </TASK>
[  277.584863] watchdog: BUG: soft lockup - CPU#0 stuck for 250s! [(udev-worker):120]
[  277.585810] Modules linked in: crypto_simd cryptd
[  277.586503] CPU: 0 PID: 120 Comm: (udev-worker) Tainted: G             L     6.8.0-31-generic #31-Ubuntu
[  277.587266] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS Heads-v0.2.0-2178-gcf08056 01/01/1970
[  277.587547] RIP: 0010:clear_page_rep+0x7/0x10
[  277.587715] Code: 5b 41 5c 5d 31 ff c3 cc cc cc cc cc cc cc cc cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab c3 cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90
[  277.588332] RSP: 0018:ffffaf6cc027fa98 EFLAGS: 00000246
[  277.588503] RAX: 0000000000000000 RBX: 0000000000000100 RCX: 00000000000000bf
[  277.588770] RDX: fffff7aec414c000 RSI: fffff7aec4148b80 RDI: ffff932bc522ea08
[  277.589053] RBP: ffffaf6cc027fac8 R08: 0000000000000000 R09: 0000000000000000
[  277.589324] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000004000
[  277.589551] R13: 0000000000000008 R14: fffff7aec4148000 R15: 0000000000000001
[  277.590251] FS:  00007c9dc13958c0(0000) GS:ffff932d3fc00000(0000) knlGS:0000000000000000
[  277.590484] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  277.590662] CR2: 00007c9dc13495de CR3: 00000001050f6000 CR4: 00000000000006f0
[  277.590855] Call Trace:
[  277.590956]  <IRQ>
[  277.591051]  ? show_regs+0x6d/0x80
[  277.591182]  ? watchdog_timer_fn+0x206/0x290
[  277.591313]  ? __pfx_watchdog_timer_fn+0x10/0x10
[  277.591448]  ? __hrtimer_run_queues+0x112/0x2a0
[  277.591674]  ? clockevents_program_event+0xc1/0x150
[  277.591914]  ? hrtimer_interrupt+0xf6/0x250
[  277.592112]  ? __sysvec_apic_timer_interrupt+0x51/0x150
[  277.592339]  ? sysvec_apic_timer_interrupt+0x8d/0xd0
[  277.592518]  </IRQ>
[  277.592602]  <TASK>
[  277.592705]  ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
[  277.592867]  ? clear_page_rep+0x7/0x10
[  277.592973]  ? post_alloc_hook+0xcd/0x120
[  277.593082]  get_page_from_freelist+0x452/0x6d0
[  277.593195]  ? _copy_to_iter+0x34d/0x590
[  277.593307]  __alloc_pages+0x1e9/0x350
[  277.593410]  __kmalloc_large_node+0x75/0x130
[  277.593519]  __kmalloc_node+0x3ed/0x500
[  277.593619]  ? kvmalloc_node+0x5d/0x100
[  277.594306]  kvmalloc_node+0x5d/0x100
[  277.594953]  ? kvmalloc_node+0x5d/0x100
[  277.595506]  module_zstd_decompress+0xe1/0x2a0
[  277.596158]  module_decompress+0x37/0xc0
[  277.596720]  init_module_from_file+0xd0/0x100
[  277.597384]  idempotent_init_module+0x11c/0x2b0
[  277.597617]  __x64_sys_finit_module+0x64/0xd0
[  277.597823]  x64_sys_call+0x1d6e/0x25c0
[  277.597969]  do_syscall_64+0x7f/0x180
[  277.598098]  ? irqentry_exit_to_user_mode+0x7b/0x260
[  277.598257]  ? irqentry_exit+0x43/0x50
[  277.598375]  entry_SYSCALL_64_after_hwframe+0x73/0x7b
[  277.598530] RIP: 0033:0x7c9dc1b5a25d
[  277.598867] Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 8b bb 0d 00 f7 d8 64 89 01 48
[  277.601831] RSP: 002b:00007fff26524238 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[  277.602176] RAX: ffffffffffffffda RBX: 000058d8cd8cad50 RCX: 00007c9dc1b5a25d
[  277.602417] RDX: 0000000000000004 RSI: 00007c9dc1c9707d RDI: 000000000000001e
[  277.602728] RBP: 00007fff265242f0 R08: 0000000000000040 R09: 00007fff26524280
[  277.603081] R10: 00007c9dc1c36b20 R11: 0000000000000246 R12: 00007c9dc1c9707d
[  277.603347] R13: 0000000000020000 R14: 000058d8cd8de7b0 R15: 000058d8cd8df5d0
[  277.603620]  </TASK>

pit works just fine.
Signed-off-by: Thierry Laurion insurgo@riseup.net

tsc results in

[   30.260883] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 3703355526 wd_nsec: 3703355450
[   58.769175] watchdog: BUG: soft lockup - CPU#0 stuck for 46s! [(udev-worker):120]
[   58.770863] Modules linked in: crypto_simd cryptd
[   58.772911] CPU: 0 PID: 120 Comm: (udev-worker) Not tainted 6.8.0-31-generic linuxboot#31-Ubuntu
[   58.773335] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS Heads-v0.2.0-2178-gcf08056 01/01/1970
[   58.774045] RIP: 0010:clear_page_rep+0x7/0x10
[   58.774393] Code: 5b 41 5c 5d 31 ff c3 cc cc cc cc cc cc cc cc cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab c3 cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90
[   58.775262] RSP: 0018:ffffaf6cc027fa98 EFLAGS: 00000246
[   58.775501] RAX: 0000000000000000 RBX: 0000000000000100 RCX: 00000000000001de
[   58.775958] RDX: fffff7aec414c000 RSI: fffff7aec4148b00 RDI: ffff932bc522c110
[   58.776205] RBP: ffffaf6cc027fac8 R08: 0000000000000000 R09: 0000000000000000
[   58.776442] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000004000
[   58.776680] R13: 0000000000000008 R14: fffff7aec4148000 R15: 0000000000000001
[   58.776958] FS:  00007c9dc13958c0(0000) GS:ffff932d3fc00000(0000) knlGS:0000000000000000
[   58.777274] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   58.777481] CR2: 00007c9dc13495de CR3: 00000001050f6000 CR4: 00000000000006f0
[   58.777982] Call Trace:
[   58.778786]  <IRQ>
[   58.779118]  ? show_regs+0x6d/0x80
[   58.779301]  ? watchdog_timer_fn+0x206/0x290
[   58.779414]  ? __pfx_watchdog_timer_fn+0x10/0x10
[   58.779528]  ? __hrtimer_run_queues+0x112/0x2a0
[   58.779682]  ? hrtimer_interrupt+0xf6/0x250
[   58.779818]  ? __sysvec_apic_timer_interrupt+0x51/0x150
[   58.779957]  ? sysvec_apic_timer_interrupt+0x8d/0xd0
[   58.780110]  </IRQ>
[   58.780189]  <TASK>
[   58.780254]  ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
[   58.780429]  ? clear_page_rep+0x7/0x10
[   58.780534]  ? post_alloc_hook+0xcd/0x120
[   58.780688]  get_page_from_freelist+0x452/0x6d0
[   58.780943]  ? _copy_to_iter+0x34d/0x590
[   58.781122]  __alloc_pages+0x1e9/0x350
[   58.781342]  __kmalloc_large_node+0x75/0x130
[   58.781503]  __kmalloc_node+0x3ed/0x500
[   58.781677]  ? kvmalloc_node+0x5d/0x100
[   58.782303]  kvmalloc_node+0x5d/0x100
[   58.782770]  ? kvmalloc_node+0x5d/0x100
[   58.782931]  module_zstd_decompress+0xe1/0x2a0
[   58.783105]  module_decompress+0x37/0xc0
[   58.783236]  init_module_from_file+0xd0/0x100
[   58.783389]  idempotent_init_module+0x11c/0x2b0
[   58.783639]  __x64_sys_finit_module+0x64/0xd0
[   58.783933]  x64_sys_call+0x1d6e/0x25c0
[   58.784097]  do_syscall_64+0x7f/0x180
[   58.784229]  ? irqentry_exit_to_user_mode+0x7b/0x260
[   58.784400]  ? irqentry_exit+0x43/0x50
[   58.784543]  entry_SYSCALL_64_after_hwframe+0x73/0x7b
[   58.784887] RIP: 0033:0x7c9dc1b5a25d
[   58.785368] Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 8b bb 0d 00 f7 d8 64 89 01 48
[   58.786678] RSP: 002b:00007fff26524238 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[   58.787122] RAX: ffffffffffffffda RBX: 000058d8cd8cad50 RCX: 00007c9dc1b5a25d
[   58.787346] RDX: 0000000000000004 RSI: 00007c9dc1c9707d RDI: 000000000000001e
[   58.787556] RBP: 00007fff265242f0 R08: 0000000000000040 R09: 00007fff26524280
[   58.787777] R10: 00007c9dc1c36b20 R11: 0000000000000246 R12: 00007c9dc1c9707d
[   58.788043] R13: 0000000000020000 R14: 000058d8cd8de7b0 R15: 000058d8cd8df5d0
[   58.788427]  </TASK>
[   73.194850] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[   73.195398] rcu: 	(detected by 0, t=60952 jiffies, g=7897, q=1 ncpus=1)
[   73.195631] rcu: All QSes seen, last rcu_preempt kthread activity 60952 (4294739946-4294678994), jiffies_till_next_fqs=3, root ->qsmask 0x0
[   73.196080] rcu: rcu_preempt kthread starved for 60952 jiffies! g7897 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0
[   73.196327] rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
[   73.196524] rcu: RCU grace-period kthread stack dump:
[   73.196756] task:rcu_preempt     state:R  running task     stack:0     pid:17    tgid:17    ppid:2      flags:0x00004000
[   73.197272] Call Trace:
[   73.197400]  <TASK>
[   73.197493]  __schedule+0x27c/0x6b0
[   73.197617]  ? __pfx_rcu_gp_kthread+0x10/0x10
[   73.198112]  schedule+0x33/0x110
[   73.198444]  schedule_timeout+0x95/0x170
[   73.198901]  ? __pfx_process_timeout+0x10/0x10
[   73.199289]  rcu_gp_fqs_loop+0x105/0x580
[   73.199789]  rcu_gp_kthread+0xea/0x180
[   73.200291]  kthread+0xf2/0x120
[   73.200661]  ? __pfx_kthread+0x10/0x10
[   73.201057]  ret_from_fork+0x47/0x70
[   73.201462]  ? __pfx_kthread+0x10/0x10
[   73.201922]  ret_from_fork_asm+0x1b/0x30
[   73.202476]  </TASK>
[   73.202945] rcu: Stack dump where RCU GP kthread last ran:
[   73.203783] CPU: 0 PID: 120 Comm: (udev-worker) Tainted: G             L     6.8.0-31-generic linuxboot#31-Ubuntu
[   73.204164] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS Heads-v0.2.0-2178-gcf08056 01/01/1970
[   73.204430] RIP: 0010:clear_page_rep+0x7/0x10
[   73.204632] Code: 5b 41 5c 5d 31 ff c3 cc cc cc cc cc cc cc cc cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab c3 cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90
[   73.205404] RSP: 0018:ffffaf6cc027fa98 EFLAGS: 00000246
[   73.205587] RAX: 0000000000000000 RBX: 0000000000000100 RCX: 00000000000001d3
[   73.205792] RDX: fffff7aec414c000 RSI: fffff7aec4148b00 RDI: ffff932bc522c168
[   73.205961] RBP: ffffaf6cc027fac8 R08: 0000000000000000 R09: 0000000000000000
[   73.206138] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000004000
[   73.206331] R13: 0000000000000008 R14: fffff7aec4148000 R15: 0000000000000001
[   73.206531] FS:  00007c9dc13958c0(0000) GS:ffff932d3fc00000(0000) knlGS:0000000000000000
[   73.206757] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   73.206901] CR2: 00007c9dc13495de CR3: 00000001050f6000 CR4: 00000000000006f0
[   73.207074] Call Trace:
[   73.207150]  <IRQ>
[   73.207220]  ? show_regs+0x6d/0x80
[   73.207328]  ? dump_cpu_task+0x77/0x90
[   73.207457]  ? rcu_check_gp_kthread_starvation+0x1ce/0x280
[   73.207730]  ? print_other_cpu_stall+0x2a3/0x580
[   73.207954]  ? check_cpu_stall+0x1ca/0x230
[   73.208129]  ? rcu_pending+0x32/0x1f0
[   73.208310]  ? account_system_index_time+0x9a/0xd0
[   73.208519]  ? rcu_sched_clock_irq+0xd5/0x3c0
[   73.208756]  ? update_process_times+0x76/0xb0
[   73.209042]  ? tick_sched_handle+0x28/0x70
[   73.209190]  ? tick_nohz_highres_handler+0x78/0xa0
[   73.209349]  ? __pfx_tick_nohz_highres_handler+0x10/0x10
[   73.209528]  ? __hrtimer_run_queues+0x112/0x2a0
[   73.209770]  ? hrtimer_interrupt+0xf6/0x250
[   73.210281]  ? __sysvec_apic_timer_interrupt+0x51/0x150
[   73.210943]  ? sysvec_apic_timer_interrupt+0x8d/0xd0
[   73.211495]  </IRQ>
[   73.211703]  <TASK>
[   73.211791]  ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
[   73.211977]  ? clear_page_rep+0x7/0x10
[   73.212109]  ? post_alloc_hook+0xcd/0x120
[   73.212243]  get_page_from_freelist+0x452/0x6d0
[   73.212374]  ? _copy_to_iter+0x34d/0x590
[   73.212502]  __alloc_pages+0x1e9/0x350
[   73.212626]  __kmalloc_large_node+0x75/0x130
[   73.212826]  __kmalloc_node+0x3ed/0x500
[   73.212999]  ? kvmalloc_node+0x5d/0x100
[   73.213181]  kvmalloc_node+0x5d/0x100
[   73.213378]  ? kvmalloc_node+0x5d/0x100
[   73.213575]  module_zstd_decompress+0xe1/0x2a0
[   73.213765]  module_decompress+0x37/0xc0
[   73.213905]  init_module_from_file+0xd0/0x100
[   73.214063]  idempotent_init_module+0x11c/0x2b0
[   73.214224]  __x64_sys_finit_module+0x64/0xd0
[   73.214444]  x64_sys_call+0x1d6e/0x25c0
[   73.214604]  do_syscall_64+0x7f/0x180
[   73.214978]  ? irqentry_exit_to_user_mode+0x7b/0x260
[   73.215176]  ? irqentry_exit+0x43/0x50
[   73.215322]  entry_SYSCALL_64_after_hwframe+0x73/0x7b
[   73.215493] RIP: 0033:0x7c9dc1b5a25d
[   73.215673] Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 8b bb 0d 00 f7 d8 64 89 01 48
[   73.216258] RSP: 002b:00007fff26524238 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[   73.216505] RAX: ffffffffffffffda RBX: 000058d8cd8cad50 RCX: 00007c9dc1b5a25d
[   73.216717] RDX: 0000000000000004 RSI: 00007c9dc1c9707d RDI: 000000000000001e
[   73.217042] RBP: 00007fff265242f0 R08: 0000000000000040 R09: 00007fff26524280
[   73.217410] R10: 00007c9dc1c36b20 R11: 0000000000000246 R12: 00007c9dc1c9707d
[   73.217696] R13: 0000000000020000 R14: 000058d8cd8de7b0 R15: 000058d8cd8df5d0
[   73.217966]  </TASK>
[   98.119999] watchdog: BUG: soft lockup - CPU#0 stuck for 83s! [(udev-worker):120]
[   98.120326] Modules linked in: crypto_simd cryptd
[   98.120494] CPU: 0 PID: 120 Comm: (udev-worker) Tainted: G             L     6.8.0-31-generic linuxboot#31-Ubuntu
[   98.120796] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS Heads-v0.2.0-2178-gcf08056 01/01/1970
[   98.121190] RIP: 0010:clear_page_rep+0x7/0x10
[   98.121387] Code: 5b 41 5c 5d 31 ff c3 cc cc cc cc cc cc cc cc cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab c3 cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90
[   98.122079] RSP: 0018:ffffaf6cc027fa98 EFLAGS: 00000246
[   98.122280] RAX: 0000000000000000 RBX: 0000000000000100 RCX: 000000000000019d
[   98.122517] RDX: fffff7aec414c000 RSI: fffff7aec4148b00 RDI: ffff932bc522c318
[   98.122844] RBP: ffffaf6cc027fac8 R08: 0000000000000000 R09: 0000000000000000
[   98.123143] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000004000
[   98.123373] R13: 0000000000000008 R14: fffff7aec4148000 R15: 0000000000000001
[   98.123629] FS:  00007c9dc13958c0(0000) GS:ffff932d3fc00000(0000) knlGS:0000000000000000
[   98.124009] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   98.124366] CR2: 00007c9dc13495de CR3: 00000001050f6000 CR4: 00000000000006f0
[   98.124616] Call Trace:
[   98.124744]  <IRQ>
[   98.124848]  ? show_regs+0x6d/0x80
[   98.125027]  ? watchdog_timer_fn+0x206/0x290
[   98.125351]  ? __pfx_watchdog_timer_fn+0x10/0x10
[   98.125549]  ? __hrtimer_run_queues+0x112/0x2a0
[   98.125738]  ? hrtimer_interrupt+0xf6/0x250
[   98.125931]  ? __sysvec_apic_timer_interrupt+0x51/0x150
[   98.126227]  ? sysvec_apic_timer_interrupt+0x8d/0xd0
[   98.126406]  </IRQ>
[   98.126490]  <TASK>
[   98.126574]  ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
[   98.127177]  ? clear_page_rep+0x7/0x10
[   98.127842]  ? post_alloc_hook+0xcd/0x120
[   98.128038]  get_page_from_freelist+0x452/0x6d0
[   98.128196]  ? _copy_to_iter+0x34d/0x590
[   98.128337]  __alloc_pages+0x1e9/0x350
[   98.128467]  __kmalloc_large_node+0x75/0x130
[   98.128717]  __kmalloc_node+0x3ed/0x500
[   98.128882]  ? kvmalloc_node+0x5d/0x100
[   98.129044]  kvmalloc_node+0x5d/0x100
[   98.129175]  ? kvmalloc_node+0x5d/0x100
[   98.129292]  module_zstd_decompress+0xe1/0x2a0
[   98.129431]  module_decompress+0x37/0xc0
[   98.129549]  init_module_from_file+0xd0/0x100
[   98.129771]  idempotent_init_module+0x11c/0x2b0
[   98.130078]  __x64_sys_finit_module+0x64/0xd0
[   98.130252]  x64_sys_call+0x1d6e/0x25c0
[   98.130404]  do_syscall_64+0x7f/0x180
[   98.130541]  ? irqentry_exit_to_user_mode+0x7b/0x260
[   98.130840]  ? irqentry_exit+0x43/0x50
[   98.131054]  entry_SYSCALL_64_after_hwframe+0x73/0x7b
[   98.131294] RIP: 0033:0x7c9dc1b5a25d
[   98.131574] Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 8b bb 0d 00 f7 d8 64 89 01 48
[   98.132318] RSP: 002b:00007fff26524238 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[   98.132978] RAX: ffffffffffffffda RBX: 000058d8cd8cad50 RCX: 00007c9dc1b5a25d
[   98.133312] RDX: 0000000000000004 RSI: 00007c9dc1c9707d RDI: 000000000000001e
[   98.133603] RBP: 00007fff265242f0 R08: 0000000000000040 R09: 00007fff26524280
[   98.134005] R10: 00007c9dc1c36b20 R11: 0000000000000246 R12: 00007c9dc1c9707d
[   98.134323] R13: 0000000000020000 R14: 000058d8cd8de7b0 R15: 000058d8cd8df5d0
[   98.134846]  </TASK>
[  125.509373] watchdog: BUG: soft lockup - CPU#0 stuck for 108s! [(udev-worker):120]
[  125.509705] Modules linked in: crypto_simd cryptd
[  125.509908] CPU: 0 PID: 120 Comm: (udev-worker) Tainted: G             L     6.8.0-31-generic linuxboot#31-Ubuntu
[  125.510225] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS Heads-v0.2.0-2178-gcf08056 01/01/1970
[  125.510678] RIP: 0010:clear_page_rep+0x7/0x10
[  125.510921] Code: 5b 41 5c 5d 31 ff c3 cc cc cc cc cc cc cc cc cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab c3 cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90
[  125.511613] RSP: 0018:ffffaf6cc027fa98 EFLAGS: 00000246
[  125.511827] RAX: 0000000000000000 RBX: 0000000000000100 RCX: 0000000000000044
[  125.512017] RDX: fffff7aec414c000 RSI: fffff7aec4148b00 RDI: ffff932bc522cde0
[  125.512247] RBP: ffffaf6cc027fac8 R08: 0000000000000000 R09: 0000000000000000
[  125.512460] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000004000
[  125.512733] R13: 0000000000000008 R14: fffff7aec4148000 R15: 0000000000000001
[  125.513557] FS:  00007c9dc13958c0(0000) GS:ffff932d3fc00000(0000) knlGS:0000000000000000
[  125.514233] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  125.514425] CR2: 00007c9dc13495de CR3: 00000001050f6000 CR4: 00000000000006f0
[  125.514628] Call Trace:
[  125.514732]  <IRQ>
[  125.514867]  ? show_regs+0x6d/0x80
[  125.515055]  ? watchdog_timer_fn+0x206/0x290
[  125.515221]  ? __pfx_watchdog_timer_fn+0x10/0x10
[  125.515381]  ? __hrtimer_run_queues+0x112/0x2a0
[  125.515551]  ? hrtimer_interrupt+0xf6/0x250
[  125.515738]  ? __sysvec_apic_timer_interrupt+0x51/0x150
[  125.515967]  ? sysvec_apic_timer_interrupt+0x8d/0xd0
[  125.516144]  </IRQ>
[  125.516223]  <TASK>
[  125.516300]  ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
[  125.516474]  ? clear_page_rep+0x7/0x10
[  125.516701]  ? post_alloc_hook+0xcd/0x120
[  125.516929]  get_page_from_freelist+0x452/0x6d0
[  125.517146]  ? _copy_to_iter+0x34d/0x590
[  125.517345]  __alloc_pages+0x1e9/0x350
[  125.517540]  __kmalloc_large_node+0x75/0x130
[  125.517739]  __kmalloc_node+0x3ed/0x500
[  125.517907]  ? kvmalloc_node+0x5d/0x100
[  125.518060]  kvmalloc_node+0x5d/0x100
[  125.518195]  ? kvmalloc_node+0x5d/0x100
[  125.518317]  module_zstd_decompress+0xe1/0x2a0
[  125.518460]  module_decompress+0x37/0xc0
[  125.518587]  init_module_from_file+0xd0/0x100
[  125.518746]  idempotent_init_module+0x11c/0x2b0
[  125.518896]  __x64_sys_finit_module+0x64/0xd0
[  125.519026]  x64_sys_call+0x1d6e/0x25c0
[  125.519133]  do_syscall_64+0x7f/0x180
[  125.519238]  ? irqentry_exit_to_user_mode+0x7b/0x260
[  125.519370]  ? irqentry_exit+0x43/0x50
[  125.519557]  entry_SYSCALL_64_after_hwframe+0x73/0x7b
[  125.519798] RIP: 0033:0x7c9dc1b5a25d
[  125.519956] Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 8b bb 0d 00 f7 d8 64 89 01 48
[  125.520425] RSP: 002b:00007fff26524238 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[  125.520651] RAX: ffffffffffffffda RBX: 000058d8cd8cad50 RCX: 00007c9dc1b5a25d
[  125.520842] RDX: 0000000000000004 RSI: 00007c9dc1c9707d RDI: 000000000000001e
[  125.521029] RBP: 00007fff265242f0 R08: 0000000000000040 R09: 00007fff26524280
[  125.521207] R10: 00007c9dc1c36b20 R11: 0000000000000246 R12: 00007c9dc1c9707d
[  125.521383] R13: 0000000000020000 R14: 000058d8cd8de7b0 R15: 000058d8cd8df5d0
[  125.521571]  </TASK>
[  153.925148] watchdog: BUG: soft lockup - CPU#0 stuck for 135s! [(udev-worker):120]
[  153.925494] Modules linked in: crypto_simd cryptd
[  153.926140] CPU: 0 PID: 120 Comm: (udev-worker) Tainted: G             L     6.8.0-31-generic linuxboot#31-Ubuntu
[  153.926817] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS Heads-v0.2.0-2178-gcf08056 01/01/1970
[  153.927150] RIP: 0010:clear_page_rep+0x7/0x10
[  153.927386] Code: 5b 41 5c 5d 31 ff c3 cc cc cc cc cc cc cc cc cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab c3 cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90
[  153.928196] RSP: 0018:ffffaf6cc027fa98 EFLAGS: 00000246
[  153.928445] RAX: 0000000000000000 RBX: 0000000000000100 RCX: 00000000000001c9
[  153.928812] RDX: fffff7aec414c000 RSI: fffff7aec4148b40 RDI: ffff932bc522d1b8
[  153.929065] RBP: ffffaf6cc027fac8 R08: 0000000000000000 R09: 0000000000000000
[  153.929296] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000004000
[  153.929559] R13: 0000000000000008 R14: fffff7aec4148000 R15: 0000000000000001
[  153.929793] FS:  00007c9dc13958c0(0000) GS:ffff932d3fc00000(0000) knlGS:0000000000000000
[  153.930085] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  153.930279] CR2: 00007c9dc13495de CR3: 00000001050f6000 CR4: 00000000000006f0
[  153.930566] Call Trace:
[  153.930749]  <IRQ>
[  153.930938]  ? show_regs+0x6d/0x80
[  153.931104]  ? watchdog_timer_fn+0x206/0x290
[  153.931256]  ? __pfx_watchdog_timer_fn+0x10/0x10
[  153.931436]  ? __hrtimer_run_queues+0x112/0x2a0
[  153.931801]  ? hrtimer_interrupt+0xf6/0x250
[  153.932033]  ? __sysvec_apic_timer_interrupt+0x51/0x150
[  153.932229]  ? sysvec_apic_timer_interrupt+0x8d/0xd0
[  153.932404]  </IRQ>
[  153.932501]  <TASK>
[  153.932628]  ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
[  153.933064]  ? clear_page_rep+0x7/0x10
[  153.933216]  ? post_alloc_hook+0xcd/0x120
[  153.933446]  get_page_from_freelist+0x452/0x6d0
[  153.933726]  ? _copy_to_iter+0x34d/0x590
[  153.933889]  __alloc_pages+0x1e9/0x350
[  153.934027]  __kmalloc_large_node+0x75/0x130
[  153.934163]  __kmalloc_node+0x3ed/0x500
[  153.934283]  ? kvmalloc_node+0x5d/0x100
[  153.934415]  kvmalloc_node+0x5d/0x100
[  153.934535]  ? kvmalloc_node+0x5d/0x100
[  153.934769]  module_zstd_decompress+0xe1/0x2a0
[  153.935014]  module_decompress+0x37/0xc0
[  153.935237]  init_module_from_file+0xd0/0x100
[  153.935481]  idempotent_init_module+0x11c/0x2b0
[  153.935714]  __x64_sys_finit_module+0x64/0xd0
[  153.935890]  x64_sys_call+0x1d6e/0x25c0
[  153.936039]  do_syscall_64+0x7f/0x180
[  153.936177]  ? irqentry_exit_to_user_mode+0x7b/0x260
[  153.936357]  ? irqentry_exit+0x43/0x50
[  153.936505]  entry_SYSCALL_64_after_hwframe+0x73/0x7b
[  153.936953] RIP: 0033:0x7c9dc1b5a25d
[  153.937358] Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 8b bb 0d 00 f7 d8 64 89 01 48
[  153.939607] RSP: 002b:00007fff26524238 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[  153.940385] RAX: ffffffffffffffda RBX: 000058d8cd8cad50 RCX: 00007c9dc1b5a25d
[  153.940693] RDX: 0000000000000004 RSI: 00007c9dc1c9707d RDI: 000000000000001e
[  153.940909] RBP: 00007fff265242f0 R08: 0000000000000040 R09: 00007fff26524280
[  153.941105] R10: 00007c9dc1c36b20 R11: 0000000000000246 R12: 00007c9dc1c9707d
[  153.941298] R13: 0000000000020000 R14: 000058d8cd8de7b0 R15: 000058d8cd8df5d0
[  153.941555]  </TASK>
[  181.553279] watchdog: BUG: soft lockup - CPU#0 stuck for 161s! [(udev-worker):120]
[  181.554138] Modules linked in: crypto_simd cryptd
[  181.554739] CPU: 0 PID: 120 Comm: (udev-worker) Tainted: G             L     6.8.0-31-generic linuxboot#31-Ubuntu
[  181.555940] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS Heads-v0.2.0-2178-gcf08056 01/01/1970
[  181.556396] RIP: 0010:clear_page_rep+0x7/0x10
[  181.556580] Code: 5b 41 5c 5d 31 ff c3 cc cc cc cc cc cc cc cc cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab c3 cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90
[  181.557167] RSP: 0018:ffffaf6cc027fa98 EFLAGS: 00000246
[  181.557350] RAX: 0000000000000000 RBX: 0000000000000100 RCX: 00000000000001fc
[  181.557558] RDX: fffff7aec414c000 RSI: fffff7aec4148b80 RDI: ffff932bc522e020
[  181.558219] RBP: ffffaf6cc027fac8 R08: 0000000000000000 R09: 0000000000000000
[  181.559037] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000004000
[  181.559249] R13: 0000000000000008 R14: fffff7aec4148000 R15: 0000000000000001
[  181.559448] FS:  00007c9dc13958c0(0000) GS:ffff932d3fc00000(0000) knlGS:0000000000000000
[  181.559753] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  181.559968] CR2: 00007c9dc13495de CR3: 00000001050f6000 CR4: 00000000000006f0
[  181.560208] Call Trace:
[  181.560310]  <IRQ>
[  181.560407]  ? show_regs+0x6d/0x80
[  181.560530]  ? watchdog_timer_fn+0x206/0x290
[  181.560677]  ? __pfx_watchdog_timer_fn+0x10/0x10
[  181.560852]  ? __hrtimer_run_queues+0x112/0x2a0
[  181.561074]  ? hrtimer_interrupt+0xf6/0x250
[  181.561304]  ? __sysvec_apic_timer_interrupt+0x51/0x150
[  181.561514]  ? sysvec_apic_timer_interrupt+0x8d/0xd0
[  181.561693]  </IRQ>
[  181.561769]  <TASK>
[  181.561840]  ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
[  181.561995]  ? clear_page_rep+0x7/0x10
[  181.562095]  ? post_alloc_hook+0xcd/0x120
[  181.562204]  get_page_from_freelist+0x452/0x6d0
[  181.562316]  ? _copy_to_iter+0x34d/0x590
[  181.562426]  __alloc_pages+0x1e9/0x350
[  181.562530]  __kmalloc_large_node+0x75/0x130
[  181.562656]  __kmalloc_node+0x3ed/0x500
[  181.562811]  ? kvmalloc_node+0x5d/0x100
[  181.563042]  kvmalloc_node+0x5d/0x100
[  181.563201]  ? kvmalloc_node+0x5d/0x100
[  181.563416]  module_zstd_decompress+0xe1/0x2a0
[  181.563606]  module_decompress+0x37/0xc0
[  181.563782]  init_module_from_file+0xd0/0x100
[  181.563951]  idempotent_init_module+0x11c/0x2b0
[  181.564098]  __x64_sys_finit_module+0x64/0xd0
[  181.564226]  x64_sys_call+0x1d6e/0x25c0
[  181.564340]  do_syscall_64+0x7f/0x180
[  181.564451]  ? irqentry_exit_to_user_mode+0x7b/0x260
[  181.564594]  ? irqentry_exit+0x43/0x50
[  181.565072]  entry_SYSCALL_64_after_hwframe+0x73/0x7b
[  181.565796] RIP: 0033:0x7c9dc1b5a25d
[  181.566259] Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 8b bb 0d 00 f7 d8 64 89 01 48
[  181.567956] RSP: 002b:00007fff26524238 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[  181.568189] RAX: ffffffffffffffda RBX: 000058d8cd8cad50 RCX: 00007c9dc1b5a25d
[  181.568358] RDX: 0000000000000004 RSI: 00007c9dc1c9707d RDI: 000000000000001e
[  181.568510] RBP: 00007fff265242f0 R08: 0000000000000040 R09: 00007fff26524280
[  181.568687] R10: 00007c9dc1c36b20 R11: 0000000000000246 R12: 00007c9dc1c9707d
[  181.568936] R13: 0000000000020000 R14: 000058d8cd8de7b0 R15: 000058d8cd8df5d0
[  181.569176]  </TASK>
[  214.072068] watchdog: BUG: soft lockup - CPU#0 stuck for 191s! [(udev-worker):120]
[  214.072405] Modules linked in: crypto_simd cryptd
[  214.072671] CPU: 0 PID: 120 Comm: (udev-worker) Tainted: G             L     6.8.0-31-generic linuxboot#31-Ubuntu
[  214.073211] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS Heads-v0.2.0-2178-gcf08056 01/01/1970
[  214.073504] RIP: 0010:clear_page_rep+0x7/0x10
[  214.073697] Code: 5b 41 5c 5d 31 ff c3 cc cc cc cc cc cc cc cc cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab c3 cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90
[  214.074245] RSP: 0018:ffffaf6cc027fa98 EFLAGS: 00000246
[  214.074459] RAX: 0000000000000000 RBX: 0000000000000100 RCX: 00000000000001d8
[  214.074774] RDX: fffff7aec414c000 RSI: fffff7aec4148b80 RDI: ffff932bc522e140
[  214.075107] RBP: ffffaf6cc027fac8 R08: 0000000000000000 R09: 0000000000000000
[  214.075298] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000004000
[  214.075481] R13: 0000000000000008 R14: fffff7aec4148000 R15: 0000000000000001
[  214.075747] FS:  00007c9dc13958c0(0000) GS:ffff932d3fc00000(0000) knlGS:0000000000000000
[  214.076045] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  214.076215] CR2: 00007c9dc13495de CR3: 00000001050f6000 CR4: 00000000000006f0
[  214.076403] Call Trace:
[  214.076550]  <IRQ>
[  214.076692]  ? show_regs+0x6d/0x80
[  214.076867]  ? watchdog_timer_fn+0x206/0x290
[  214.077067]  ? __pfx_watchdog_timer_fn+0x10/0x10
[  214.077245]  ? __hrtimer_run_queues+0x112/0x2a0
[  214.077390]  ? clockevents_program_event+0xc1/0x150
[  214.077550]  ? hrtimer_interrupt+0xf6/0x250
[  214.077720]  ? __sysvec_apic_timer_interrupt+0x51/0x150
[  214.078002]  ? sysvec_apic_timer_interrupt+0x8d/0xd0
[  214.078233]  </IRQ>
[  214.078345]  <TASK>
[  214.078442]  ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
[  214.078697]  ? clear_page_rep+0x7/0x10
[  214.078849]  ? post_alloc_hook+0xcd/0x120
[  214.079012]  get_page_from_freelist+0x452/0x6d0
[  214.079153]  ? _copy_to_iter+0x34d/0x590
[  214.079275]  __alloc_pages+0x1e9/0x350
[  214.079381]  __kmalloc_large_node+0x75/0x130
[  214.079491]  __kmalloc_node+0x3ed/0x500
[  214.079655]  ? kvmalloc_node+0x5d/0x100
[  214.079823]  kvmalloc_node+0x5d/0x100
[  214.079967]  ? kvmalloc_node+0x5d/0x100
[  214.080106]  module_zstd_decompress+0xe1/0x2a0
[  214.080248]  module_decompress+0x37/0xc0
[  214.080430]  init_module_from_file+0xd0/0x100
[  214.080692]  idempotent_init_module+0x11c/0x2b0
[  214.080886]  __x64_sys_finit_module+0x64/0xd0
[  214.081088]  x64_sys_call+0x1d6e/0x25c0
[  214.081230]  do_syscall_64+0x7f/0x180
[  214.081388]  ? irqentry_exit_to_user_mode+0x7b/0x260
[  214.081647]  ? irqentry_exit+0x43/0x50
[  214.081852]  entry_SYSCALL_64_after_hwframe+0x73/0x7b
[  214.082090] RIP: 0033:0x7c9dc1b5a25d
[  214.082241] Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 8b bb 0d 00 f7 d8 64 89 01 48
[  214.083087] RSP: 002b:00007fff26524238 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[  214.083690] RAX: ffffffffffffffda RBX: 000058d8cd8cad50 RCX: 00007c9dc1b5a25d
[  214.084173] RDX: 0000000000000004 RSI: 00007c9dc1c9707d RDI: 000000000000001e
[  214.084711] RBP: 00007fff265242f0 R08: 0000000000000040 R09: 00007fff26524280
[  214.085111] R10: 00007c9dc1c36b20 R11: 0000000000000246 R12: 00007c9dc1c9707d
[  214.085312] R13: 0000000000020000 R14: 000058d8cd8de7b0 R15: 000058d8cd8df5d0
[  214.085524]  </TASK>
[  242.577519] watchdog: BUG: soft lockup - CPU#0 stuck for 217s! [(udev-worker):120]
[  242.578518] Modules linked in: crypto_simd cryptd
[  242.579070] CPU: 0 PID: 120 Comm: (udev-worker) Tainted: G             L     6.8.0-31-generic linuxboot#31-Ubuntu
[  242.580110] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS Heads-v0.2.0-2178-gcf08056 01/01/1970
[  242.580672] RIP: 0010:clear_page_rep+0x7/0x10
[  242.580853] Code: 5b 41 5c 5d 31 ff c3 cc cc cc cc cc cc cc cc cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab c3 cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90
[  242.581453] RSP: 0018:ffffaf6cc027fa98 EFLAGS: 00000246
[  242.581659] RAX: 0000000000000000 RBX: 0000000000000100 RCX: 00000000000001af
[  242.581935] RDX: fffff7aec414c000 RSI: fffff7aec4148b80 RDI: ffff932bc522e288
[  242.582163] RBP: ffffaf6cc027fac8 R08: 0000000000000000 R09: 0000000000000000
[  242.582401] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000004000
[  242.582709] R13: 0000000000000008 R14: fffff7aec4148000 R15: 0000000000000001
[  242.582944] FS:  00007c9dc13958c0(0000) GS:ffff932d3fc00000(0000) knlGS:0000000000000000
[  242.583291] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  242.583484] CR2: 00007c9dc13495de CR3: 00000001050f6000 CR4: 00000000000006f0
[  242.583702] Call Trace:
[  242.583796]  <IRQ>
[  242.583911]  ? show_regs+0x6d/0x80
[  242.584076]  ? watchdog_timer_fn+0x206/0x290
[  242.584225]  ? __pfx_watchdog_timer_fn+0x10/0x10
[  242.584384]  ? __hrtimer_run_queues+0x112/0x2a0
[  242.584583]  ? hrtimer_interrupt+0xf6/0x250
[  242.584761]  ? __sysvec_apic_timer_interrupt+0x51/0x150
[  242.584944]  ? sysvec_apic_timer_interrupt+0x8d/0xd0
[  242.585099]  </IRQ>
[  242.585170]  <TASK>
[  242.585247]  ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
[  242.585410]  ? clear_page_rep+0x7/0x10
[  242.585545]  ? post_alloc_hook+0xcd/0x120
[  242.585695]  get_page_from_freelist+0x452/0x6d0
[  242.585834]  ? _copy_to_iter+0x34d/0x590
[  242.585965]  __alloc_pages+0x1e9/0x350
[  242.586099]  __kmalloc_large_node+0x75/0x130
[  242.586309]  __kmalloc_node+0x3ed/0x500
[  242.586499]  ? kvmalloc_node+0x5d/0x100
[  242.586699]  kvmalloc_node+0x5d/0x100
[  242.586829]  ? kvmalloc_node+0x5d/0x100
[  242.586955]  module_zstd_decompress+0xe1/0x2a0
[  242.587102]  module_decompress+0x37/0xc0
[  242.587205]  init_module_from_file+0xd0/0x100
[  242.587333]  idempotent_init_module+0x11c/0x2b0
[  242.587501]  __x64_sys_finit_module+0x64/0xd0
[  242.587685]  x64_sys_call+0x1d6e/0x25c0
[  242.587974]  do_syscall_64+0x7f/0x180
[  242.588098]  ? irqentry_exit_to_user_mode+0x7b/0x260
[  242.588238]  ? irqentry_exit+0x43/0x50
[  242.588358]  entry_SYSCALL_64_after_hwframe+0x73/0x7b
[  242.588504] RIP: 0033:0x7c9dc1b5a25d
[  242.588617] Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 8b bb 0d 00 f7 d8 64 89 01 48
[  242.589047] RSP: 002b:00007fff26524238 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[  242.589230] RAX: ffffffffffffffda RBX: 000058d8cd8cad50 RCX: 00007c9dc1b5a25d
[  242.589395] RDX: 0000000000000004 RSI: 00007c9dc1c9707d RDI: 000000000000001e
[  242.589612] RBP: 00007fff265242f0 R08: 0000000000000040 R09: 00007fff26524280
[  242.589973] R10: 00007c9dc1c36b20 R11: 0000000000000246 R12: 00007c9dc1c9707d
[  242.590316] R13: 0000000000020000 R14: 000058d8cd8de7b0 R15: 000058d8cd8df5d0
[  242.590553]  </TASK>
[  253.341960] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[  253.342313] rcu: 	(detected by 0, t=241099 jiffies, g=7897, q=1 ncpus=1)
[  253.342526] rcu: All QSes seen, last rcu_preempt kthread activity 241099 (4294920093-4294678994), jiffies_till_next_fqs=3, root ->qsmask 0x0
[  253.342971] rcu: rcu_preempt kthread starved for 241099 jiffies! g7897 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0
[  253.343307] rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
[  253.343542] rcu: RCU grace-period kthread stack dump:
[  253.343754] task:rcu_preempt     state:R  running task     stack:0     pid:17    tgid:17    ppid:2      flags:0x00004000
[  253.344138] Call Trace:
[  253.344241]  <TASK>
[  253.344341]  __schedule+0x27c/0x6b0
[  253.344551]  ? __pfx_rcu_gp_kthread+0x10/0x10
[  253.344783]  schedule+0x33/0x110
[  253.344967]  schedule_timeout+0x95/0x170
[  253.345110]  ? __pfx_process_timeout+0x10/0x10
[  253.345249]  rcu_gp_fqs_loop+0x105/0x580
[  253.345376]  rcu_gp_kthread+0xea/0x180
[  253.345558]  kthread+0xf2/0x120
[  253.345700]  ? __pfx_kthread+0x10/0x10
[  253.345820]  ret_from_fork+0x47/0x70
[  253.345927]  ? __pfx_kthread+0x10/0x10
[  253.346033]  ret_from_fork_asm+0x1b/0x30
[  253.346190]  </TASK>
[  253.346310] rcu: Stack dump where RCU GP kthread last ran:
[  253.346571] CPU: 0 PID: 120 Comm: (udev-worker) Tainted: G             L     6.8.0-31-generic linuxboot#31-Ubuntu
[  253.346979] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS Heads-v0.2.0-2178-gcf08056 01/01/1970
[  253.347306] RIP: 0010:clear_page_rep+0x7/0x10
[  253.347481] Code: 5b 41 5c 5d 31 ff c3 cc cc cc cc cc cc cc cc cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab c3 cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90
[  253.348260] RSP: 0018:ffffaf6cc027fa98 EFLAGS: 00000246
[  253.348657] RAX: 0000000000000000 RBX: 0000000000000100 RCX: 0000000000000198
[  253.349312] RDX: fffff7aec414c000 RSI: fffff7aec4148b80 RDI: ffff932bc522e340
[  253.349533] RBP: ffffaf6cc027fac8 R08: 0000000000000000 R09: 0000000000000000
[  253.349725] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000004000
[  253.349995] R13: 0000000000000008 R14: fffff7aec4148000 R15: 0000000000000001
[  253.350265] FS:  00007c9dc13958c0(0000) GS:ffff932d3fc00000(0000) knlGS:0000000000000000
[  253.350510] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  253.350681] CR2: 00007c9dc13495de CR3: 00000001050f6000 CR4: 00000000000006f0
[  253.350977] Call Trace:
[  253.351078]  <IRQ>
[  253.351158]  ? show_regs+0x6d/0x80
[  253.351276]  ? dump_cpu_task+0x77/0x90
[  253.351407]  ? rcu_check_gp_kthread_starvation+0x1ce/0x280
[  253.351581]  ? print_other_cpu_stall+0x2a3/0x580
[  253.352116]  ? check_cpu_stall+0x1ca/0x230
[  253.352597]  ? rcu_pending+0x32/0x1f0
[  253.352903]  ? account_system_index_time+0x9a/0xd0
[  253.353219]  ? rcu_sched_clock_irq+0xd5/0x3c0
[  253.353500]  ? update_process_times+0x76/0xb0
[  253.353847]  ? tick_sched_handle+0x28/0x70
[  253.354179]  ? tick_nohz_highres_handler+0x78/0xa0
[  253.354334]  ? __pfx_tick_nohz_highres_handler+0x10/0x10
[  253.354489]  ? __hrtimer_run_queues+0x112/0x2a0
[  253.354621]  ? clockevents_program_event+0xc1/0x150
[  253.354915]  ? hrtimer_interrupt+0xf6/0x250
[  253.355175]  ? __sysvec_apic_timer_interrupt+0x51/0x150
[  253.355479]  ? sysvec_apic_timer_interrupt+0x8d/0xd0
[  253.355704]  </IRQ>
[  253.355797]  <TASK>
[  253.355882]  ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
[  253.356071]  ? clear_page_rep+0x7/0x10
[  253.356194]  ? post_alloc_hook+0xcd/0x120
[  253.356331]  get_page_from_freelist+0x452/0x6d0
[  253.356475]  ? _copy_to_iter+0x34d/0x590
[  253.356612]  __alloc_pages+0x1e9/0x350
[  253.357109]  __kmalloc_large_node+0x75/0x130
[  253.357612]  __kmalloc_node+0x3ed/0x500
[  253.358092]  ? kvmalloc_node+0x5d/0x100
[  253.358662]  kvmalloc_node+0x5d/0x100
[  253.359345]  ? kvmalloc_node+0x5d/0x100
[  253.359962]  module_zstd_decompress+0xe1/0x2a0
[  253.360404]  module_decompress+0x37/0xc0
[  253.360564]  init_module_from_file+0xd0/0x100
[  253.360747]  idempotent_init_module+0x11c/0x2b0
[  253.360930]  __x64_sys_finit_module+0x64/0xd0
[  253.361148]  x64_sys_call+0x1d6e/0x25c0
[  253.361423]  do_syscall_64+0x7f/0x180
[  253.361616]  ? irqentry_exit_to_user_mode+0x7b/0x260
[  253.361991]  ? irqentry_exit+0x43/0x50
[  253.362167]  entry_SYSCALL_64_after_hwframe+0x73/0x7b
[  253.362358] RIP: 0033:0x7c9dc1b5a25d
[  253.362568] Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 8b bb 0d 00 f7 d8 64 89 01 48
[  253.363575] RSP: 002b:00007fff26524238 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[  253.363937] RAX: ffffffffffffffda RBX: 000058d8cd8cad50 RCX: 00007c9dc1b5a25d
[  253.364191] RDX: 0000000000000004 RSI: 00007c9dc1c9707d RDI: 000000000000001e
[  253.364427] RBP: 00007fff265242f0 R08: 0000000000000040 R09: 00007fff26524280
[  253.364682] R10: 00007c9dc1c36b20 R11: 0000000000000246 R12: 00007c9dc1c9707d
[  253.364949] R13: 0000000000020000 R14: 000058d8cd8de7b0 R15: 000058d8cd8df5d0
[  253.365195]  </TASK>
[  277.584863] watchdog: BUG: soft lockup - CPU#0 stuck for 250s! [(udev-worker):120]
[  277.585810] Modules linked in: crypto_simd cryptd
[  277.586503] CPU: 0 PID: 120 Comm: (udev-worker) Tainted: G             L     6.8.0-31-generic linuxboot#31-Ubuntu
[  277.587266] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS Heads-v0.2.0-2178-gcf08056 01/01/1970
[  277.587547] RIP: 0010:clear_page_rep+0x7/0x10
[  277.587715] Code: 5b 41 5c 5d 31 ff c3 cc cc cc cc cc cc cc cc cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab c3 cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90
[  277.588332] RSP: 0018:ffffaf6cc027fa98 EFLAGS: 00000246
[  277.588503] RAX: 0000000000000000 RBX: 0000000000000100 RCX: 00000000000000bf
[  277.588770] RDX: fffff7aec414c000 RSI: fffff7aec4148b80 RDI: ffff932bc522ea08
[  277.589053] RBP: ffffaf6cc027fac8 R08: 0000000000000000 R09: 0000000000000000
[  277.589324] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000004000
[  277.589551] R13: 0000000000000008 R14: fffff7aec4148000 R15: 0000000000000001
[  277.590251] FS:  00007c9dc13958c0(0000) GS:ffff932d3fc00000(0000) knlGS:0000000000000000
[  277.590484] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  277.590662] CR2: 00007c9dc13495de CR3: 00000001050f6000 CR4: 00000000000006f0
[  277.590855] Call Trace:
[  277.590956]  <IRQ>
[  277.591051]  ? show_regs+0x6d/0x80
[  277.591182]  ? watchdog_timer_fn+0x206/0x290
[  277.591313]  ? __pfx_watchdog_timer_fn+0x10/0x10
[  277.591448]  ? __hrtimer_run_queues+0x112/0x2a0
[  277.591674]  ? clockevents_program_event+0xc1/0x150
[  277.591914]  ? hrtimer_interrupt+0xf6/0x250
[  277.592112]  ? __sysvec_apic_timer_interrupt+0x51/0x150
[  277.592339]  ? sysvec_apic_timer_interrupt+0x8d/0xd0
[  277.592518]  </IRQ>
[  277.592602]  <TASK>
[  277.592705]  ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
[  277.592867]  ? clear_page_rep+0x7/0x10
[  277.592973]  ? post_alloc_hook+0xcd/0x120
[  277.593082]  get_page_from_freelist+0x452/0x6d0
[  277.593195]  ? _copy_to_iter+0x34d/0x590
[  277.593307]  __alloc_pages+0x1e9/0x350
[  277.593410]  __kmalloc_large_node+0x75/0x130
[  277.593519]  __kmalloc_node+0x3ed/0x500
[  277.593619]  ? kvmalloc_node+0x5d/0x100
[  277.594306]  kvmalloc_node+0x5d/0x100
[  277.594953]  ? kvmalloc_node+0x5d/0x100
[  277.595506]  module_zstd_decompress+0xe1/0x2a0
[  277.596158]  module_decompress+0x37/0xc0
[  277.596720]  init_module_from_file+0xd0/0x100
[  277.597384]  idempotent_init_module+0x11c/0x2b0
[  277.597617]  __x64_sys_finit_module+0x64/0xd0
[  277.597823]  x64_sys_call+0x1d6e/0x25c0
[  277.597969]  do_syscall_64+0x7f/0x180
[  277.598098]  ? irqentry_exit_to_user_mode+0x7b/0x260
[  277.598257]  ? irqentry_exit+0x43/0x50
[  277.598375]  entry_SYSCALL_64_after_hwframe+0x73/0x7b
[  277.598530] RIP: 0033:0x7c9dc1b5a25d
[  277.598867] Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 8b bb 0d 00 f7 d8 64 89 01 48
[  277.601831] RSP: 002b:00007fff26524238 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[  277.602176] RAX: ffffffffffffffda RBX: 000058d8cd8cad50 RCX: 00007c9dc1b5a25d
[  277.602417] RDX: 0000000000000004 RSI: 00007c9dc1c9707d RDI: 000000000000001e
[  277.602728] RBP: 00007fff265242f0 R08: 0000000000000040 R09: 00007fff26524280
[  277.603081] R10: 00007c9dc1c36b20 R11: 0000000000000246 R12: 00007c9dc1c9707d
[  277.603347] R13: 0000000000020000 R14: 000058d8cd8de7b0 R15: 000058d8cd8df5d0
[  277.603620]  </TASK>

pit is fine.
Signed-off-by: Thierry Laurion <insurgo@riseup.net>
Signed-off-by: Thierry Laurion <insurgo@riseup.net>
Copy link
Collaborator

@JonathonHall-Purism JonathonHall-Purism left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works fine for me on Qemu/KVM as well 👍

@tlaurion tlaurion marked this pull request as draft May 21, 2024 13:54
@tlaurion
Copy link
Collaborator Author

Unfortunately, this is side effect of qubes tsc mixed in qemu q35 coreboot not implementing acpi correctly. So the real solution here, until coreboot fixes acpi, is probably to add acpi=off but i'm still struggling with the proper parameters.

Putting as draft since there is crazy variation in behavior across boots.

SRC https://review.coreboot.org/c/coreboot/+/82558

…se ttyS0 console after systemd not getting access to it... This is a mess.

Signed-off-by: Thierry Laurion <insurgo@riseup.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants