Kernel bug on the daily tests with debug kernel #2411
Description
Apr 25 06:39:07 192.168.122.209 [53316.444242] Start test: t_stress.test_stress.TestTdbStressHttp.test_cache_eviction_tdb
Apr 25 06:39:17 192.168.122.209 [53326.477389] ------------[ cut here ]------------
Apr 25 06:39:17 192.168.122.209 [53326.478527] kernel BUG at /root/tempesta/db/core/htrie.c:177!
Apr 25 06:39:17 192.168.122.209 [53326.480204] invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC KASAN NOPTI
Apr 25 06:39:17 192.168.122.209 [53326.481972] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G OE 5.10.35.tfw-39c65d3-debug #1
Apr 25 06:39:17 192.168.122.209 [53326.484392] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
Apr 25 06:39:17 192.168.122.209 [53326.486911] RIP: 0010:tdb_put_blk+0x16f/0x190 [tempesta_db]
Apr 25 06:39:17 192.168.122.209 [53326.488412] Code: 24 20 e8 b4 c7 e9 eb 4c 89 f7 49 c7 44 24 20 01 00 00 00 e8 a3 c7 e9 eb 49 c7 06 00 00 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 <0f> 0b 4c 89 6b 28 4c 89 f7 e8 83 c7 e9 eb 49 c7 06 00 00 00 00 eb
Apr 25 06:39:17 192.168.122.209 [53326.490543] RSP: 0018:ffff88814c088db8 EFLAGS: 00010297
Apr 25 06:39:17 192.168.122.209 [53326.491176] RAX: 00000000ffffffff RBX: ffff88813aa00000 RCX: ffffffffc19f8b05
Apr 25 06:39:17 192.168.122.209 [53326.491999] RDX: ffffed10293ffeb0 RSI: 0000000000000004 RDI: ffff888149fff578
Apr 25 06:39:17 192.168.122.209 [53326.492824] RBP: ffff88814c088de0 R08: 0000000000000001 R09: ffff888149fff57b
Apr 25 06:39:17 192.168.122.209 [53326.493652] R10: ffffed10293ffeaf R11: 0000000000000001 R12: ffff888149fff578
Apr 25 06:39:17 192.168.122.209 [53326.494477] R13: 000000000f55e000 R14: ffff88814c088ff0 R15: 000000000f55f000
Apr 25 06:39:17 192.168.122.209 [53326.495309] FS: 0000000000000000(0000) GS:ffff88814c080000(0000) knlGS:0000000000000000
Apr 25 06:39:17 192.168.122.209 [53326.496238] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Apr 25 06:39:17 192.168.122.209 [53326.496918] CR2: 00007fa09cfcb710 CR3: 0000000117e14005 CR4: 0000000000770ee0
Apr 25 06:39:17 192.168.122.209 [53326.497746] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Apr 25 06:39:17 192.168.122.209 [53326.498575] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Apr 25 06:39:17 192.168.122.209 [53326.499260] PKRU: 55555554
Apr 25 06:39:17 192.168.122.209 [53326.499515] Call Trace:
Apr 25 06:39:17 192.168.122.209 [53326.499744] <IRQ>
Apr 25 06:39:17 192.168.122.209 [53326.499941] tdb_alloc_data+0x134/0x190 [tempesta_db]
Apr 25 06:39:17 192.168.122.209 [53326.500397] tdb_htrie_insert+0x2d1/0x820 [tempesta_db]
Apr 25 06:39:17 192.168.122.209 [53326.500893] ? __cache_entry_size+0x676/0x8f0 [tempesta_fw]
Apr 25 06:39:17 192.168.122.209 [53326.501414] ? tfw_cache_entry_key_eq+0x300/0x300 [tempesta_fw]
Apr 25 06:39:17 192.168.122.209 [53326.501937] ? tdb_htrie_remove+0x1a0/0x1a0 [tempesta_db]
Apr 25 06:39:17 192.168.122.209 [53326.502447] ? tfw_str_next_str_val+0x157/0x200 [tempesta_fw]
Apr 25 06:39:17 192.168.122.209 [53326.502984] ? tfw_str_to_cstr+0x110/0x110 [tempesta_fw]
Apr 25 06:39:17 192.168.122.209 [53326.503493] ? tfw_cache_entry_key_eq+0x300/0x300 [tempesta_fw]
Apr 25 06:39:17 192.168.122.209 [53326.504016] tdb_entry_alloc_unique+0x50/0x80 [tempesta_db]
Apr 25 06:39:17 192.168.122.209 [53326.504537] __cache_add_node+0x14d/0x260 [tempesta_fw]
Apr 25 06:39:17 192.168.122.209 [53326.505027] ? tfw_cache_copy_resp+0x24f0/0x24f0 [tempesta_fw]
Apr 25 06:39:17 192.168.122.209 [53326.505595] ? __bitmap_weight+0x62/0xa0
Apr 25 06:39:17 192.168.122.209 [53326.505971] tfw_cache_do_action+0xbf9/0xe10 [tempesta_fw]
Apr 25 06:39:17 192.168.122.209 [53326.506484] ? tfw_http_req_redir+0x890/0x890 [tempesta_fw]
Apr 25 06:39:17 192.168.122.209 [53326.507002] ? __flush_tlb_all+0x37/0x50
Apr 25 06:39:17 192.168.122.209 [53326.507361] ? __kernel_map_pages+0x108/0x180
Apr 25 06:39:17 192.168.122.209 [53326.507751] ? set_direct_map_default_noflush+0x130/0x130
Apr 25 06:39:17 192.168.122.209 [53326.508251] ? __cache_add_node+0x260/0x260 [tempesta_fw]
Apr 25 06:39:17 192.168.122.209 [53326.508760] ? tfw_gfsm_move+0xc0/0x310 [tempesta_fw]
Apr 25 06:39:17 192.168.122.209 [53326.509234] ? frang_resp_handler+0x21a/0x6d0 [tempesta_fw]
Apr 25 06:39:17 192.168.122.209 [53326.509745] ? __kasan_check_write+0x14/0x20
Apr 25 06:39:17 192.168.122.209 [53326.510148] ? _raw_spin_lock+0x7b/0xd0
Apr 25 06:39:17 192.168.122.209 [53326.510526] ? frang_http_req_incomplete_hdrs_check.isra.0+0x280/0x280 [tempesta_fw]
Apr 25 06:39:17 192.168.122.209 [53326.511224] ? tfw_http_req_zap_error+0x22/0x1b0 [tempesta_fw]
Apr 25 06:39:17 192.168.122.209 [53326.511748] ? get_page_from_freelist+0x2f3f/0x3140
Apr 25 06:39:17 192.168.122.209 [53326.512206] tfw_cache_process+0x1e3/0x750 [tempesta_fw]
Apr 25 06:39:17 192.168.122.209 [53326.512705] ? tfw_http_req_redir+0x890/0x890 [tempesta_fw]
Apr 25 06:39:17 192.168.122.209 [53326.513220] ? tfw_cache_put_entry+0x40/0x40 [tempesta_fw]
Apr 25 06:39:17 192.168.122.209 [53326.513731] ? tfw_gfsm_move+0xc0/0x310 [tempesta_fw]
Apr 25 06:39:17 192.168.122.209 [53326.514205] tfw_http_resp_cache+0x2cc/0x3d0 [tempesta_fw]
Apr 25 06:39:17 192.168.122.209 [53326.514723] ? tfw_http_resp_gfsm+0xf0/0xf0 [tempesta_fw]
Apr 25 06:39:17 192.168.122.209 [53326.515227] tfw_http_msg_process_generic+0x907/0xc70 [tempesta_fw]
Apr 25 06:39:17 192.168.122.209 [53326.515802] ? __ip_finish_output+0x16b/0x340
Apr 25 06:39:17 192.168.122.209 [53326.516216] ? tfw_http_req_process+0x11a0/0x11a0 [tempesta_fw]
Apr 25 06:39:17 192.168.122.209 [53326.516747] ? __kasan_check_write+0x14/0x20
Apr 25 06:39:17 192.168.122.209 [53326.517139] ? pg_skb_alloc+0x60e/0x680
Apr 25 06:39:17 192.168.122.209 [53326.517487] ? __kasan_check_write+0x14/0x20
Apr 25 06:39:17 192.168.122.209 [53326.517868] ? __alloc_skb+0x278/0x350
Apr 25 06:39:17 192.168.122.209 [53326.518211] ? __kasan_check_write+0x14/0x20
Apr 25 06:39:17 192.168.122.209 [53326.518624] ? __coalesce_frag+0x136/0x370 [tempesta_fw]
Apr 25 06:39:17 192.168.122.209 [53326.519120] tfw_http_msg_process+0x80/0xc0 [tempesta_fw]
Apr 25 06:39:17 192.168.122.209 [53326.519602] ? skb_release_data+0x8c/0x320
Apr 25 06:39:17 192.168.122.209 [53326.519997] tfw_connection_recv+0x12e/0x250 [tempesta_fw]
Apr 25 06:39:17 192.168.122.209 [53326.520511] ? tfw_connection_send+0x70/0x70 [tempesta_fw]
Apr 25 06:39:17 192.168.122.209 [53326.521024] ? ss_skb_unroll+0x3d6/0x540 [tempesta_fw]
Apr 25 06:39:17 192.168.122.209 [53326.521510] ss_tcp_process_data+0x436/0x8b0 [tempesta_fw]
Apr 25 06:39:17 192.168.122.209 [53326.522025] ? __sk_close_locked+0x100/0x100 [tempesta_fw]
Apr 25 06:39:17 192.168.122.209 [53326.522513] ? __kasan_check_write+0x14/0x20
Apr 25 06:39:17 192.168.122.209 [53326.522894] ? sk_reset_timer+0x37/0x70
Apr 25 06:39:17 192.168.122.209 [53326.523268] ss_tcp_data_ready+0x7b/0x150 [tempesta_fw]
Apr 25 06:39:17 192.168.122.209 [53326.523743] tcp_data_ready+0x73/0x210
Apr 25 06:39:17 192.168.122.209 [53326.524083] tcp_rcv_established+0x95b/0xb40
Apr 25 06:39:17 192.168.122.209 [53326.524469] ? tcp_data_queue+0x1d80/0x1d80
Apr 25 06:39:17 192.168.122.209 [53326.524846] ? __kasan_check_read+0x11/0x20
Apr 25 06:39:17 192.168.122.209 [53326.525215] tcp_v4_do_rcv+0x25f/0x360
Apr 25 06:39:17 192.168.122.209 [53326.525567] tcp_v4_rcv+0x1600/0x1770
Apr 25 06:39:17 192.168.122.209 [53326.525898] ? tcp_v4_early_demux+0x2f0/0x2f0
Apr 25 06:39:17 192.168.122.209 [53326.526287] ip_protocol_deliver_rcu+0x46/0x2e0
Apr 25 06:39:17 192.168.122.209 [53326.526699] ip_local_deliver_finish+0xc6/0xe0
Apr 25 06:39:17 192.168.122.209 [53326.527098] ip_local_deliver+0x1f5/0x210
Apr 25 06:39:17 192.168.122.209 [53326.527461] ? ip_local_deliver_finish+0xe0/0xe0
Apr 25 06:39:17 192.168.122.209 [53326.527887] ? nf_nat_inet_fn+0x2b3/0x440 [nf_nat]
Apr 25 06:39:17 192.168.122.209 [53326.528321] ? ip_protocol_deliver_rcu+0x2e0/0x2e0
Apr 25 06:39:17 192.168.122.209 [53326.528753] ? ip_rcv_finish_core.constprop.0+0x17c/0x890
Apr 25 06:39:17 192.168.122.209 [53326.529217] ip_rcv_finish+0xcf/0xf0
Apr 25 06:39:17 192.168.122.209 [53326.529556] ip_rcv+0x16d/0x180
Apr 25 06:39:17 192.168.122.209 [53326.529844] ? ip_local_deliver+0x210/0x210
Apr 25 06:39:17 192.168.122.209 [53326.530219] ? ip_rcv_finish_core.constprop.0+0x890/0x890
Apr 25 06:39:17 192.168.122.209 [53326.530700] ? ip_local_deliver+0x210/0x210
Apr 25 06:39:17 192.168.122.209 [53326.531080] __netif_receive_skb_one_core+0x132/0x140
Apr 25 06:39:17 192.168.122.209 [53326.531534] ? napi_gro_receive+0x260/0x260
Apr 25 06:39:17 192.168.122.209 [53326.531909] ? __kasan_check_write+0x14/0x20
Apr 25 06:39:17 192.168.122.209 [53326.532291] ? _raw_spin_lock+0x7b/0xd0
Apr 25 06:39:17 192.168.122.209 [53326.532685] __netif_receive_skb+0x26/0xb0
Apr 25 06:39:17 192.168.122.209 [53326.533055] process_backlog+0xfe/0x290
Apr 25 06:39:17 192.168.122.209 [53326.533405] net_rx_action+0x287/0x6b0
Apr 25 06:39:17 192.168.122.209 [53326.533741] ? napi_complete_done+0x2c0/0x2c0
Apr 25 06:39:17 192.168.122.209 [53326.534143] ? switch_fpu_return+0x120/0x120
Apr 25 06:39:17 192.168.122.209 [53326.534535] __do_softirq+0xfe/0x383
Apr 25 06:39:17 192.168.122.209 [53326.534858] asm_call_irq_on_stack+0xf/0x20
Apr 25 06:39:17 192.168.122.209 [53326.535229] </IRQ>
Apr 25 06:39:17 192.168.122.209 [53326.535437] do_softirq_own_stack+0x3d/0x50
Apr 25 06:39:17 192.168.122.209 [53326.535824] irq_exit_rcu+0xc7/0x120
Apr 25 06:39:17 192.168.122.209 [53326.536156] sysvec_call_function_single+0x3a/0x90
Apr 25 06:39:17 192.168.122.209 [53326.536584] asm_sysvec_call_function_single+0x12/0x20
Apr 25 06:39:17 192.168.122.209 [53326.537064] RIP: 0010:native_safe_halt+0xe/0x10
Apr 25 06:39:17 192.168.122.209 [53326.537473] Code: 49 8b 04 24 a8 08 0f 85 71 ff ff ff e9 0f ff ff ff cc cc cc cc cc cc cc cc cc cc cc e9 07 00 00 00 0f 00 2d d4 62 51 00 fb f4 <c3> 90 e9 07 00 00 00 0f 00 2d c4 62 51 00 f4 c3 cc cc 0f 1f 44 00
Apr 25 06:39:17 192.168.122.209 [53326.539077] RSP: 0018:ffff888100f37db8 EFLAGS: 00000202
Apr 25 06:39:17 192.168.122.209 [53326.539540] RAX: ffffffffae7426a0 RBX: ffff888100f2a080 RCX: ffffffffae72c475
Apr 25 06:39:17 192.168.122.209 [53326.540162] RDX: 00000000134783a6 RSI: 0000000000000004 RDI: ffff88814c0b5040
Apr 25 06:39:17 192.168.122.209 [53326.540791] RBP: ffff888100f37dc0 R08: 0000000000000001 R09: ffff88814c0b5043
Apr 25 06:39:17 192.168.122.209 [53326.541437] R10: ffffed1029816a08 R11: 0000000000000001 R12: 0000000000000001
Apr 25 06:39:17 192.168.122.209 [53326.542057] R13: ffffffffafee9360 R14: 0000000000000000 R15: 0000000000000000
Apr 25 06:39:17 192.168.122.209 [53326.542706] ? __cpuidle_text_start+0x8/0x8
Apr 25 06:39:17 192.168.122.209 [53326.543085] ? rcu_eqs_enter.constprop.0+0x85/0xa0
Apr 25 06:39:17 192.168.122.209 [53326.543515] ? default_idle+0xe/0x20
Apr 25 06:39:17 192.168.122.209 [53326.543836] arch_cpu_idle+0x15/0x20
Apr 25 06:39:17 192.168.122.209 [53326.544161] default_idle_call+0x66/0x160
Apr 25 06:39:17 192.168.122.209 [53326.544534] do_idle+0x36f/0x430
Apr 25 06:39:17 192.168.122.209 [53326.544829] ? asm_sysvec_call_function_single+0x12/0x20
Apr 25 06:39:17 192.168.122.209 [53326.545302] ? arch_cpu_idle_exit+0x40/0x40
Apr 25 06:39:17 192.168.122.209 [53326.545697] cpu_startup_entry+0x20/0x30
Apr 25 06:39:17 192.168.122.209 [53326.546063] start_secondary+0x1c1/0x210
Apr 25 06:39:17 192.168.122.209 [53326.546420] ? set_cpu_sibling_map+0xd70/0xd70
Apr 25 06:39:17 192.168.122.209 [53326.546826] ? set_bringup_idt_handler.constprop.0+0x84/0x90
Apr 25 06:39:17 192.168.122.209 [53326.547327] ? start_cpu0+0xc/0xc
Apr 25 06:39:17 192.168.122.209 [53326.547630] secondary_startup_64_no_verify+0xb0/0xbb
Apr 25 06:39:17 192.168.122.209 [53326.548083] Modules linked in: tempesta_fw(OE) tempesta_db(OE) tempesta_tls(OE) tempesta_lib(OE) xt_mark tcp_diag inet_diag xt_nat xt_tcpudp veth ib_core sha256_ssse3 sha512_ssse3 nvme_tcp nvme_fabrics nvme_core vhost_vsock vmw_vsock_virtio_transport_common vhost vhost_iotlb vsock tls xt_conntrack xt_MASQUERADE nf_conntrack_netlink xfrm_user xfrm_algo xt_addrtype nft_compat nft_masq nft_counter nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 bridge stp llc nf_tables overlay cfg80211 intel_rapl_msr intel_rapl_common isst_if_common sch_fq_codel nfit kvm_intel kvm rapl i2c_i801 i2c_smbus lpc_ich binfmt_misc joydev input_leds mac_hid serio_raw dm_multipath netconsole msr efi_pstore nfnetlink dmi_sysfs qemu_fw_cfg ip_tables x_tables autofs4 btrfs blake2b_generic zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel cirrus cryp
Apr 25 06:39:17 192.168.122.209 to_simd
Apr 25 06:39:17 192.168.122.209 [53326.548339] cryptd glue_helper drm_kms_helper syscopyarea sysfillrect psmouse sysimgblt virtio_net fb_sys_fops cec ahci net_failover virtio_blk libahci failover drm virtio_rng [last unloaded: tempesta_lib]
Apr 25 06:39:17 192.168.122.209 [53326.557564] ---[ end trace db572a37a6b0a4a2 ]---
Apr 25 06:39:17 192.168.122.209 [53326.557987] RIP: 0010:tdb_put_blk+0x16f/0x190 [tempesta_db]
Apr 25 06:39:17 192.168.122.209 [53326.558497] Code: 24 20 e8 b4 c7 e9 eb 4c 89 f7 49 c7 44 24 20 01 00 00 00 e8 a3 c7 e9 eb 49 c7 06 00 00 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 <0f> 0b 4c 89 6b 28 4c 89 f7 e8 83 c7 e9 eb 49 c7 06 00 00 00 00 eb
Apr 25 06:39:17 192.168.122.209 [53326.560134] RSP: 0018:ffff88814c088db8 EFLAGS: 00010297
Apr 25 06:39:17 192.168.122.209 [53326.560607] RAX: 00000000ffffffff RBX: ffff88813aa00000 RCX: ffffffffc19f8b05
Apr 25 06:39:17 192.168.122.209 [53326.561233] RDX: ffffed10293ffeb0 RSI: 0000000000000004 RDI: ffff888149fff578
Apr 25 06:39:17 192.168.122.209 [53326.561865] RBP: ffff88814c088de0 R08: 0000000000000001 R09: ffff888149fff57b
Apr 25 06:39:17 192.168.122.209 [53326.573016] R10: ffffed10293ffeaf R11: 0000000000000001 R12: ffff888149fff578
Apr 25 06:39:17 192.168.122.209 [53326.573684] R13: 000000000f55e000 R14: ffff88814c088ff0 R15: 000000000f55f000
Apr 25 06:39:17 192.168.122.209 [53326.574320] FS: 0000000000000000(0000) GS:ffff88814c080000(0000) knlGS:0000000000000000
Apr 25 06:39:17 192.168.122.209 [53326.575037] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Apr 25 06:39:17 192.168.122.209 [53326.575551] CR2: 00007fa09cfcb710 CR3: 0000000117e14005 CR4: 0000000000770ee0
Apr 25 06:39:17 192.168.122.209 [53326.576188] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Apr 25 06:39:17 192.168.122.209 [53326.576815] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Apr 25 06:39:17 192.168.122.209 [53326.577450] PKRU: 55555554
Apr 25 06:39:17 192.168.122.209 [53326.577724] Kernel panic - not syncing: Fatal exception in interrupt
Apr 25 06:39:17 192.168.122.209 [53326.578538] Kernel Offset: 0x2c400000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
Apr 25 06:39:17 192.168.122.209 [53326.579521] Rebooting in 1 seconds..
Metadata
Metadata
Assignees
Labels
No labels