mysql crash during resiliency test

[jd-zhang]

*Issue migrated from trac ticket # 886 www.kunlunbase.com *

component: kunlun-storage | priority: major

2022-07-08 16:08:20: zhangjindong@zettadb.com created the issue

---

info:

kunlun@192.168.0.110:
gdb --core=resiliencybase/storage_datadir/instance_data/data_dir_path/5301/dbdata_raw/data/core.mysqld.24254.1657264940 resiliencybase/program_binaries/kunlun-storage-0.9.2/bin/mysqld
(gdb) bt
#0  __pthread_kill (threadid=<optimized out>, signo=11) at ../sysdeps/unix/sysv/linux/pthread_kill.c:56
#1  0x000000000510e4c0 in my_write_core (sig=11) at /home/kunlun/debugbuild/percona-server/mysys/stacktrace.cc:409
#2  0x0000000003e365b5 in handle_fatal_signal (sig=11) at /home/kunlun/debugbuild/percona-server/sql/signal_handler.cc:194
#3  <signal handler called>
#4  je_bitmap_unset (binfo=0x7f20641dfa00, bit=0, bitmap=0x7f20bb8a38d8) at include/jemalloc/internal/bitmap.h:156
#5  arena_run_reg_dalloc (ptr=<optimized out>, run=0x7f203600f000) at src/arena.c:323
#6  je_arena_dalloc_bin_locked (arena=0x7f2062cc0300, chunk=0x7f2036000000, ptr=<optimized out>, mapelm=<optimized out>) at src/arena.c:1893
#7  0x00007f2063fcd137 in je_tcache_bin_flush_small (tbin=0x7f20363e20c8, binind=5, rem=0, tcache=0x7f20363e2000) at src/tcache.c:127
#8  0x00007f2063fcdaf8 in je_tcache_destroy (tcache=0x7f20363e2000) at src/tcache.c:325
#9  0x00007f2063fcdec3 in je_tcache_thread_cleanup (arg=<optimized out>) at src/tcache.c:396
#10 0x00007f2063f6f5a1 in __nptl_deallocate_tsd () at pthread_create.c:301
#11 0x00007f2063f7062a in __nptl_deallocate_tsd () at pthread_create.c:256
#12 start_thread (arg=<optimized out>) at pthread_create.c:488
#13 0x00007f206328e293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95



kunlun@192.168.0.100:
gdb --core=resiliencybase/storage_datadir/instance_data/data_dir_path/5302/dbdata_raw/data/core.mysqld.37021.1657265561  resiliencybase/program_binaries/kunlun-storage-0.9.2/bin/mysqld
(gdb) bt
#0  __pthread_kill (threadid=<optimized out>, signo=6) at ../sysdeps/unix/sysv/linux/pthread_kill.c:56
#1  0x000000000510e4c0 in my_write_core (sig=6) at /home/kunlun/debugbuild/percona-server/mysys/stacktrace.cc:409
#2  0x0000000003e365b5 in handle_fatal_signal (sig=6) at /home/kunlun/debugbuild/percona-server/sql/signal_handler.cc:194
#3  <signal handler called>
#4  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#5  0x00007ff3f6711859 in __GI_abort () at abort.c:79
#6  0x00007ff3f6aeb911 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#7  0x00007ff3f6af738c in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#8  0x00007ff3f6af73f7 in std::terminate() () from /lib/x86_64-linux-gnu/libstdc++.so.6
#9  0x00007ff3f6af76a9 in __cxa_throw () from /lib/x86_64-linux-gnu/libstdc++.so.6
#10 0x0000000003d4dc7d in UContext::fullsync_jumpout (this=0x7ff39b075800) at /home/kunlun/debugbuild/percona-server/sql/fullsync.cc:1404
#11 0x0000000003bf5867 in dispatch_command (thd=0x7ff39beed000, com_data=0x7ff3db8dd210, command=COM_QUERY) at /home/kunlun/debugbuild/percona-server/sql/sql_parse.cc:2424
#12 0x0000000003bf2041 in do_command (thd=0x7ff39beed000) at /home/kunlun/debugbuild/percona-server/sql/sql_parse.cc:1412
#13 0x0000000003d4cde7 in uctx_call_do_command (thd=0x7ff39beed000) at /home/kunlun/debugbuild/percona-server/sql/fullsync.cc:1049
#14 0x0000000003db023e in threadpool_process_request (thd=0x7ff39beed000) at /home/kunlun/debugbuild/percona-server/sql/threadpool_common.cc:264
#15 0x0000000003df19af in handle_event (connection=0x7ff39bd2e260) at /home/kunlun/debugbuild/percona-server/sql/threadpool_unix.cc:1855
#16 0x0000000003df1dda in worker_main (param=0x89bc600 <all_groups+8064>) at /home/kunlun/debugbuild/percona-server/sql/threadpool_unix.cc:1982
#17 0x00000000058f4b1e in pfs_spawn_thread (arg=0x7ff3a4563fa0) at /home/kunlun/debugbuild/percona-server/storage/perfschema/pfs.cc:2899
#18 0x00007ff3f74f1609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#19 0x00007ff3f680e163 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95