Concurrent DDL in a single core eloqdoc server

[zhangh43]

** Reproduce **

EloqDoc server spec: 1 core

Using YCSB load with 128 concurrency, the DDL will hung.

[xiexiaoy]

(gdb) thread apply all bt

Thread 48 (Thread 0x7fc4bbb326c0 (LWP 1667212) "eloqdoc"):
#0  __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x407fdad5680) at ./nptl/futex-internal.c:57
eloqdata/tx_service#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x407fdad5680) at ./nptl/futex-internal.c:87
eloqdata/tx_service#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x407fdad5680, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
eloqdata/tx_service#3  0x00007fc525a8e5f8 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x407fdad5630, cond=0x407fdad5658) at ./nptl/pthread_cond_wait.c:503
eloqdata/tx_service#4  ___pthread_cond_wait (cond=0x407fdad5658, mutex=0x407fdad5630) at ./nptl/pthread_cond_wait.c:627
eloqdata/tx_service#5  0x00007fc51a3a8bb9 in std::condition_variable::wait<txservice::remote::CcStreamSender::ResendMessageToNode()::<lambda()> >(std::unique_lock<std::mutex> &, struct {...}) (this=0x407fdad5658, __lock=..., __p=...) at /usr/include/c++/12/condition_variable:102
eloqdata/tx_service#6  0x00007fc51a3a5ada in txservice::remote::CcStreamSender::ResendMessageToNode (this=0x407fdad5400) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/tx_service/src/remote/cc_stream_sender.cpp:644
eloqdata/tx_service#7  0x00007fc51a3a2a0f in operator() (__closure=0x407fce8a598) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/tx_service/src/remote/cc_stream_sender.cpp:60
eloqdata/tx_service#8  0x00007fc51a3ac1a1 in std::__invoke_impl<void, txservice::remote::CcStreamSender::CcStreamSender(moodycamel::ConcurrentQueue<std::unique_ptr<txservice::remote::CcMessage> >&)::<lambda()> >(std::__invoke_other, struct {...} &&) (__f=...) at /usr/include/c++/12/bits/invoke.h:61
eloqdata/tx_service#9  0x00007fc51a3ac111 in std::__invoke<txservice::remote::CcStreamSender::CcStreamSender(moodycamel::ConcurrentQueue<std::unique_ptr<txservice::remote::CcMessage> >&)::<lambda()> >(struct {...} &&) (__fn=...) at /usr/include/c++/12/bits/invoke.h:96
eloqdata/tx_service#10 0x00007fc51a3ac092 in std::thread::_Invoker<std::tuple<txservice::remote::CcStreamSender::CcStreamSender(moodycamel::ConcurrentQueue<std::unique_ptr<txservice::remote::CcMessage> >&)::<lambda()> > >::_M_invoke<0>(std::_Index_tuple<0>) (this=0x407fce8a598) at /usr/include/c++/12/bits/std_thread.h:279
eloqdata/tx_service#11 0x00007fc51a3ac04a in std::thread::_Invoker<std::tuple<txservice::remote::CcStreamSender::CcStreamSender(moodycamel::ConcurrentQueue<std::unique_ptr<txservice::remote::CcMessage> >&)::<lambda()> > >::operator()(void) (this=0x407fce8a598) at /usr/include/c++/12/bits/std_thread.h:286
eloqdata/tx_service#12 0x00007fc51a3ac00a in std::thread::_State_impl<std::thread::_Invoker<std::tuple<txservice::remote::CcStreamSender::CcStreamSender(moodycamel::ConcurrentQueue<std::unique_ptr<txservice::remote::CcMessage> >&)::<lambda()> > > >::_M_run(void) (this=0x407fce8a590) at /usr/include/c++/12/bits/std_thread.h:231
eloqdata/tx_service#13 0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#14 0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#15 0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 47 (Thread 0x7fc4bd87f6c0 (LWP 1667211) "eloqdoc"):
#0  __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x7fc4bd870d90, op=137, expected=0, futex_word=0x407fdad5540) at ./nptl/futex-internal.c:57
eloqdata/tx_service#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x7fc4bd870d90, clockid=0, expected=0, futex_word=0x407fdad5540) at ./nptl/futex-internal.c:87
eloqdata/tx_service#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x407fdad5540, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x7fc4bd870d90, private=private@entry=0) at ./nptl/futex-internal.c:139
eloqdata/tx_service#3  0x00007fc525a8eead in __pthread_cond_wait_common (abstime=<optimized out>, clockid=<optimized out>, mutex=0x407fdad54f0, cond=0x407fdad5518) at ./nptl/pthread_cond_wait.c:503
eloqdata/tx_service#4  ___pthread_cond_clockwait64 (abstime=<optimized out>, clockid=<optimized out>, mutex=0x407fdad54f0, cond=0x407fdad5518) at ./nptl/pthread_cond_wait.c:691
eloqdata/tx_service#5  ___pthread_cond_clockwait64 (cond=0x407fdad5518, mutex=0x407fdad54f0, clockid=<optimized out>, abstime=<optimized out>) at ./nptl/pthread_cond_wait.c:679
eloqdata/tx_service#6  0x00007fc525963159 in std::__condvar::wait_until (this=0x407fdad5518, __m=..., __clock=1, __abs_time=...) at /usr/include/c++/12/bits/std_mutex.h:169
eloqdata/tx_service#7  0x00007fc52596d162 in std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x407fdad5518, __lock=..., __atime=std::chrono::_V2::steady_clock time_point = {291041508394533ns}) at /usr/include/c++/12/condition_variable:200
eloqdata/tx_service#8  0x00007fc52596a5f1 in std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x407fdad5518, __lock=..., __atime=std::chrono::_V2::steady_clock time_point = {291041508394533ns}) at /usr/include/c++/12/condition_variable:110
eloqdata/tx_service#9  0x00007fc51a3a8ead in std::condition_variable::wait_until<std::chrono::_V2::steady_clock, std::chrono::duration<long int, std::ratio<1, 1000000000> >, txservice::remote::CcStreamSender::ConnectStreams()::<lambda()> >(std::unique_lock<std::mutex> &, const std::chrono::time_point<std::chrono::_V2::steady_clock, std::chrono::duration<long, std::ratio<1, 1000000000> > > &, struct {...}) (this=0x407fdad5518, __lock=..., __atime=std::chrono::_V2::steady_clock time_point = {291041508394533ns}, __p=...) at /usr/include/c++/12/condition_variable:151
eloqdata/tx_service#10 0x00007fc51a3a8ce6 in std::condition_variable::wait_for<long int, std::ratio<1>, txservice::remote::CcStreamSender::ConnectStreams()::<lambda()> >(std::unique_lock<std::mutex> &, const std::chrono::duration<long, std::ratio<1, 1> > &, struct {...}) (this=0x407fdad5518, __lock=..., __rtime=std::chrono::duration = { 1s }, __p=...) at /usr/include/c++/12/condition_variable:174
eloqdata/tx_service#11 0x00007fc51a3a64fa in txservice::remote::CcStreamSender::ConnectStreams (this=0x407fdad5400) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/tx_service/src/remote/cc_stream_sender.cpp:784
eloqdata/tx_service#12 0x00007fc51a3a29f1 in operator() (__closure=0x407fce8aac8) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/tx_service/src/remote/cc_stream_sender.cpp:59
eloqdata/tx_service#13 0x00007fc51a3ac1de in std::__invoke_impl<void, txservice::remote::CcStreamSender::CcStreamSender(moodycamel::ConcurrentQueue<std::unique_ptr<txservice::remote::CcMessage> >&)::<lambda()> >(std::__invoke_other, struct {...} &&) (__f=...) at /usr/include/c++/12/bits/invoke.h:61
eloqdata/tx_service#14 0x00007fc51a3ac164 in std::__invoke<txservice::remote::CcStreamSender::CcStreamSender(moodycamel::ConcurrentQueue<std::unique_ptr<txservice::remote::CcMessage> >&)::<lambda()> >(struct {...} &&) (__fn=...) at /usr/include/c++/12/bits/invoke.h:96
eloqdata/tx_service#15 0x00007fc51a3ac0be in std::thread::_Invoker<std::tuple<txservice::remote::CcStreamSender::CcStreamSender(moodycamel::ConcurrentQueue<std::unique_ptr<txservice::remote::CcMessage> >&)::<lambda()> > >::_M_invoke<0>(std::_Index_tuple<0>) (this=0x407fce8aac8) at /usr/include/c++/12/bits/std_thread.h:279
eloqdata/tx_service#16 0x00007fc51a3ac066 in std::thread::_Invoker<std::tuple<txservice::remote::CcStreamSender::CcStreamSender(moodycamel::ConcurrentQueue<std::unique_ptr<txservice::remote::CcMessage> >&)::<lambda()> > >::operator()(void) (this=0x407fce8aac8) at /usr/include/c++/12/bits/std_thread.h:286
eloqdata/tx_service#17 0x00007fc51a3ac02e in std::thread::_State_impl<std::thread::_Invoker<std::tuple<txservice::remote::CcStreamSender::CcStreamSender(moodycamel::ConcurrentQueue<std::unique_ptr<txservice::remote::CcMessage> >&)::<lambda()> > > >::_M_run(void) (this=0x407fce8aac0) at /usr/include/c++/12/bits/std_thread.h:231
eloqdata/tx_service#18 0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#19 0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#20 0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 46 (Thread 0x7fc4c2e586c0 (LWP 1667210) "eloqdoc"):
#0  syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
eloqdata/tx_service#1  0x00007fc5263fe502 in bthread::futex_wait_private (timeout=0x0, expected=0, addr1=0x7fc4c2e48fd0) at /home/xiexy/github.com/eloqdata/brpc/src/bthread/sys_futex.h:40
eloqdata/tx_service#2  bthread::wait_pthread (pw=..., abstime=<optimized out>, abstime@entry=0x0) at /home/xiexy/github.com/eloqdata/brpc/src/bthread/butex.cpp:153
eloqdata/tx_service#3  0x00007fc5263fe927 in bthread::butex_wait_from_pthread (abstime=0x0, expected_value=1, b=0x40836130040, g=<optimized out>) at /home/xiexy/github.com/eloqdata/brpc/src/bthread/butex.cpp:669
eloqdata/tx_service#4  bthread::butex_wait (arg=arg@entry=0x40836130040, expected_value=expected_value@entry=1, abstime=abstime@entry=0x0) at /home/xiexy/github.com/eloqdata/brpc/src/bthread/butex.cpp:702
eloqdata/tx_service#5  0x00007fc526407d78 in bthread_id_join (id=...) at /home/xiexy/github.com/eloqdata/brpc/src/bthread/id.cpp:529
eloqdata/tx_service#6  0x00007fc5264a3f75 in brpc::Channel::CallMethod (this=0x7fc4c2e49320, method=0x407fd883940, controller_base=0x7fc4c2e49460, request=0x7fc4c2e492f0, response=0x7fc4c2e49270, done=0x0) at /home/xiexy/github.com/eloqdata/brpc/src/brpc/channel.cpp:581
eloqdata/tx_service#7  0x00007fc517b104c1 in braft::rtb::refresh_leader (group="lg0", timeout_ms=1000) at /home/xiexy/github.com/eloqdata/braft/src/braft/route_table.cpp:237
eloqdata/tx_service#8  0x00007fc51a462932 in txlog::LogAgent::RefreshLeader (this=0x407fcb2c408, log_group_id=0, timeout_ms=1000) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/tx_service/tx-log-protos/log_agent.cpp:170
eloqdata/tx_service#9  0x00007fc51a464c5b in txlog::LogAgent::ReplayLog (this=0x407fcb2c408, cc_node_group_id=0, term=1, source_ip="127.0.0.1", source_port=8083, log_group=-1, start_ts=0, interrupt=std::atomic<bool> = { false }, no_replay=false) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/tx_service/tx-log-protos/log_agent.cpp:512
eloqdata/tx_service#10 0x00007fc51b2e70ac in Eloq::MongoLogAgent::ReplayLog (this=0x407fcb2c400, cc_node_group_id=0, term=1, source_ip="127.0.0.1", source_port=8083, log_group=-1, start_ts=0, interrupt=std::atomic<bool> = { false }) at src/mongo/db/modules/eloq/src/base/eloq_log_agent.h:71
eloqdata/tx_service#11 0x00007fc51a3c7f87 in txservice::fault::RecoveryService::ProcessReplayLogTask (this=0x407fdac5780, task=...) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/tx_service/src/fault/log_replay_service.cpp:862
eloqdata/tx_service#12 0x00007fc51a3c4895 in operator() (__closure=0x407fce8a198) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/tx_service/src/fault/log_replay_service.cpp:151
eloqdata/tx_service#13 0x00007fc51a3cd546 in std::__invoke_impl<void, txservice::fault::RecoveryService::RecoveryService(txservice::LocalCcShards&, txservice::TxLog*, std::string, uint16_t)::<lambda()> >(std::__invoke_other, struct {...} &&) (__f=...) at /usr/include/c++/12/bits/invoke.h:61
--Type <RET> for more, q to quit, c to continue without paging--c
eloqdata/tx_service#14 0x00007fc51a3cd509 in std::__invoke<txservice::fault::RecoveryService::RecoveryService(txservice::LocalCcShards&, txservice::TxLog*, std::string, uint16_t)::<lambda()> >(struct {...} &&) (__fn=...) at /usr/include/c++/12/bits/invoke.h:96
eloqdata/tx_service#15 0x00007fc51a3cd4b6 in std::thread::_Invoker<std::tuple<txservice::fault::RecoveryService::RecoveryService(txservice::LocalCcShards&, txservice::TxLog*, std::string, uint16_t)::<lambda()> > >::_M_invoke<0>(std::_Index_tuple<0>) (this=0x407fce8a198) at /usr/include/c++/12/bits/std_thread.h:279
eloqdata/tx_service#16 0x00007fc51a3cd48a in std::thread::_Invoker<std::tuple<txservice::fault::RecoveryService::RecoveryService(txservice::LocalCcShards&, txservice::TxLog*, std::string, uint16_t)::<lambda()> > >::operator()(void) (this=0x407fce8a198) at /usr/include/c++/12/bits/std_thread.h:286
eloqdata/tx_service#17 0x00007fc51a3cd46e in std::thread::_State_impl<std::thread::_Invoker<std::tuple<txservice::fault::RecoveryService::RecoveryService(txservice::LocalCcShards&, txservice::TxLog*, std::string, uint16_t)::<lambda()> > > >::_M_run(void) (this=0x407fce8a190) at /usr/include/c++/12/bits/std_thread.h:231
eloqdata/tx_service#18 0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#19 0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#20 0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 45 (Thread 0x7fc4c4ba56c0 (LWP 1667209) "eloqdoc"):
#0  __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x407fcaf3380) at ./nptl/futex-internal.c:57
eloqdata/tx_service#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x407fcaf3380) at ./nptl/futex-internal.c:87
eloqdata/tx_service#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x407fcaf3380, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
eloqdata/tx_service#3  0x00007fc525a8e5f8 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x407fcaf3330, cond=0x407fcaf3358) at ./nptl/pthread_cond_wait.c:503
eloqdata/tx_service#4  ___pthread_cond_wait (cond=0x407fcaf3358, mutex=0x407fcaf3330) at ./nptl/pthread_cond_wait.c:627
eloqdata/tx_service#5  0x00007fc519e6ca8d in std::condition_variable::wait<txservice::TxWorkerPool::TxWorkerPool(size_t)::<lambda(size_t)>::<lambda()> >(std::unique_lock<std::mutex> &, struct {...}) (this=0x407fcaf3358, __lock=..., __p=...) at /usr/include/c++/12/condition_variable:102
eloqdata/tx_service#6  0x00007fc519e6c3f1 in operator() (__closure=0x407fde1f790, worker_id=0) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/tx_service/src/tx_worker_pool.cpp:41
eloqdata/tx_service#7  0x00007fc519e6cf88 in std::__invoke_impl<void, txservice::TxWorkerPool::TxWorkerPool(size_t)::<lambda(size_t)>, long unsigned int>(std::__invoke_other, struct {...} &&) (__f=...) at /usr/include/c++/12/bits/invoke.h:61
eloqdata/tx_service#8  0x00007fc519e6cf2d in std::__invoke<txservice::TxWorkerPool::TxWorkerPool(size_t)::<lambda(size_t)>, long unsigned int>(struct {...} &&) (__fn=...) at /usr/include/c++/12/bits/invoke.h:96
eloqdata/tx_service#9  0x00007fc519e6ce9d in std::thread::_Invoker<std::tuple<txservice::TxWorkerPool::TxWorkerPool(size_t)::<lambda(size_t)>, long unsigned int> >::_M_invoke<0, 1>(std::_Index_tuple<0, 1>) (this=0x407fde1f788) at /usr/include/c++/12/bits/std_thread.h:279
eloqdata/tx_service#10 0x00007fc519e6ce56 in std::thread::_Invoker<std::tuple<txservice::TxWorkerPool::TxWorkerPool(size_t)::<lambda(size_t)>, long unsigned int> >::operator()(void) (this=0x407fde1f788) at /usr/include/c++/12/bits/std_thread.h:286
eloqdata/tx_service#11 0x00007fc519e6ce3a in std::thread::_State_impl<std::thread::_Invoker<std::tuple<txservice::TxWorkerPool::TxWorkerPool(size_t)::<lambda(size_t)>, long unsigned int> > >::_M_run(void) (this=0x407fde1f780) at /usr/include/c++/12/bits/std_thread.h:231
eloqdata/tx_service#12 0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#13 0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#14 0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 44 (Thread 0x7fc4c68f26c0 (LWP 1667208) "eloqdoc"):
#0  __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x407fcaf5f40) at ./nptl/futex-internal.c:57
eloqdata/tx_service#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x407fcaf5f40) at ./nptl/futex-internal.c:87
eloqdata/tx_service#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x407fcaf5f40, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
eloqdata/tx_service#3  0x00007fc525a8e5f8 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x407fcaf5ef0, cond=0x407fcaf5f18) at ./nptl/pthread_cond_wait.c:503
eloqdata/tx_service#4  ___pthread_cond_wait (cond=0x407fcaf5f18, mutex=0x407fcaf5ef0) at ./nptl/pthread_cond_wait.c:627
eloqdata/tx_service#5  0x00007fc519e6ca8d in std::condition_variable::wait<txservice::TxWorkerPool::TxWorkerPool(size_t)::<lambda(size_t)>::<lambda()> >(std::unique_lock<std::mutex> &, struct {...}) (this=0x407fcaf5f18, __lock=..., __p=...) at /usr/include/c++/12/condition_variable:102
eloqdata/tx_service#6  0x00007fc519e6c3f1 in operator() (__closure=0x407fde1f770, worker_id=0) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/tx_service/src/tx_worker_pool.cpp:41
eloqdata/tx_service#7  0x00007fc519e6cf88 in std::__invoke_impl<void, txservice::TxWorkerPool::TxWorkerPool(size_t)::<lambda(size_t)>, long unsigned int>(std::__invoke_other, struct {...} &&) (__f=...) at /usr/include/c++/12/bits/invoke.h:61
eloqdata/tx_service#8  0x00007fc519e6cf2d in std::__invoke<txservice::TxWorkerPool::TxWorkerPool(size_t)::<lambda(size_t)>, long unsigned int>(struct {...} &&) (__fn=...) at /usr/include/c++/12/bits/invoke.h:96
eloqdata/tx_service#9  0x00007fc519e6ce9d in std::thread::_Invoker<std::tuple<txservice::TxWorkerPool::TxWorkerPool(size_t)::<lambda(size_t)>, long unsigned int> >::_M_invoke<0, 1>(std::_Index_tuple<0, 1>) (this=0x407fde1f768) at /usr/include/c++/12/bits/std_thread.h:279
eloqdata/tx_service#10 0x00007fc519e6ce56 in std::thread::_Invoker<std::tuple<txservice::TxWorkerPool::TxWorkerPool(size_t)::<lambda(size_t)>, long unsigned int> >::operator()(void) (this=0x407fde1f768) at /usr/include/c++/12/bits/std_thread.h:286
eloqdata/tx_service#11 0x00007fc519e6ce3a in std::thread::_State_impl<std::thread::_Invoker<std::tuple<txservice::TxWorkerPool::TxWorkerPool(size_t)::<lambda(size_t)>, long unsigned int> > >::_M_run(void) (this=0x407fde1f760) at /usr/include/c++/12/bits/std_thread.h:231
eloqdata/tx_service#12 0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#13 0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#14 0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 43 (Thread 0x7fc4c863f6c0 (LWP 1667207) "eloqdoc"):
#0  __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x7fc51b7b2f40 <txservice::store::SnapshotManager::Instance()::instance_+96>) at ./nptl/futex-internal.c:57
eloqdata/tx_service#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x7fc51b7b2f40 <txservice::store::SnapshotManager::Instance()::instance_+96>) at ./nptl/futex-internal.c:87
eloqdata/tx_service#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x7fc51b7b2f40 <txservice::store::SnapshotManager::Instance()::instance_+96>, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
eloqdata/tx_service#3  0x00007fc525a8e5f8 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x7fc51b7b2ef0 <txservice::store::SnapshotManager::Instance()::instance_+16>, cond=0x7fc51b7b2f18 <txservice::store::SnapshotManager::Instance()::instance_+56>) at ./nptl/pthread_cond_wait.c:503
eloqdata/tx_service#4  ___pthread_cond_wait (cond=0x7fc51b7b2f18 <txservice::store::SnapshotManager::Instance()::instance_+56>, mutex=0x7fc51b7b2ef0 <txservice::store::SnapshotManager::Instance()::instance_+16>) at ./nptl/pthread_cond_wait.c:627
eloqdata/tx_service#5  0x00007fc51a44b789 in std::condition_variable::wait<txservice::store::SnapshotManager::StandbySyncWorker()::<lambda()> >(std::unique_lock<std::mutex> &, struct {...}) (this=0x7fc51b7b2f18 <txservice::store::SnapshotManager::Instance()::instance_+56>, __lock=..., __p=...) at /usr/include/c++/12/condition_variable:102
eloqdata/tx_service#6  0x00007fc51a447ade in txservice::store::SnapshotManager::StandbySyncWorker (this=0x7fc51b7b2ee0 <txservice::store::SnapshotManager::Instance()::instance_>) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/tx_service/src/store/snapshot_manager.cpp:57
eloqdata/tx_service#7  0x00007fc51a4478f3 in operator() (__closure=0x407fce8a948) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/tx_service/src/store/snapshot_manager.cpp:36
eloqdata/tx_service#8  0x00007fc51a4500ee in std::__invoke_impl<void, txservice::store::SnapshotManager::Start()::<lambda()> >(std::__invoke_other, struct {...} &&) (__f=...) at /usr/include/c++/12/bits/invoke.h:61
eloqdata/tx_service#9  0x00007fc51a4500b1 in std::__invoke<txservice::store::SnapshotManager::Start()::<lambda()> >(struct {...} &&) (__fn=...) at /usr/include/c++/12/bits/invoke.h:96
eloqdata/tx_service#10 0x00007fc51a45005e in std::thread::_Invoker<std::tuple<txservice::store::SnapshotManager::Start()::<lambda()> > >::_M_invoke<0>(std::_Index_tuple<0>) (this=0x407fce8a948) at /usr/include/c++/12/bits/std_thread.h:279
eloqdata/tx_service#11 0x00007fc51a450032 in std::thread::_Invoker<std::tuple<txservice::store::SnapshotManager::Start()::<lambda()> > >::operator()(void) (this=0x407fce8a948) at /usr/include/c++/12/bits/std_thread.h:286
eloqdata/tx_service#12 0x00007fc51a450016 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<txservice::store::SnapshotManager::Start()::<lambda()> > > >::_M_run(void) (this=0x407fce8a940) at /usr/include/c++/12/bits/std_thread.h:231
eloqdata/tx_service#13 0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#14 0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#15 0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 42 (Thread 0x7fc4ca38c6c0 (LWP 1667206) "eloqdoc"):
#0  __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x7fc51b7b3118 <txservice::store::SnapshotManager::Instance()::instance_+568>) at ./nptl/futex-internal.c:57
eloqdata/tx_service#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x7fc51b7b3118 <txservice::store::SnapshotManager::Instance()::instance_+568>) at ./nptl/futex-internal.c:87
eloqdata/tx_service#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x7fc51b7b3118 <txservice::store::SnapshotManager::Instance()::instance_+568>, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
eloqdata/tx_service#3  0x00007fc525a8e5f8 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x7fc51b7b30c8 <txservice::store::SnapshotManager::Instance()::instance_+488>, cond=0x7fc51b7b30f0 <txservice::store::SnapshotManager::Instance()::instance_+528>) at ./nptl/pthread_cond_wait.c:503
eloqdata/tx_service#4  ___pthread_cond_wait (cond=0x7fc51b7b30f0 <txservice::store::SnapshotManager::Instance()::instance_+528>, mutex=0x7fc51b7b30c8 <txservice::store::SnapshotManager::Instance()::instance_+488>) at ./nptl/pthread_cond_wait.c:627
eloqdata/tx_service#5  0x00007fc519e6ca8d in std::condition_variable::wait<txservice::TxWorkerPool::TxWorkerPool(size_t)::<lambda(size_t)>::<lambda()> >(std::unique_lock<std::mutex> &, struct {...}) (this=0x7fc51b7b30f0 <txservice::store::SnapshotManager::Instance()::instance_+528>, __lock=..., __p=...) at /usr/include/c++/12/condition_variable:102
eloqdata/tx_service#6  0x00007fc519e6c3f1 in operator() (__closure=0x407fde1f610, worker_id=0) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/tx_service/src/tx_worker_pool.cpp:41
eloqdata/tx_service#7  0x00007fc519e6cf88 in std::__invoke_impl<void, txservice::TxWorkerPool::TxWorkerPool(size_t)::<lambda(size_t)>, long unsigned int>(std::__invoke_other, struct {...} &&) (__f=...) at /usr/include/c++/12/bits/invoke.h:61
eloqdata/tx_service#8  0x00007fc519e6cf2d in std::__invoke<txservice::TxWorkerPool::TxWorkerPool(size_t)::<lambda(size_t)>, long unsigned int>(struct {...} &&) (__fn=...) at /usr/include/c++/12/bits/invoke.h:96
eloqdata/tx_service#9  0x00007fc519e6ce9d in std::thread::_Invoker<std::tuple<txservice::TxWorkerPool::TxWorkerPool(size_t)::<lambda(size_t)>, long unsigned int> >::_M_invoke<0, 1>(std::_Index_tuple<0, 1>) (this=0x407fde1f608) at /usr/include/c++/12/bits/std_thread.h:279
eloqdata/tx_service#10 0x00007fc519e6ce56 in std::thread::_Invoker<std::tuple<txservice::TxWorkerPool::TxWorkerPool(size_t)::<lambda(size_t)>, long unsigned int> >::operator()(void) (this=0x407fde1f608) at /usr/include/c++/12/bits/std_thread.h:286
eloqdata/tx_service#11 0x00007fc519e6ce3a in std::thread::_State_impl<std::thread::_Invoker<std::tuple<txservice::TxWorkerPool::TxWorkerPool(size_t)::<lambda(size_t)>, long unsigned int> > >::_M_run(void) (this=0x407fde1f600) at /usr/include/c++/12/bits/std_thread.h:231
eloqdata/tx_service#12 0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#13 0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#14 0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 41 (Thread 0x7fc4cc0d96c0 (LWP 1667205) "eloqdoc"):
#0  syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
eloqdata/tx_service#1  0x00007fc5263fe502 in bthread::futex_wait_private (timeout=0x0, expected=0, addr1=0x7fc4cc0caaf0) at /home/xiexy/github.com/eloqdata/brpc/src/bthread/sys_futex.h:40
eloqdata/tx_service#2  bthread::wait_pthread (pw=..., abstime=<optimized out>, abstime@entry=0x0) at /home/xiexy/github.com/eloqdata/brpc/src/bthread/butex.cpp:153
eloqdata/tx_service#3  0x00007fc5263fe927 in bthread::butex_wait_from_pthread (abstime=0x0, expected_value=0, b=0x40834050040, g=<optimized out>) at /home/xiexy/github.com/eloqdata/brpc/src/bthread/butex.cpp:669
eloqdata/tx_service#4  bthread::butex_wait (arg=0x40834050040, expected_value=expected_value@entry=0, abstime=abstime@entry=0x0) at /home/xiexy/github.com/eloqdata/brpc/src/bthread/butex.cpp:702
eloqdata/tx_service#5  0x00007fc526401745 in bthread_cond_wait(bthread_cond_t * __restrict__, bthread_mutex_t * __restrict__) (c=0x7fc4cc0cace0, m=0x7fc4cc0cacc8) at /home/xiexy/github.com/eloqdata/brpc/src/bthread/condition_variable.cpp:103
eloqdata/tx_service#6  0x00007fc51b3e5500 in bthread::ConditionVariable::wait (this=0x7fc4cc0cace0, lock=...) at /usr/include/bthread/condition_variable.h:59
eloqdata/tx_service#7  0x00007fc519e5879e in txservice::CkptTsCc::Wait (this=0x7fc4cc0caca0) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/tx_service/include/cc/cc_request.h:3095
eloqdata/tx_service#8  0x00007fc519e502ca in txservice::Checkpointer::GetNewCheckpointTs (this=0x407fc7d8ed0, node_group_id=0, is_last_ckpt=false) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/tx_service/src/checkpointer.cpp:98
eloqdata/tx_service#9  0x00007fc519e50edf in txservice::Checkpointer::Ckpt (this=0x407fc7d8ed0, is_last_ckpt=false) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/tx_service/src/checkpointer.cpp:219
eloqdata/tx_service#10 0x00007fc519e5211d in txservice::Checkpointer::Run (this=0x407fc7d8ed0) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/tx_service/src/checkpointer.cpp:446
eloqdata/tx_service#11 0x00007fc519e4fe53 in operator() (__closure=0x407fce8a988) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/tx_service/src/checkpointer.cpp:77
eloqdata/tx_service#12 0x00007fc519e56c28 in std::__invoke_impl<void, txservice::Checkpointer::Checkpointer(txservice::LocalCcShards&, txservice::store::DataStoreHandler*, const uint32_t&, txservice::TxLog*, uint32_t)::<lambda()> >(std::__invoke_other, struct {...} &&) (__f=...) at /usr/include/c++/12/bits/invoke.h:61
eloqdata/tx_service#13 0x00007fc519e56beb in std::__invoke<txservice::Checkpointer::Checkpointer(txservice::LocalCcShards&, txservice::store::DataStoreHandler*, const uint32_t&, txservice::TxLog*, uint32_t)::<lambda()> >(struct {...} &&) (__fn=...) at /usr/include/c++/12/bits/invoke.h:96
eloqdata/tx_service#14 0x00007fc519e56b98 in std::thread::_Invoker<std::tuple<txservice::Checkpointer::Checkpointer(txservice::LocalCcShards&, txservice::store::DataStoreHandler*, const uint32_t&, txservice::TxLog*, uint32_t)::<lambda()> > >::_M_invoke<0>(std::_Index_tuple<0>) (this=0x407fce8a988) at /usr/include/c++/12/bits/std_thread.h:279
eloqdata/tx_service#15 0x00007fc519e56b6c in std::thread::_Invoker<std::tuple<txservice::Checkpointer::Checkpointer(txservice::LocalCcShards&, txservice::store::DataStoreHandler*, const uint32_t&, txservice::TxLog*, uint32_t)::<lambda()> > >::operator()(void) (this=0x407fce8a988) at /usr/include/c++/12/bits/std_thread.h:286
eloqdata/tx_service#16 0x00007fc519e56b50 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<txservice::Checkpointer::Checkpointer(txservice::LocalCcShards&, txservice::store::DataStoreHandler*, const uint32_t&, txservice::TxLog*, uint32_t)::<lambda()> > > >::_M_run(void) (this=0x407fce8a980) at /usr/include/c++/12/bits/std_thread.h:231
eloqdata/tx_service#17 0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#18 0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#19 0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 40 (Thread 0x7fc4cde266c0 (LWP 1667204) "eloqdoc"):
#0  __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x7fc4cde18190, op=137, expected=0, futex_word=0x407fc7d8650) at ./nptl/futex-internal.c:57
eloqdata/tx_service#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x7fc4cde18190, clockid=0, expected=0, futex_word=0x407fc7d8650) at ./nptl/futex-internal.c:87
eloqdata/tx_service#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x407fc7d8650, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x7fc4cde18190, private=private@entry=0) at ./nptl/futex-internal.c:139
eloqdata/tx_service#3  0x00007fc525a8eead in __pthread_cond_wait_common (abstime=<optimized out>, clockid=<optimized out>, mutex=0x407fc7d8600, cond=0x407fc7d8628) at ./nptl/pthread_cond_wait.c:503
eloqdata/tx_service#4  ___pthread_cond_clockwait64 (abstime=<optimized out>, clockid=<optimized out>, mutex=0x407fc7d8600, cond=0x407fc7d8628) at ./nptl/pthread_cond_wait.c:691
eloqdata/tx_service#5  ___pthread_cond_clockwait64 (cond=0x407fc7d8628, mutex=0x407fc7d8600, clockid=<optimized out>, abstime=<optimized out>) at ./nptl/pthread_cond_wait.c:679
eloqdata/tx_service#6  0x00007fc525963159 in std::__condvar::wait_until (this=0x407fc7d8628, __m=..., __clock=1, __abs_time=...) at /usr/include/c++/12/bits/std_mutex.h:169
eloqdata/tx_service#7  0x00007fc52596d162 in std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x407fc7d8628, __lock=..., __atime=std::chrono::_V2::steady_clock time_point = {291041140153917ns}) at /usr/include/c++/12/condition_variable:200
eloqdata/tx_service#8  0x00007fc52596a5f1 in std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x407fc7d8628, __lock=..., __atime=std::chrono::_V2::steady_clock time_point = {291041140153917ns}) at /usr/include/c++/12/condition_variable:110
eloqdata/tx_service#9  0x00007fc51a11e73d in std::condition_variable::wait_until<std::chrono::_V2::steady_clock, std::chrono::duration<long int, std::ratio<1, 1000000000> >, txservice::LocalCcShards::TimerRun()::<lambda()> >(std::unique_lock<std::mutex> &, const std::chrono::time_point<std::chrono::_V2::steady_clock, std::chrono::duration<long, std::ratio<1, 1000000000> > > &, struct {...}) (this=0x407fc7d8628, __lock=..., __atime=std::chrono::_V2::steady_clock time_point = {291041140153917ns}, __p=...) at /usr/include/c++/12/condition_variable:151
eloqdata/tx_service#10 0x00007fc51a11cdea in std::condition_variable::wait_for<long int, std::ratio<1, 1000>, txservice::LocalCcShards::TimerRun()::<lambda()> >(std::unique_lock<std::mutex> &, const std::chrono::duration<long, std::ratio<1, 1000> > &, struct {...}) (this=0x407fc7d8628, __lock=..., __rtime=std::chrono::duration = { 100ms }, __p=...) at /usr/include/c++/12/condition_variable:174
eloqdata/tx_service#11 0x00007fc51a0fc373 in txservice::LocalCcShards::TimerRun (this=0x407fc7d8400) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/tx_service/src/cc/local_cc_shards.cpp:372
eloqdata/tx_service#12 0x00007fc51a0f8ecb in operator() (__closure=0x407fce8a528) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/tx_service/src/cc/local_cc_shards.cpp:164
eloqdata/tx_service#13 0x00007fc51a12a818 in std::__invoke_impl<void, txservice::LocalCcShards::LocalCcShards(uint32_t, uint32_t, const std::map<std::__cxx11::basic_string<char>, unsigned int>&, txservice::CatalogFactory**, txservice::SystemHandler*, std::unordered_map<unsigned int, std::vector<txservice::NodeConfig> >*, uint64_t, txservice::store::DataStoreHandler*, txservice::TxService*, bool, metrics::MetricsRegistry*, metrics::CommonLabels, std::unordered_map<txservice::TableName, std::__cxx11::basic_string<char> >*, std::function<void(std::basic_string_view<char>, std::basic_string_view<char>)>)::<lambda()> >(std::__invoke_other, struct {...} &&) (__f=...) at /usr/include/c++/12/bits/invoke.h:61
eloqdata/tx_service#14 0x00007fc51a12a5b6 in std::__invoke<txservice::LocalCcShards::LocalCcShards(uint32_t, uint32_t, const std::map<std::__cxx11::basic_string<char>, unsigned int>&, txservice::CatalogFactory**, txservice::SystemHandler*, std::unordered_map<unsigned int, std::vector<txservice::NodeConfig> >*, uint64_t, txservice::store::DataStoreHandler*, txservice::TxService*, bool, metrics::MetricsRegistry*, metrics::CommonLabels, std::unordered_map<txservice::TableName, std::__cxx11::basic_string<char> >*, std::function<void(std::basic_string_view<char>, std::basic_string_view<char>)>)::<lambda()> >(struct {...} &&) (__fn=...) at /usr/include/c++/12/bits/invoke.h:96
eloqdata/tx_service#15 0x00007fc51a12a278 in std::thread::_Invoker<std::tuple<txservice::LocalCcShards::LocalCcShards(uint32_t, uint32_t, const std::map<std::__cxx11::basic_string<char>, unsigned int>&, txservice::CatalogFactory**, txservice::SystemHandler*, std::unordered_map<unsigned int, std::vector<txservice::NodeConfig> >*, uint64_t, txservice::store::DataStoreHandler*, txservice::TxService*, bool, metrics::MetricsRegistry*, metrics::CommonLabels, std::unordered_map<txservice::TableName, std::__cxx11::basic_string<char> >*, std::function<void(std::basic_string_view<char>, std::basic_string_view<char>)>)::<lambda()> > >::_M_invoke<0>(std::_Index_tuple<0>) (this=0x407fce8a528) at /usr/include/c++/12/bits/std_thread.h:279
eloqdata/tx_service#16 0x00007fc51a12a096 in std::thread::_Invoker<std::tuple<txservice::LocalCcShards::LocalCcShards(uint32_t, uint32_t, const std::map<std::__cxx11::basic_string<char>, unsigned int>&, txservice::CatalogFactory**, txservice::SystemHandler*, std::unordered_map<unsigned int, std::vector<txservice::NodeConfig> >*, uint64_t, txservice::store::DataStoreHandler*, txservice::TxService*, bool, metrics::MetricsRegistry*, metrics::CommonLabels, std::unordered_map<txservice::TableName, std::__cxx11::basic_string<char> >*, std::function<void(std::basic_string_view<char>, std::basic_string_view<char>)>)::<lambda()> > >::operator()(void) (this=0x407fce8a528) at /usr/include/c++/12/bits/std_thread.h:286
eloqdata/tx_service#17 0x00007fc51a129eb8 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<txservice::LocalCcShards::LocalCcShards(uint32_t, uint32_t, const std::map<std::__cxx11::basic_string<char>, unsigned int>&, txservice::CatalogFactory**, txservice::SystemHandler*, std::unordered_map<unsigned int, std::vector<txservice::NodeConfig> >*, uint64_t, txservice::store::DataStoreHandler*, txservice::TxService*, bool, metrics::MetricsRegistry*, metrics::CommonLabels, std::unordered_map<txservice::TableName, std::__cxx11::basic_string<char> >*, std::function<void(std::basic_string_view<char>, std::basic_string_view<char>)>)::<lambda()> > > >::_M_run(void) (this=0x407fce8a520) at /usr/include/c++/12/bits/std_thread.h:231
eloqdata/tx_service#18 0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#19 0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#20 0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 39 (Thread 0x7fc4cfb736c0 (LWP 1667203) "eloqdoc"):
#0  syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
eloqdata/tx_service#1  0x00007fc5263fe502 in bthread::futex_wait_private (timeout=0x0, expected=0, addr1=0x7fc4cfb645a0) at /home/xiexy/github.com/eloqdata/brpc/src/bthread/sys_futex.h:40
eloqdata/tx_service#2  bthread::wait_pthread (pw=..., abstime=<optimized out>, abstime@entry=0x0) at /home/xiexy/github.com/eloqdata/brpc/src/bthread/butex.cpp:153
eloqdata/tx_service#3  0x00007fc5263fe927 in bthread::butex_wait_from_pthread (abstime=0x0, expected_value=1, b=0x40832100040, g=<optimized out>) at /home/xiexy/github.com/eloqdata/brpc/src/bthread/butex.cpp:669
eloqdata/tx_service#4  bthread::butex_wait (arg=arg@entry=0x40832100040, expected_value=expected_value@entry=1, abstime=abstime@entry=0x0) at /home/xiexy/github.com/eloqdata/brpc/src/bthread/butex.cpp:702
eloqdata/tx_service#5  0x00007fc526407d78 in bthread_id_join (id=...) at /home/xiexy/github.com/eloqdata/brpc/src/bthread/id.cpp:529
eloqdata/tx_service#6  0x00007fc5264a3f75 in brpc::Channel::CallMethod (this=0x7fc4cfb648f0, method=0x407fd883940, controller_base=0x7fc4cfb64a30, request=0x7fc4cfb648c0, response=0x7fc4cfb64840, done=0x0) at /home/xiexy/github.com/eloqdata/brpc/src/brpc/channel.cpp:581
eloqdata/tx_service#7  0x00007fc517b104c1 in braft::rtb::refresh_leader (group="lg0", timeout_ms=500) at /home/xiexy/github.com/eloqdata/braft/src/braft/route_table.cpp:237
eloqdata/tx_service#8  0x00007fc51a462932 in txlog::LogAgent::RefreshLeader (this=0x407fcb2c408, log_group_id=0, timeout_ms=500) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/tx_service/tx-log-protos/log_agent.cpp:170
eloqdata/tx_service#9  0x00007fc51a466059 in txlog::LogAgent::CheckLeaderRun (this=0x407fcb2c408) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/tx_service/tx-log-protos/log_agent.cpp:696
eloqdata/tx_service#10 0x00007fc51b2e69df in txlog::LogAgent::LogAgent()::{lambda()#1}::operator()() const (__closure=0x407fce8a538) at src/mongo/db/modules/eloq/tx_service/tx-log-protos/log_agent.h:69
eloqdata/tx_service#11 0x00007fc51b3a44ab in std::__invoke_impl<void, txlog::LogAgent::LogAgent()::{lambda()#1}>(std::__invoke_other, txlog::LogAgent::LogAgent()::{lambda()#1}&&) (__f=...) at /usr/include/c++/12/bits/invoke.h:61
eloqdata/tx_service#12 0x00007fc51b3a4337 in std::__invoke<txlog::LogAgent::LogAgent()::{lambda()#1}>(txlog::LogAgent::LogAgent()::{lambda()#1}&&) (__fn=...) at /usr/include/c++/12/bits/invoke.h:96
eloqdata/tx_service#13 0x00007fc51b3a4134 in std::thread::_Invoker<std::tuple<txlog::LogAgent::LogAgent()::{lambda()#1}> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) (this=0x407fce8a538) at /usr/include/c++/12/bits/std_thread.h:279
eloqdata/tx_service#14 0x00007fc51b3a3e44 in std::thread::_Invoker<std::tuple<txlog::LogAgent::LogAgent()::{lambda()#1}> >::operator()() (this=0x407fce8a538) at /usr/include/c++/12/bits/std_thread.h:286
eloqdata/tx_service#15 0x00007fc51b3a2aca in std::thread::_State_impl<std::thread::_Invoker<std::tuple<txlog::LogAgent::LogAgent()::{lambda()#1}> > >::_M_run() (this=0x407fce8a530) at /usr/include/c++/12/bits/std_thread.h:231
eloqdata/tx_service#16 0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#17 0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#18 0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 38 (Thread 0x7fc4d18c26c0 (LWP 1667202) "eloqdoc"):
#0  __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x7fc4d18b4170, op=137, expected=0, futex_word=0x407fc8a8920) at ./nptl/futex-internal.c:57
eloqdata/tx_service#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x7fc4d18b4170, clockid=22, expected=0, futex_word=0x407fc8a8920) at ./nptl/futex-internal.c:87
eloqdata/tx_service#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x407fc8a8920, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x7fc4d18b4170, private=private@entry=0) at ./nptl/futex-internal.c:139
eloqdata/tx_service#3  0x00007fc525a8eead in __pthread_cond_wait_common (abstime=<optimized out>, clockid=<optimized out>, mutex=0x407fc8a88d0, cond=0x407fc8a88f8) at ./nptl/pthread_cond_wait.c:503
eloqdata/tx_service#4  ___pthread_cond_clockwait64 (abstime=<optimized out>, clockid=<optimized out>, mutex=0x407fc8a88d0, cond=0x407fc8a88f8) at ./nptl/pthread_cond_wait.c:691
eloqdata/tx_service#5  ___pthread_cond_clockwait64 (cond=0x407fc8a88f8, mutex=0x407fc8a88d0, clockid=<optimized out>, abstime=<optimized out>) at ./nptl/pthread_cond_wait.c:679
eloqdata/tx_service#6  0x00007fc525963159 in std::__condvar::wait_until (this=0x407fc8a88f8, __m=..., __clock=1, __abs_time=...) at /usr/include/c++/12/bits/std_mutex.h:169
eloqdata/tx_service#7  0x00007fc52596d162 in std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x407fc8a88f8, __lock=..., __atime=std::chrono::_V2::steady_clock time_point = {291610474975032ns}) at /usr/include/c++/12/condition_variable:200
eloqdata/tx_service#8  0x00007fc52596a5f1 in std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x407fc8a88f8, __lock=..., __atime=std::chrono::_V2::steady_clock time_point = {291610474975032ns}) at /usr/include/c++/12/condition_variable:110
eloqdata/tx_service#9  0x00007fc518926f2f in std::condition_variable::wait_until<std::chrono::_V2::steady_clock, std::chrono::duration<long int, std::ratio<1, 1000000000> >, txlog::CronJob::Start(uint32_t&, uint32_t&, uint32_t&, uint32_t&, std::function<void(std::function<long int(long int*)>)>, bool)::<lambda()>::<lambda()> >(std::unique_lock<std::mutex> &, const std::chrono::time_point<std::chrono::_V2::steady_clock, std::chrono::duration<long, std::ratio<1, 1000000000> > > &, struct {...}) (this=0x407fc8a88f8, __lock=..., __atime=std::chrono::_V2::steady_clock time_point = {291610474975032ns}, __p=...) at /usr/include/c++/12/condition_variable:151
eloqdata/tx_service#10 0x00007fc518926a22 in std::condition_variable::wait_for<long int, std::ratio<1>, txlog::CronJob::Start(uint32_t&, uint32_t&, uint32_t&, uint32_t&, std::function<void(std::function<long int(long int*)>)>, bool)::<lambda()>::<lambda()> >(std::unique_lock<std::mutex> &, const std::chrono::duration<long, std::ratio<1, 1> > &, struct {...}) (this=0x407fc8a88f8, __lock=..., __rtime=std::chrono::duration = { 600s }, __p=...) at /usr/include/c++/12/condition_variable:174
eloqdata/tx_service#11 0x00007fc51891df95 in operator() (__closure=0x407fd093938) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/eloq_log_service/src/log_state_rocksdb_cloud_impl.cpp:101
eloqdata/tx_service#12 0x00007fc518928bb8 in std::__invoke_impl<void, txlog::CronJob::Start(uint32_t&, uint32_t&, uint32_t&, uint32_t&, std::function<void(std::function<long int(long int*)>)>, bool)::<lambda()> >(std::__invoke_other, struct {...} &&) (__f=...) at /usr/include/c++/12/bits/invoke.h:61
eloqdata/tx_service#13 0x00007fc518928b3e in std::__invoke<txlog::CronJob::Start(uint32_t&, uint32_t&, uint32_t&, uint32_t&, std::function<void(std::function<long int(long int*)>)>, bool)::<lambda()> >(struct {...} &&) (__fn=...) at /usr/include/c++/12/bits/invoke.h:96
eloqdata/tx_service#14 0x00007fc518928a98 in std::thread::_Invoker<std::tuple<txlog::CronJob::Start(uint32_t&, uint32_t&, uint32_t&, uint32_t&, std::function<void(std::function<long int(long int*)>)>, bool)::<lambda()> > >::_M_invoke<0>(std::_Index_tuple<0>) (this=0x407fd093938) at /usr/include/c++/12/bits/std_thread.h:279
eloqdata/tx_service#15 0x00007fc518928a40 in std::thread::_Invoker<std::tuple<txlog::CronJob::Start(uint32_t&, uint32_t&, uint32_t&, uint32_t&, std::function<void(std::function<long int(long int*)>)>, bool)::<lambda()> > >::operator()(void) (this=0x407fd093938) at /usr/include/c++/12/bits/std_thread.h:286
eloqdata/tx_service#16 0x00007fc518928a08 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<txlog::CronJob::Start(uint32_t&, uint32_t&, uint32_t&, uint32_t&, std::function<void(std::function<long int(long int*)>)>, bool)::<lambda()> > > >::_M_run(void) (this=0x407fd093930) at /usr/include/c++/12/bits/std_thread.h:231
eloqdata/tx_service#17 0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#18 0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#19 0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 37 (Thread 0x7fc4d36136c0 (LWP 1667201) "eloqdoc"):
#0  __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x407fd7fc160) at ./nptl/futex-internal.c:57
eloqdata/tx_service#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x407fd7fc160) at ./nptl/futex-internal.c:87
eloqdata/tx_service#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x407fd7fc160, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
eloqdata/tx_service#3  0x00007fc525a8e5f8 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x407fd7fc110, cond=0x407fd7fc138) at ./nptl/pthread_cond_wait.c:503
eloqdata/tx_service#4  ___pthread_cond_wait (cond=0x407fd7fc138, mutex=0x407fd7fc110) at ./nptl/pthread_cond_wait.c:627
eloqdata/tx_service#5  0x00007fc51825fd67 in std::condition_variable::wait<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()>::<lambda()> >(std::unique_lock<std::mutex> &, struct {...}) (this=0x407fd7fc138, __lock=..., __p=...) at /usr/include/c++/12/condition_variable:102
eloqdata/tx_service#6  0x00007fc51825f841 in operator() (__closure=0x407fce8aee8) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/store_handler/eloq_data_store_service/thread_worker_pool.cpp:41
eloqdata/tx_service#7  0x00007fc518260164 in std::__invoke_impl<void, EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> >(std::__invoke_other, struct {...} &&) (__f=...) at /usr/include/c++/12/bits/invoke.h:61
eloqdata/tx_service#8  0x00007fc518260127 in std::__invoke<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> >(struct {...} &&) (__fn=...) at /usr/include/c++/12/bits/invoke.h:96
eloqdata/tx_service#9  0x00007fc5182600d4 in std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > >::_M_invoke<0>(std::_Index_tuple<0>) (this=0x407fce8aee8) at /usr/include/c++/12/bits/std_thread.h:279
eloqdata/tx_service#10 0x00007fc5182600a8 in std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > >::operator()(void) (this=0x407fce8aee8) at /usr/include/c++/12/bits/std_thread.h:286
eloqdata/tx_service#11 0x00007fc51826008c in std::thread::_State_impl<std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > > >::_M_run(void) (this=0x407fce8aee0) at /usr/include/c++/12/bits/std_thread.h:231
eloqdata/tx_service#12 0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#13 0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#14 0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 36 (Thread 0x7fc4d55666c0 (LWP 1667200) "brpc_worker:0"):
#0  0x00007fc525b1e546 in epoll_wait (epfd=44, events=0x7fc4d3714d10, maxevents=32, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
eloqdata/tx_service#1  0x00007fc5264d8d1f in brpc::EventDispatcher::Run (this=this@entry=0x407fd1e0348) at /home/xiexy/github.com/eloqdata/brpc/src/brpc/event_dispatcher_epoll.cpp:259
eloqdata/tx_service#2  0x00007fc5264d8f0c in brpc::EventDispatcher::RunThis (arg=0x407fd1e0348) at /home/xiexy/github.com/eloqdata/brpc/src/brpc/event_dispatcher_epoll.cpp:237
eloqdata/tx_service#3  0x00007fc52641fe97 in bthread::TaskGroup::task_runner (skip_remained=<optimized out>) at /home/xiexy/github.com/eloqdata/brpc/src/bthread/task_group.cpp:383
eloqdata/tx_service#4  0x00007fc526401891 in bthread_make_fcontext () from /lib/libbrpc.so
eloqdata/tx_service#5  0x0000000000000000 in ?? ()

Thread 35 (Thread 0x7fc4d72b36c0 (LWP 1667199) "brpc_timer"):
#0  syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
eloqdata/tx_service#1  0x00007fc526426339 in bthread::futex_wait_private (timeout=<optimized out>, expected=4, addr1=0x407fc3a4b48) at /home/xiexy/github.com/eloqdata/brpc/src/bthread/sys_futex.h:40
eloqdata/tx_service#2  bthread::TimerThread::run (this=this@entry=0x407fc3a4b00) at /home/xiexy/github.com/eloqdata/brpc/src/bthread/timer_thread.cpp:433
eloqdata/tx_service#3  0x00007fc526426b04 in bthread::TimerThread::run_this (arg=0x407fc3a4b00) at /home/xiexy/github.com/eloqdata/brpc/src/bthread/timer_thread.cpp:122
eloqdata/tx_service#4  0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#5  0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 34 (Thread 0x7fc4bf5cc6c0 (LWP 1667197) "eloqdoc"):
#0  __futex_abstimed_wait_common64 (private=-1084497400, cancel=true, abstime=0x7fc4bf5be2c0, op=393, expected=0, futex_word=0x407fcb3fe70) at ./nptl/futex-internal.c:57
eloqdata/tx_service#1  __futex_abstimed_wait_common (cancel=true, private=-1084497400, abstime=0x7fc4bf5be2c0, clockid=187, expected=0, futex_word=0x407fcb3fe70) at ./nptl/futex-internal.c:87
eloqdata/tx_service#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x407fcb3fe70, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x7fc4bf5be2c0, private=private@entry=0) at ./nptl/futex-internal.c:139
eloqdata/tx_service#3  0x00007fc525a8ea6a in __pthread_cond_wait_common (abstime=0x7fc4bf5be2c0, clockid=0, mutex=0x407fcb3fe08, cond=0x407fcb3fe48) at ./nptl/pthread_cond_wait.c:503
eloqdata/tx_service#4  ___pthread_cond_timedwait64 (cond=0x407fcb3fe48, mutex=0x407fcb3fe08, abstime=0x7fc4bf5be2c0) at ./nptl/pthread_cond_wait.c:652
eloqdata/tx_service#5  0x00007fc51719ae62 in rocksdb::port::CondVar::TimedWait (this=this@entry=0x407fcb3fe48, abs_time_us=abs_time_us@entry=1763548172795482) at port/port_posix.cc:133
eloqdata/tx_service#6  0x00007fc51714a009 in rocksdb::InstrumentedCondVar::TimedWaitInternal (this=this@entry=0x407fcb3fe48, abs_time_us=abs_time_us@entry=1763548172795482) at monitoring/instrumented_mutex.cc:87
eloqdata/tx_service#7  0x00007fc51714a12f in rocksdb::InstrumentedCondVar::TimedWait (this=this@entry=0x407fcb3fe48, abs_time_us=1763548172795482) at monitoring/instrumented_mutex.cc:76
eloqdata/tx_service#8  0x00007fc517003a00 in rocksdb::Timer::Run (this=0x407fcb3fe00) at ./util/timer.h:252
eloqdata/tx_service#9  0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#10 0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#11 0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 33 (Thread 0x7fc4dad536c0 (LWP 1667181) "rocksdb:high"):
#0  __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x407fcb3f6d4) at ./nptl/futex-internal.c:57
eloqdata/tx_service#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x407fcb3f6d4) at ./nptl/futex-internal.c:87
eloqdata/tx_service#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x407fcb3f6d4, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
eloqdata/tx_service#3  0x00007fc525a8e5f8 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x407fcb3f680, cond=0x407fcb3f6a8) at ./nptl/pthread_cond_wait.c:503
eloqdata/tx_service#4  ___pthread_cond_wait (cond=0x407fcb3f6a8, mutex=0x407fcb3f680) at ./nptl/pthread_cond_wait.c:627
eloqdata/tx_service#5  0x00007fc5172aa558 in rocksdb::ThreadPoolImpl::Impl::BGThread (this=this@entry=0x407fcb3f600, thread_id=thread_id@entry=0) at util/threadpool_imp.cc:236
eloqdata/tx_service#6  0x00007fc5172aa904 in rocksdb::ThreadPoolImpl::Impl::BGThreadWrapper (arg=0x407fce8aee0) at util/threadpool_imp.cc:352
eloqdata/tx_service#7  0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#8  0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#9  0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 32 (Thread 0x7fc4dcaa06c0 (LWP 1667180) "rocksdb:low"):
#0  __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x407fcb3f9d0) at ./nptl/futex-internal.c:57
eloqdata/tx_service#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x407fcb3f9d0) at ./nptl/futex-internal.c:87
eloqdata/tx_service#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x407fcb3f9d0, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
eloqdata/tx_service#3  0x00007fc525a8e5f8 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x407fcb3f980, cond=0x407fcb3f9a8) at ./nptl/pthread_cond_wait.c:503
eloqdata/tx_service#4  ___pthread_cond_wait (cond=0x407fcb3f9a8, mutex=0x407fcb3f980) at ./nptl/pthread_cond_wait.c:627
eloqdata/tx_service#5  0x00007fc5172aa558 in rocksdb::ThreadPoolImpl::Impl::BGThread (this=this@entry=0x407fcb3f900, thread_id=thread_id@entry=2) at util/threadpool_imp.cc:236
eloqdata/tx_service#6  0x00007fc5172aa904 in rocksdb::ThreadPoolImpl::Impl::BGThreadWrapper (arg=0x407fce8ab10) at util/threadpool_imp.cc:352
eloqdata/tx_service#7  0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#8  0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#9  0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 31 (Thread 0x7fc4de7f16c0 (LWP 1667179) "rocksdb:low"):
#0  __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x407fcb3f9d0) at ./nptl/futex-internal.c:57
eloqdata/tx_service#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x407fcb3f9d0) at ./nptl/futex-internal.c:87
eloqdata/tx_service#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x407fcb3f9d0, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
eloqdata/tx_service#3  0x00007fc525a8e5f8 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x407fcb3f980, cond=0x407fcb3f9a8) at ./nptl/pthread_cond_wait.c:503
eloqdata/tx_service#4  ___pthread_cond_wait (cond=0x407fcb3f9a8, mutex=0x407fcb3f980) at ./nptl/pthread_cond_wait.c:627
eloqdata/tx_service#5  0x00007fc5172aa558 in rocksdb::ThreadPoolImpl::Impl::BGThread (this=this@entry=0x407fcb3f900, thread_id=thread_id@entry=1) at util/threadpool_imp.cc:236
eloqdata/tx_service#6  0x00007fc5172aa904 in rocksdb::ThreadPoolImpl::Impl::BGThreadWrapper (arg=0x407fce8aff0) at util/threadpool_imp.cc:352
eloqdata/tx_service#7  0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#8  0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#9  0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 30 (Thread 0x7fc4e053e6c0 (LWP 1667178) "rocksdb:low"):
#0  __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x407fcb3f9d0) at ./nptl/futex-internal.c:57
eloqdata/tx_service#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x407fcb3f9d0) at ./nptl/futex-internal.c:87
eloqdata/tx_service#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x407fcb3f9d0, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
eloqdata/tx_service#3  0x00007fc525a8e5f8 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x407fcb3f980, cond=0x407fcb3f9a8) at ./nptl/pthread_cond_wait.c:503
eloqdata/tx_service#4  ___pthread_cond_wait (cond=0x407fcb3f9a8, mutex=0x407fcb3f980) at ./nptl/pthread_cond_wait.c:627
eloqdata/tx_service#5  0x00007fc5172aa558 in rocksdb::ThreadPoolImpl::Impl::BGThread (this=this@entry=0x407fcb3f900, thread_id=thread_id@entry=0) at util/threadpool_imp.cc:236
eloqdata/tx_service#6  0x00007fc5172aa904 in rocksdb::ThreadPoolImpl::Impl::BGThreadWrapper (arg=0x407fce8afe0) at util/threadpool_imp.cc:352
eloqdata/tx_service#7  0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#8  0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#9  0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 29 (Thread 0x7fc4e228d6c0 (LWP 1667177) "eloqdoc"):
#0  __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x7fc4e227f180, op=137, expected=0, futex_word=0x407fcaf69d0) at ./nptl/futex-internal.c:57
eloqdata/tx_service#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x7fc4e227f180, clockid=-500699056, expected=0, futex_word=0x407fcaf69d0) at ./nptl/futex-internal.c:87
eloqdata/tx_service#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x407fcaf69d0, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x7fc4e227f180, private=private@entry=0) at ./nptl/futex-internal.c:139
eloqdata/tx_service#3  0x00007fc525a8eead in __pthread_cond_wait_common (abstime=<optimized out>, clockid=<optimized out>, mutex=0x407fcaf6980, cond=0x407fcaf69a8) at ./nptl/pthread_cond_wait.c:503
eloqdata/tx_service#4  ___pthread_cond_clockwait64 (abstime=<optimized out>, clockid=<optimized out>, mutex=0x407fcaf6980, cond=0x407fcaf69a8) at ./nptl/pthread_cond_wait.c:691
eloqdata/tx_service#5  ___pthread_cond_clockwait64 (cond=0x407fcaf69a8, mutex=0x407fcaf6980, clockid=<optimized out>, abstime=<optimized out>) at ./nptl/pthread_cond_wait.c:679
eloqdata/tx_service#6  0x00007fc525963159 in std::__condvar::wait_until (this=0x407fcaf69a8, __m=..., __clock=1, __abs_time=...) at /usr/include/c++/12/bits/std_mutex.h:169
eloqdata/tx_service#7  0x00007fc52596d162 in std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x407fcaf69a8, __lock=..., __atime=std::chrono::_V2::steady_clock time_point = {291070405886477ns}) at /usr/include/c++/12/condition_variable:200
eloqdata/tx_service#8  0x00007fc52596a5f1 in std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x407fcaf69a8, __lock=..., __atime=std::chrono::_V2::steady_clock time_point = {291070405886477ns}) at /usr/include/c++/12/condition_variable:110
eloqdata/tx_service#9  0x00007fc51830f299 in std::condition_variable::wait_until<std::chrono::_V2::steady_clock, std::chrono::duration<long int, std::ratio<1, 1000000000> >, EloqDS::SlidingWindow::TimerWorker()::<lambda()> >(std::unique_lock<std::mutex> &, const std::chrono::time_point<std::chrono::_V2::steady_clock, std::chrono::duration<long, std::ratio<1, 1000000000> > > &, struct {...}) (this=0x407fcaf69a8, __lock=..., __atime=std::chrono::_V2::steady_clock time_point = {291070405886477ns}, __p=...) at /usr/include/c++/12/condition_variable:151
eloqdata/tx_service#10 0x00007fc51830f240 in std::condition_variable::wait_for<long int, std::ratio<1, 1000>, EloqDS::SlidingWindow::TimerWorker()::<lambda()> >(std::unique_lock<std::mutex> &, const std::chrono::duration<long, std::ratio<1, 1000> > &, struct {...}) (this=0x407fcaf69a8, __lock=..., __rtime=std::chrono::duration = { 30000ms }, __p=...) at /usr/include/c++/12/condition_variable:174
eloqdata/tx_service#11 0x00007fc51830edc1 in EloqDS::SlidingWindow::TimerWorker (this=0x407fcaf6900) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/store_handler/eloq_data_store_service/purger_sliding_window.cpp:329
eloqdata/tx_service#12 0x00007fc518312f28 in std::__invoke_impl<void, void (EloqDS::SlidingWindow::*)(), EloqDS::SlidingWindow*> (__f=@0x407fcf58fd0: (void (EloqDS::SlidingWindow::*)(EloqDS::SlidingWindow * const)) 0x7fc51830ece0 <EloqDS::SlidingWindow::TimerWorker()>, __t=@0x407fcf58fc8: 0x407fcaf6900) at /usr/include/c++/12/bits/invoke.h:74
eloqdata/tx_service#13 0x00007fc518312e7b in std::__invoke<void (EloqDS::SlidingWindow::*)(), EloqDS::SlidingWindow*> (__fn=@0x407fcf58fd0: (void (EloqDS::SlidingWindow::*)(EloqDS::SlidingWindow * const)) 0x7fc51830ece0 <EloqDS::SlidingWindow::TimerWorker()>) at /usr/include/c++/12/bits/invoke.h:96
eloqdata/tx_service#14 0x00007fc518312ddb in std::thread::_Invoker<std::tuple<void (EloqDS::SlidingWindow::*)(), EloqDS::SlidingWindow*> >::_M_invoke<0ul, 1ul> (this=0x407fcf58fc8) at /usr/include/c++/12/bits/std_thread.h:279
eloqdata/tx_service#15 0x00007fc518312d90 in std::thread::_Invoker<std::tuple<void (EloqDS::SlidingWindow::*)(), EloqDS::SlidingWindow*> >::operator() (this=0x407fcf58fc8) at /usr/include/c++/12/bits/std_thread.h:286
eloqdata/tx_service#16 0x00007fc518312d70 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (EloqDS::SlidingWindow::*)(), EloqDS::SlidingWindow*> > >::_M_run (this=0x407fcf58fc0) at /usr/include/c++/12/bits/std_thread.h:231
eloqdata/tx_service#17 0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#18 0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#19 0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 28 (Thread 0x7fc4e3fda6c0 (LWP 1667176) "eloqdoc"):
#0  __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x7fc4e3fcc3a0, op=137, expected=0, futex_word=0x407fcad1ce8) at ./nptl/futex-internal.c:57
eloqdata/tx_service#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x7fc4e3fcc3a0, clockid=65440, expected=0, futex_word=0x407fcad1ce8) at ./nptl/futex-internal.c:87
eloqdata/tx_service#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x407fcad1ce8, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x7fc4e3fcc3a0, private=private@entry=0) at ./nptl/futex-internal.c:139
eloqdata/tx_service#3  0x00007fc525a8eead in __pthread_cond_wait_common (abstime=<optimized out>, clockid=<optimized out>, mutex=0x407fcad1c98, cond=0x407fcad1cc0) at ./nptl/pthread_cond_wait.c:503
eloqdata/tx_service#4  ___pthread_cond_clockwait64 (abstime=<optimized out>, clockid=<optimized out>, mutex=0x407fcad1c98, cond=0x407fcad1cc0) at ./nptl/pthread_cond_wait.c:691
eloqdata/tx_service#5  ___pthread_cond_clockwait64 (cond=0x407fcad1cc0, mutex=0x407fcad1c98, clockid=<optimized out>, abstime=<optimized out>) at ./nptl/pthread_cond_wait.c:679
eloqdata/tx_service#6  0x00007fc516e10eed in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x407fcad1cc0) at /usr/include/c++/12/bits/std_mutex.h:169
eloqdata/tx_service#7  std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__lock=<synthetic pointer>..., __atime=..., this=0x407fcad1cc0) at /usr/include/c++/12/condition_variable:200
eloqdata/tx_service#8  std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=<synthetic pointer>..., this=0x407fcad1cc0) at /usr/include/c++/12/condition_variable:110
eloqdata/tx_service#9  std::condition_variable::wait_until<std::chrono::_V2::steady_clock, std::chrono::duration<long int, std::ratio<1, 1000000000> >, rocksdb::CloudFileSystemImpl::Purger()::<lambda()> > (__p=..., __atime=..., __lock=<synthetic pointer>..., this=0x407fcad1cc0) at /usr/include/c++/12/condition_variable:151
eloqdata/tx_service#10 std::condition_variable::wait_for<long int, std::ratio<1, 1000>, rocksdb::CloudFileSystemImpl::Purger()::<lambda()> > (__rtime=..., __p=..., __lock=<synthetic pointer>..., this=0x407fcad1cc0) at /usr/include/c++/12/condition_variable:174
eloqdata/tx_service#11 rocksdb::CloudFileSystemImpl::Purger (this=0x407fcad1900) at cloud/eloq_purger.cc:528
eloqdata/tx_service#12 0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#13 0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#14 0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 27 (Thread 0x7fc4e5d276c0 (LWP 1667175) "eloqdoc"):
#0  __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x407fc6cdff0) at ./nptl/futex-internal.c:57
eloqdata/tx_service#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x407fc6cdff0) at ./nptl/futex-internal.c:87
eloqdata/tx_service#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x407fc6cdff0, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
eloqdata/tx_service#3  0x00007fc525a8e5f8 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x407fc6cdfa0, cond=0x407fc6cdfc8) at ./nptl/pthread_cond_wait.c:503
eloqdata/tx_service#4  ___pthread_cond_wait (cond=0x407fc6cdfc8, mutex=0x407fc6cdfa0) at ./nptl/pthread_cond_wait.c:627
eloqdata/tx_service#5  0x00007fc516e16560 in rocksdb::CloudSchedulerImpl::DoWork (this=0x407fc6cdf90) at cloud/cloud_scheduler.cc:320
eloqdata/tx_service#6  0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#7  0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#8  0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 26 (Thread 0x7fc4e7a786c0 (LWP 1667174) "eloqdoc"):
#0  __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x407fcaf86a0) at ./nptl/futex-internal.c:57
eloqdata/tx_service#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x407fcaf86a0) at ./nptl/futex-internal.c:87
eloqdata/tx_service#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x407fcaf86a0, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
eloqdata/tx_service#3  0x00007fc525a8e5f8 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x407fcaf8650, cond=0x407fcaf8678) at ./nptl/pthread_cond_wait.c:503
eloqdata/tx_service#4  ___pthread_cond_wait (cond=0x407fcaf8678, mutex=0x407fcaf8650) at ./nptl/pthread_cond_wait.c:627
eloqdata/tx_service#5  0x00007fc51825fd67 in std::condition_variable::wait<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()>::<lambda()> >(std::unique_lock<std::mutex> &, struct {...}) (this=0x407fcaf8678, __lock=..., __p=...) at /usr/include/c++/12/condition_variable:102
eloqdata/tx_service#6  0x00007fc51825f841 in operator() (__closure=0x407fce85f38) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/store_handler/eloq_data_store_service/thread_worker_pool.cpp:41
eloqdata/tx_service#7  0x00007fc518260164 in std::__invoke_impl<void, EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> >(std::__invoke_other, struct {...} &&) (__f=...) at /usr/include/c++/12/bits/invoke.h:61
eloqdata/tx_service#8  0x00007fc518260127 in std::__invoke<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> >(struct {...} &&) (__fn=...) at /usr/include/c++/12/bits/invoke.h:96
eloqdata/tx_service#9  0x00007fc5182600d4 in std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > >::_M_invoke<0>(std::_Index_tuple<0>) (this=0x407fce85f38) at /usr/include/c++/12/bits/std_thread.h:279
eloqdata/tx_service#10 0x00007fc5182600a8 in std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > >::operator()(void) (this=0x407fce85f38) at /usr/include/c++/12/bits/std_thread.h:286
eloqdata/tx_service#11 0x00007fc51826008c in std::thread::_State_impl<std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > > >::_M_run(void) (this=0x407fce85f30) at /usr/include/c++/12/bits/std_thread.h:231
eloqdata/tx_service#12 0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#13 0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#14 0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 25 (Thread 0x7fc4e97c56c0 (LWP 1667173) "eloqdoc"):
#0  __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x407fcaf86a0) at ./nptl/futex-internal.c:57
eloqdata/tx_service#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x407fcaf86a0) at ./nptl/futex-internal.c:87
eloqdata/tx_service#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x407fcaf86a0, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
eloqdata/tx_service#3  0x00007fc525a8e5f8 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x407fcaf8650, cond=0x407fcaf8678) at ./nptl/pthread_cond_wait.c:503
eloqdata/tx_service#4  ___pthread_cond_wait (cond=0x407fcaf8678, mutex=0x407fcaf8650) at ./nptl/pthread_cond_wait.c:627
eloqdata/tx_service#5  0x00007fc51825fd67 in std::condition_variable::wait<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()>::<lambda()> >(std::unique_lock<std::mutex> &, struct {...}) (this=0x407fcaf8678, __lock=..., __p=...) at /usr/include/c++/12/condition_variable:102
eloqdata/tx_service#6  0x00007fc51825f841 in operator() (__closure=0x407fce85f48) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/store_handler/eloq_data_store_service/thread_worker_pool.cpp:41
eloqdata/tx_service#7  0x00007fc518260164 in std::__invoke_impl<void, EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> >(std::__invoke_other, struct {...} &&) (__f=...) at /usr/include/c++/12/bits/invoke.h:61
eloqdata/tx_service#8  0x00007fc518260127 in std::__invoke<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> >(struct {...} &&) (__fn=...) at /usr/include/c++/12/bits/invoke.h:96
eloqdata/tx_service#9  0x00007fc5182600d4 in std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > >::_M_invoke<0>(std::_Index_tuple<0>) (this=0x407fce85f48) at /usr/include/c++/12/bits/std_thread.h:279
eloqdata/tx_service#10 0x00007fc5182600a8 in std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > >::operator()(void) (this=0x407fce85f48) at /usr/include/c++/12/bits/std_thread.h:286
eloqdata/tx_service#11 0x00007fc51826008c in std::thread::_State_impl<std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > > >::_M_run(void) (this=0x407fce85f40) at /usr/include/c++/12/bits/std_thread.h:231
eloqdata/tx_service#12 0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#13 0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#14 0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 24 (Thread 0x7fc4eb5126c0 (LWP 1667172) "eloqdoc"):
#0  __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x407fcaf86a0) at ./nptl/futex-internal.c:57
eloqdata/tx_service#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x407fcaf86a0) at ./nptl/futex-internal.c:87
eloqdata/tx_service#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x407fcaf86a0, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
eloqdata/tx_service#3  0x00007fc525a8e5f8 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x407fcaf8650, cond=0x407fcaf8678) at ./nptl/pthread_cond_wait.c:503
eloqdata/tx_service#4  ___pthread_cond_wait (cond=0x407fcaf8678, mutex=0x407fcaf8650) at ./nptl/pthread_cond_wait.c:627
eloqdata/tx_service#5  0x00007fc51825fd67 in std::condition_variable::wait<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()>::<lambda()> >(std::unique_lock<std::mutex> &, struct {...}) (this=0x407fcaf8678, __lock=..., __p=...) at /usr/include/c++/12/condition_variable:102
eloqdata/tx_service#6  0x00007fc51825f841 in operator() (__closure=0x407fce85fc8) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/store_handler/eloq_data_store_service/thread_worker_pool.cpp:41
eloqdata/tx_service#7  0x00007fc518260164 in std::__invoke_impl<void, EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> >(std::__invoke_other, struct {...} &&) (__f=...) at /usr/include/c++/12/bits/invoke.h:61
eloqdata/tx_service#8  0x00007fc518260127 in std::__invoke<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> >(struct {...} &&) (__fn=...) at /usr/include/c++/12/bits/invoke.h:96
eloqdata/tx_service#9  0x00007fc5182600d4 in std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > >::_M_invoke<0>(std::_Index_tuple<0>) (this=0x407fce85fc8) at /usr/include/c++/12/bits/std_thread.h:279
eloqdata/tx_service#10 0x00007fc5182600a8 in std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > >::operator()(void) (this=0x407fce85fc8) at /usr/include/c++/12/bits/std_thread.h:286
eloqdata/tx_service#11 0x00007fc51826008c in std::thread::_State_impl<std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > > >::_M_run(void) (this=0x407fce85fc0) at /usr/include/c++/12/bits/std_thread.h:231
eloqdata/tx_service#12 0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#13 0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#14 0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 23 (Thread 0x7fc4ed25f6c0 (LWP 1667171) "eloqdoc"):
#0  __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x407fcaf86a0) at ./nptl/futex-internal.c:57
eloqdata/tx_service#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x407fcaf86a0) at ./nptl/futex-internal.c:87
eloqdata/tx_service#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x407fcaf86a0, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
eloqdata/tx_service#3  0x00007fc525a8e5f8 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x407fcaf8650, cond=0x407fcaf8678) at ./nptl/pthread_cond_wait.c:503
eloqdata/tx_service#4  ___pthread_cond_wait (cond=0x407fcaf8678, mutex=0x407fcaf8650) at ./nptl/pthread_cond_wait.c:627
eloqdata/tx_service#5  0x00007fc51825fd67 in std::condition_variable::wait<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()>::<lambda()> >(std::unique_lock<std::mutex> &, struct {...}) (this=0x407fcaf8678, __lock=..., __p=...) at /usr/include/c++/12/condition_variable:102
eloqdata/tx_service#6  0x00007fc51825f841 in operator() (__closure=0x407fce85fb8) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/store_handler/eloq_data_store_service/thread_worker_pool.cpp:41
eloqdata/tx_service#7  0x00007fc518260164 in std::__invoke_impl<void, EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> >(std::__invoke_other, struct {...} &&) (__f=...) at /usr/include/c++/12/bits/invoke.h:61
eloqdata/tx_service#8  0x00007fc518260127 in std::__invoke<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> >(struct {...} &&) (__fn=...) at /usr/include/c++/12/bits/invoke.h:96
eloqdata/tx_service#9  0x00007fc5182600d4 in std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > >::_M_invoke<0>(std::_Index_tuple<0>) (this=0x407fce85fb8) at /usr/include/c++/12/bits/std_thread.h:279
eloqdata/tx_service#10 0x00007fc5182600a8 in std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > >::operator()(void) (this=0x407fce85fb8) at /usr/include/c++/12/bits/std_thread.h:286
eloqdata/tx_service#11 0x00007fc51826008c in std::thread::_State_impl<std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > > >::_M_run(void) (this=0x407fce85fb0) at /usr/include/c++/12/bits/std_thread.h:231
eloqdata/tx_service#12 0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#13 0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#14 0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 22 (Thread 0x7fc4eefac6c0 (LWP 1667170) "eloqdoc"):
#0  __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x407fcaf86a0) at ./nptl/futex-internal.c:57
eloqdata/tx_service#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x407fcaf86a0) at ./nptl/futex-internal.c:87
eloqdata/tx_service#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x407fcaf86a0, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
eloqdata/tx_service#3  0x00007fc525a8e5f8 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x407fcaf8650, cond=0x407fcaf8678) at ./nptl/pthread_cond_wait.c:503
eloqdata/tx_service#4  ___pthread_cond_wait (cond=0x407fcaf8678, mutex=0x407fcaf8650) at ./nptl/pthread_cond_wait.c:627
eloqdata/tx_service#5  0x00007fc51825fd67 in std::condition_variable::wait<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()>::<lambda()> >(std::unique_lock<std::mutex> &, struct {...}) (this=0x407fcaf8678, __lock=..., __p=...) at /usr/include/c++/12/condition_variable:102
eloqdata/tx_service#6  0x00007fc51825f841 in operator() (__closure=0x407fce85f88) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/store_handler/eloq_data_store_service/thread_worker_pool.cpp:41
eloqdata/tx_service#7  0x00007fc518260164 in std::__invoke_impl<void, EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> >(std::__invoke_other, struct {...} &&) (__f=...) at /usr/include/c++/12/bits/invoke.h:61
eloqdata/tx_service#8  0x00007fc518260127 in std::__invoke<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> >(struct {...} &&) (__fn=...) at /usr/include/c++/12/bits/invoke.h:96
eloqdata/tx_service#9  0x00007fc5182600d4 in std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > >::_M_invoke<0>(std::_Index_tuple<0>) (this=0x407fce85f88) at /usr/include/c++/12/bits/std_thread.h:279
eloqdata/tx_service#10 0x00007fc5182600a8 in std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > >::operator()(void) (this=0x407fce85f88) at /usr/include/c++/12/bits/std_thread.h:286
eloqdata/tx_service#11 0x00007fc51826008c in std::thread::_State_impl<std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > > >::_M_run(void) (this=0x407fce85f80) at /usr/include/c++/12/bits/std_thread.h:231
eloqdata/tx_service#12 0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#13 0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#14 0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 21 (Thread 0x7fc4f0cf96c0 (LWP 1667169) "eloqdoc"):
#0  __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x407fcaf86a0) at ./nptl/futex-internal.c:57
eloqdata/tx_service#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x407fcaf86a0) at ./nptl/futex-internal.c:87
eloqdata/tx_service#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x407fcaf86a0, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
eloqdata/tx_service#3  0x00007fc525a8e5f8 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x407fcaf8650, cond=0x407fcaf8678) at ./nptl/pthread_cond_wait.c:503
eloqdata/tx_service#4  ___pthread_cond_wait (cond=0x407fcaf8678, mutex=0x407fcaf8650) at ./nptl/pthread_cond_wait.c:627
eloqdata/tx_service#5  0x00007fc51825fd67 in std::condition_variable::wait<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()>::<lambda()> >(std::unique_lock<std::mutex> &, struct {...}) (this=0x407fcaf8678, __lock=..., __p=...) at /usr/include/c++/12/condition_variable:102
eloqdata/tx_service#6  0x00007fc51825f841 in operator() (__closure=0x407fce85f78) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/store_handler/eloq_data_store_service/thread_worker_pool.cpp:41
eloqdata/tx_service#7  0x00007fc518260164 in std::__invoke_impl<void, EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> >(std::__invoke_other, struct {...} &&) (__f=...) at /usr/include/c++/12/bits/invoke.h:61
eloqdata/tx_service#8  0x00007fc518260127 in std::__invoke<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> >(struct {...} &&) (__fn=...) at /usr/include/c++/12/bits/invoke.h:96
eloqdata/tx_service#9  0x00007fc5182600d4 in std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > >::_M_invoke<0>(std::_Index_tuple<0>) (this=0x407fce85f78) at /usr/include/c++/12/bits/std_thread.h:279
eloqdata/tx_service#10 0x00007fc5182600a8 in std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > >::operator()(void) (this=0x407fce85f78) at /usr/include/c++/12/bits/std_thread.h:286
eloqdata/tx_service#11 0x00007fc51826008c in std::thread::_State_impl<std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > > >::_M_run(void) (this=0x407fce85f70) at /usr/include/c++/12/bits/std_thread.h:231
eloqdata/tx_service#12 0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#13 0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#14 0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 20 (Thread 0x7fc4f2a466c0 (LWP 1667168) "eloqdoc"):
#0  __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x407fcaf86a0) at ./nptl/futex-internal.c:57
eloqdata/tx_service#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x407fcaf86a0) at ./nptl/futex-internal.c:87
eloqdata/tx_service#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x407fcaf86a0, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
eloqdata/tx_service#3  0x00007fc525a8e5f8 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x407fcaf8650, cond=0x407fcaf8678) at ./nptl/pthread_cond_wait.c:503
eloqdata/tx_service#4  ___pthread_cond_wait (cond=0x407fcaf8678, mutex=0x407fcaf8650) at ./nptl/pthread_cond_wait.c:627
eloqdata/tx_service#5  0x00007fc51825fd67 in std::condition_variable::wait<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()>::<lambda()> >(std::unique_lock<std::mutex> &, struct {...}) (this=0x407fcaf8678, __lock=..., __p=...) at /usr/include/c++/12/condition_variable:102
eloqdata/tx_service#6  0x00007fc51825f841 in operator() (__closure=0x407fce86018) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/store_handler/eloq_data_store_service/thread_worker_pool.cpp:41
eloqdata/tx_service#7  0x00007fc518260164 in std::__invoke_impl<void, EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> >(std::__invoke_other, struct {...} &&) (__f=...) at /usr/include/c++/12/bits/invoke.h:61
eloqdata/tx_service#8  0x00007fc518260127 in std::__invoke<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> >(struct {...} &&) (__fn=...) at /usr/include/c++/12/bits/invoke.h:96
eloqdata/tx_service#9  0x00007fc5182600d4 in std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > >::_M_invoke<0>(std::_Index_tuple<0>) (this=0x407fce86018) at /usr/include/c++/12/bits/std_thread.h:279
eloqdata/tx_service#10 0x00007fc5182600a8 in std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > >::operator()(void) (this=0x407fce86018) at /usr/include/c++/12/bits/std_thread.h:286
eloqdata/tx_service#11 0x00007fc51826008c in std::thread::_State_impl<std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > > >::_M_run(void) (this=0x407fce86010) at /usr/include/c++/12/bits/std_thread.h:231
eloqdata/tx_service#12 0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#13 0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#14 0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 19 (Thread 0x7fc4f47936c0 (LWP 1667167) "eloqdoc"):
#0  __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x407fcaf86a4) at ./nptl/futex-internal.c:57
eloqdata/tx_service#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x407fcaf86a4) at ./nptl/futex-internal.c:87
eloqdata/tx_service#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x407fcaf86a4, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
eloqdata/tx_service#3  0x00007fc525a8e5f8 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x407fcaf8650, cond=0x407fcaf8678) at ./nptl/pthread_cond_wait.c:503
eloqdata/tx_service#4  ___pthread_cond_wait (cond=0x407fcaf8678, mutex=0x407fcaf8650) at ./nptl/pthread_cond_wait.c:627
eloqdata/tx_service#5  0x00007fc51825fd67 in std::condition_variable::wait<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()>::<lambda()> >(std::unique_lock<std::mutex> &, struct {...}) (this=0x407fcaf8678, __lock=..., __p=...) at /usr/include/c++/12/condition_variable:102
eloqdata/tx_service#6  0x00007fc51825f841 in operator() (__closure=0x407fce85fa8) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/store_handler/eloq_data_store_service/thread_worker_pool.cpp:41
eloqdata/tx_service#7  0x00007fc518260164 in std::__invoke_impl<void, EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> >(std::__invoke_other, struct {...} &&) (__f=...) at /usr/include/c++/12/bits/invoke.h:61
eloqdata/tx_service#8  0x00007fc518260127 in std::__invoke<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> >(struct {...} &&) (__fn=...) at /usr/include/c++/12/bits/invoke.h:96
eloqdata/tx_service#9  0x00007fc5182600d4 in std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > >::_M_invoke<0>(std::_Index_tuple<0>) (this=0x407fce85fa8) at /usr/include/c++/12/bits/std_thread.h:279
eloqdata/tx_service#10 0x00007fc5182600a8 in std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > >::operator()(void) (this=0x407fce85fa8) at /usr/include/c++/12/bits/std_thread.h:286
eloqdata/tx_service#11 0x00007fc51826008c in std::thread::_State_impl<std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > > >::_M_run(void) (this=0x407fce85fa0) at /usr/include/c++/12/bits/std_thread.h:231
eloqdata/tx_service#12 0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#13 0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#14 0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 18 (Thread 0x7fc4f64e06c0 (LWP 1667166) "eloqdoc"):
#0  __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x407fcaf86a4) at ./nptl/futex-internal.c:57
eloqdata/tx_service#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x407fcaf86a4) at ./nptl/futex-internal.c:87
eloqdata/tx_service#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x407fcaf86a4, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
eloqdata/tx_service#3  0x00007fc525a8e5f8 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x407fcaf8650, cond=0x407fcaf8678) at ./nptl/pthread_cond_wait.c:503
eloqdata/tx_service#4  ___pthread_cond_wait (cond=0x407fcaf8678, mutex=0x407fcaf8650) at ./nptl/pthread_cond_wait.c:627
eloqdata/tx_service#5  0x00007fc51825fd67 in std::condition_variable::wait<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()>::<lambda()> >(std::unique_lock<std::mutex> &, struct {...}) (this=0x407fcaf8678, __lock=..., __p=...) at /usr/include/c++/12/condition_variable:102
eloqdata/tx_service#6  0x00007fc51825f841 in operator() (__closure=0x407fce85fe8) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/store_handler/eloq_data_store_service/thread_worker_pool.cpp:41
eloqdata/tx_service#7  0x00007fc518260164 in std::__invoke_impl<void, EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> >(std::__invoke_other, struct {...} &&) (__f=...) at /usr/include/c++/12/bits/invoke.h:61
eloqdata/tx_service#8  0x00007fc518260127 in std::__invoke<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> >(struct {...} &&) (__fn=...) at /usr/include/c++/12/bits/invoke.h:96
eloqdata/tx_service#9  0x00007fc5182600d4 in std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > >::_M_invoke<0>(std::_Index_tuple<0>) (this=0x407fce85fe8) at /usr/include/c++/12/bits/std_thread.h:279
eloqdata/tx_service#10 0x00007fc5182600a8 in std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > >::operator()(void) (this=0x407fce85fe8) at /usr/include/c++/12/bits/std_thread.h:286
eloqdata/tx_service#11 0x00007fc51826008c in std::thread::_State_impl<std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > > >::_M_run(void) (this=0x407fce85fe0) at /usr/include/c++/12/bits/std_thread.h:231
eloqdata/tx_service#12 0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#13 0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#14 0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 17 (Thread 0x7fc4f822d6c0 (LWP 1667165) "eloqdoc"):
#0  __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x407fcaf86a4) at ./nptl/futex-internal.c:57
eloqdata/tx_service#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x407fcaf86a4) at ./nptl/futex-internal.c:87
eloqdata/tx_service#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x407fcaf86a4, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
eloqdata/tx_service#3  0x00007fc525a8e5f8 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x407fcaf8650, cond=0x407fcaf8678) at ./nptl/pthread_cond_wait.c:503
eloqdata/tx_service#4  ___pthread_cond_wait (cond=0x407fcaf8678, mutex=0x407fcaf8650) at ./nptl/pthread_cond_wait.c:627
eloqdata/tx_service#5  0x00007fc51825fd67 in std::condition_variable::wait<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()>::<lambda()> >(std::unique_lock<std::mutex> &, struct {...}) (this=0x407fcaf8678, __lock=..., __p=...) at /usr/include/c++/12/condition_variable:102
eloqdata/tx_service#6  0x00007fc51825f841 in operator() (__closure=0x407fce85ff8) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/store_handler/eloq_data_store_service/thread_worker_pool.cpp:41
eloqdata/tx_service#7  0x00007fc518260164 in std::__invoke_impl<void, EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> >(std::__invoke_other, struct {...} &&) (__f=...) at /usr/include/c++/12/bits/invoke.h:61
eloqdata/tx_service#8  0x00007fc518260127 in std::__invoke<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> >(struct {...} &&) (__fn=...) at /usr/include/c++/12/bits/invoke.h:96
eloqdata/tx_service#9  0x00007fc5182600d4 in std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > >::_M_invoke<0>(std::_Index_tuple<0>) (this=0x407fce85ff8) at /usr/include/c++/12/bits/std_thread.h:279
eloqdata/tx_service#10 0x00007fc5182600a8 in std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > >::operator()(void) (this=0x407fce85ff8) at /usr/include/c++/12/bits/std_thread.h:286
eloqdata/tx_service#11 0x00007fc51826008c in std::thread::_State_impl<std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > > >::_M_run(void) (this=0x407fce85ff0) at /usr/include/c++/12/bits/std_thread.h:231
eloqdata/tx_service#12 0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#13 0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#14 0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 16 (Thread 0x7fc4f9f7a6c0 (LWP 1667164) "eloqdoc"):
#0  __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x407fcb8f7a8) at ./nptl/futex-internal.c:57
eloqdata/tx_service#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x407fcb8f7a8) at ./nptl/futex-internal.c:87
eloqdata/tx_service#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x407fcb8f7a8, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
eloqdata/tx_service#3  0x00007fc525a8e5f8 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x407fcb8f758, cond=0x407fcb8f780) at ./nptl/pthread_cond_wait.c:503
eloqdata/tx_service#4  ___pthread_cond_wait (cond=0x407fcb8f780, mutex=0x407fcb8f758) at ./nptl/pthread_cond_wait.c:627
eloqdata/tx_service#5  0x00007fc51825fd67 in std::condition_variable::wait<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()>::<lambda()> >(std::unique_lock<std::mutex> &, struct {...}) (this=0x407fcb8f780, __lock=..., __p=...) at /usr/include/c++/12/condition_variable:102
eloqdata/tx_service#6  0x00007fc51825f841 in operator() (__closure=0x407fce85f68) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/store_handler/eloq_data_store_service/thread_worker_pool.cpp:41
eloqdata/tx_service#7  0x00007fc518260164 in std::__invoke_impl<void, EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> >(std::__invoke_other, struct {...} &&) (__f=...) at /usr/include/c++/12/bits/invoke.h:61
eloqdata/tx_service#8  0x00007fc518260127 in std::__invoke<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> >(struct {...} &&) (__fn=...) at /usr/include/c++/12/bits/invoke.h:96
eloqdata/tx_service#9  0x00007fc5182600d4 in std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > >::_M_invoke<0>(std::_Index_tuple<0>) (this=0x407fce85f68) at /usr/include/c++/12/bits/std_thread.h:279
eloqdata/tx_service#10 0x00007fc5182600a8 in std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > >::operator()(void) (this=0x407fce85f68) at /usr/include/c++/12/bits/std_thread.h:286
eloqdata/tx_service#11 0x00007fc51826008c in std::thread::_State_impl<std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > > >::_M_run(void) (this=0x407fce85f60) at /usr/include/c++/12/bits/std_thread.h:231
eloqdata/tx_service#12 0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#13 0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#14 0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 15 (Thread 0x7fc4fbcc76c0 (LWP 1667163) "eloqdoc"):
#0  syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
eloqdata/tx_service#1  0x00007fc5263fe502 in bthread::futex_wait_private (timeout=0x7fc4fbcb8f90, expected=0, addr1=0x7fc4fbcb9050) at /home/xiexy/github.com/eloqdata/brpc/src/bthread/sys_futex.h:40
eloqdata/tx_service#2  bthread::wait_pthread (pw=..., abstime=<optimized out>, abstime@entry=0x7fc4fbcb9130) at /home/xiexy/github.com/eloqdata/brpc/src/bthread/butex.cpp:153
eloqdata/tx_service#3  0x00007fc5263fe927 in bthread::butex_wait_from_pthread (abstime=0x7fc4fbcb9130, expected_value=0, b=0x407fc727840, g=<optimized out>) at /home/xiexy/github.com/eloqdata/brpc/src/bthread/butex.cpp:669
eloqdata/tx_service#4  bthread::butex_wait (arg=0x407fc727840, expected_value=expected_value@entry=0, abstime=abstime@entry=0x7fc4fbcb9130) at /home/xiexy/github.com/eloqdata/brpc/src/bthread/butex.cpp:702
eloqdata/tx_service#5  0x00007fc5264017d7 in bthread_cond_timedwait(bthread_cond_t * __restrict__, bthread_mutex_t * __restrict__, const timespec * __restrict__) (c=0x407fcb8f668, m=0x407fcb8f650, abstime=0x7fc4fbcb9130) at /home/xiexy/github.com/eloqdata/brpc/src/bthread/condition_variable.cpp:137
eloqdata/tx_service#6  0x00007fc51a12c12e in bthread::ConditionVariable::wait_until (this=0x407fcb8f668, lock=..., duetime=...) at /usr/include/bthread/condition_variable.h:80
eloqdata/tx_service#7  0x00007fc51a12c0e5 in bthread::ConditionVariable::wait_for (this=0x407fcb8f668, lock=..., timeout_us=3000000) at /usr/include/bthread/condition_variable.h:70
eloqdata/tx_service#8  0x00007fc518262745 in EloqDS::TTLWrapperCache::TTLCheckWorker (this=0x407fcb8f650) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/store_handler/eloq_data_store_service/data_store_service.cpp:97
eloqdata/tx_service#9  0x00007fc51826249d in operator() (__closure=0x7fc4fbcb9310) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/store_handler/eloq_data_store_service/data_store_service.cpp:76
eloqdata/tx_service#10 0x00007fc51827003a in std::__invoke_impl<void, EloqDS::TTLWrapperCache::TTLWrapperCache()::<lambda()>&>(std::__invoke_other, struct {...} &) (__f=...) at /usr/include/c++/12/bits/invoke.h:61
eloqdata/tx_service#11 0x00007fc51826fbf9 in std::__invoke_r<void, EloqDS::TTLWrapperCache::TTLWrapperCache()::<lambda()>&>(struct {...} &) (__fn=...) at /usr/include/c++/12/bits/invoke.h:111
eloqdata/tx_service#12 0x00007fc51826f8ff in std::_Function_handler<void(), EloqDS::TTLWrapperCache::TTLWrapperCache()::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /usr/include/c++/12/bits/std_function.h:290
eloqdata/tx_service#13 0x00007fc527ca2204 in std::function<void ()>::operator()() const (this=0x7fc4fbcb9310) at /usr/include/c++/12/bits/std_function.h:591
eloqdata/tx_service#14 0x00007fc51825f8b3 in operator() (__closure=0x407fce85f98) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/store_handler/eloq_data_store_service/thread_worker_pool.cpp:57
eloqdata/tx_service#15 0x00007fc518260164 in std::__invoke_impl<void, EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> >(std::__invoke_other, struct {...} &&) (__f=...) at /usr/include/c++/12/bits/invoke.h:61
eloqdata/tx_service#16 0x00007fc518260127 in std::__invoke<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> >(struct {...} &&) (__fn=...) at /usr/include/c++/12/bits/invoke.h:96
eloqdata/tx_service#17 0x00007fc5182600d4 in std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > >::_M_invoke<0>(std::_Index_tuple<0>) (this=0x407fce85f98) at /usr/include/c++/12/bits/std_thread.h:279
eloqdata/tx_service#18 0x00007fc5182600a8 in std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > >::operator()(void) (this=0x407fce85f98) at /usr/include/c++/12/bits/std_thread.h:286
eloqdata/tx_service#19 0x00007fc51826008c in std::thread::_State_impl<std::thread::_Invoker<std::tuple<EloqDS::ThreadWorkerPool::ThreadWorkerPool(size_t)::<lambda()> > > >::_M_run(void) (this=0x407fce85f90) at /usr/include/c++/12/bits/std_thread.h:231
eloqdata/tx_service#20 0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#21 0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#22 0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 14 (Thread 0x7fc4fda146c0 (LWP 1667162) "AwsEventLoop 8"):
#0  0x00007fc525b1e546 in epoll_wait (epfd=29, events=events@entry=0x7fc4fda05e60, maxevents=maxevents@entry=100, timeout=timeout@entry=100000) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
eloqdata/tx_service#1  0x00007fc513bf85cc in aws_event_loop_listen_for_io_events (epoll_fd=<optimized out>, events=events@entry=0x7fc4fda05e60, timeout=timeout@entry=100000) at /home/xiexy/github.com/aws/aws/crt/aws-crt-cpp/crt/aws-c-io/source/linux/epoll_event_loop.c:564
eloqdata/tx_service#2  0x00007fc513bf8f47 in aws_event_loop_thread (args=<optimized out>) at /home/xiexy/github.com/aws/aws/crt/aws-crt-cpp/crt/aws-c-io/source/linux/epoll_event_loop.c:614
eloqdata/tx_service#3  0x00007fc51439c232 in thread_fn (arg=0x407fca6de40) at /home/xiexy/github.com/aws/aws/crt/aws-crt-cpp/crt/aws-c-common/source/posix/thread.c:177
eloqdata/tx_service#4  0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#5  0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 13 (Thread 0x7fc4ff7616c0 (LWP 1667161) "AwsEventLoop 7"):
#0  0x00007fc525b1e546 in epoll_wait (epfd=27, events=events@entry=0x7fc4ff752e60, maxevents=maxevents@entry=100, timeout=timeout@entry=100000) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
eloqdata/tx_service#1  0x00007fc513bf85cc in aws_event_loop_listen_for_io_events (epoll_fd=<optimized out>, events=events@entry=0x7fc4ff752e60, timeout=timeout@entry=100000) at /home/xiexy/github.com/aws/aws/crt/aws-crt-cpp/crt/aws-c-io/source/linux/epoll_event_loop.c:564
eloqdata/tx_service#2  0x00007fc513bf8f47 in aws_event_loop_thread (args=<optimized out>) at /home/xiexy/github.com/aws/aws/crt/aws-crt-cpp/crt/aws-c-io/source/linux/epoll_event_loop.c:614
eloqdata/tx_service#3  0x00007fc51439c232 in thread_fn (arg=0x407fca6dd60) at /home/xiexy/github.com/aws/aws/crt/aws-crt-cpp/crt/aws-c-common/source/posix/thread.c:177
eloqdata/tx_service#4  0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#5  0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 12 (Thread 0x7fc5014ae6c0 (LWP 1667160) "AwsEventLoop 6"):
#0  0x00007fc525b1e546 in epoll_wait (epfd=25, events=events@entry=0x7fc50149fe60, maxevents=maxevents@entry=100, timeout=timeout@entry=100000) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
eloqdata/tx_service#1  0x00007fc513bf85cc in aws_event_loop_listen_for_io_events (epoll_fd=<optimized out>, events=events@entry=0x7fc50149fe60, timeout=timeout@entry=100000) at /home/xiexy/github.com/aws/aws/crt/aws-crt-cpp/crt/aws-c-io/source/linux/epoll_event_loop.c:564
eloqdata/tx_service#2  0x00007fc513bf8f47 in aws_event_loop_thread (args=<optimized out>) at /home/xiexy/github.com/aws/aws/crt/aws-crt-cpp/crt/aws-c-io/source/linux/epoll_event_loop.c:614
eloqdata/tx_service#3  0x00007fc51439c232 in thread_fn (arg=0x407fca6d660) at /home/xiexy/github.com/aws/aws/crt/aws-crt-cpp/crt/aws-c-common/source/posix/thread.c:177
eloqdata/tx_service#4  0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#5  0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 11 (Thread 0x7fc5031fb6c0 (LWP 1667159) "AwsEventLoop 5"):
#0  0x00007fc525b1e546 in epoll_wait (epfd=23, events=events@entry=0x7fc5031ece60, maxevents=maxevents@entry=100, timeout=timeout@entry=100000) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
eloqdata/tx_service#1  0x00007fc513bf85cc in aws_event_loop_listen_for_io_events (epoll_fd=<optimized out>, events=events@entry=0x7fc5031ece60, timeout=timeout@entry=100000) at /home/xiexy/github.com/aws/aws/crt/aws-crt-cpp/crt/aws-c-io/source/linux/epoll_event_loop.c:564
eloqdata/tx_service#2  0x00007fc513bf8f47 in aws_event_loop_thread (args=<optimized out>) at /home/xiexy/github.com/aws/aws/crt/aws-crt-cpp/crt/aws-c-io/source/linux/epoll_event_loop.c:614
eloqdata/tx_service#3  0x00007fc51439c232 in thread_fn (arg=0x407fca6d7b0) at /home/xiexy/github.com/aws/aws/crt/aws-crt-cpp/crt/aws-c-common/source/posix/thread.c:177
eloqdata/tx_service#4  0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#5  0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 10 (Thread 0x7fc504f486c0 (LWP 1667158) "AwsEventLoop 4"):
#0  0x00007fc525b1e546 in epoll_wait (epfd=21, events=events@entry=0x7fc504f39e60, maxevents=maxevents@entry=100, timeout=timeout@entry=100000) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
eloqdata/tx_service#1  0x00007fc513bf85cc in aws_event_loop_listen_for_io_events (epoll_fd=<optimized out>, events=events@entry=0x7fc504f39e60, timeout=timeout@entry=100000) at /home/xiexy/github.com/aws/aws/crt/aws-crt-cpp/crt/aws-c-io/source/linux/epoll_event_loop.c:564
eloqdata/tx_service#2  0x00007fc513bf8f47 in aws_event_loop_thread (args=<optimized out>) at /home/xiexy/github.com/aws/aws/crt/aws-crt-cpp/crt/aws-c-io/source/linux/epoll_event_loop.c:614
eloqdata/tx_service#3  0x00007fc51439c232 in thread_fn (arg=0x407fca6d890) at /home/xiexy/github.com/aws/aws/crt/aws-crt-cpp/crt/aws-c-common/source/posix/thread.c:177
eloqdata/tx_service#4  0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#5  0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 9 (Thread 0x7fc506c956c0 (LWP 1667157) "AwsEventLoop 3"):
#0  0x00007fc525b1e546 in epoll_wait (epfd=19, events=events@entry=0x7fc506c86e60, maxevents=maxevents@entry=100, timeout=timeout@entry=100000) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
eloqdata/tx_service#1  0x00007fc513bf85cc in aws_event_loop_listen_for_io_events (epoll_fd=<optimized out>, events=events@entry=0x7fc506c86e60, timeout=timeout@entry=100000) at /home/xiexy/github.com/aws/aws/crt/aws-crt-cpp/crt/aws-c-io/source/linux/epoll_event_loop.c:564
eloqdata/tx_service#2  0x00007fc513bf8f47 in aws_event_loop_thread (args=<optimized out>) at /home/xiexy/github.com/aws/aws/crt/aws-crt-cpp/crt/aws-c-io/source/linux/epoll_event_loop.c:614
eloqdata/tx_service#3  0x00007fc51439c232 in thread_fn (arg=0x407fca6bde0) at /home/xiexy/github.com/aws/aws/crt/aws-crt-cpp/crt/aws-c-common/source/posix/thread.c:177
eloqdata/tx_service#4  0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#5  0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 8 (Thread 0x7fc5089e26c0 (LWP 1667156) "AwsEventLoop 2"):
#0  0x00007fc525b1e546 in epoll_wait (epfd=17, events=events@entry=0x7fc5089d3e60, maxevents=maxevents@entry=100, timeout=timeout@entry=100000) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
eloqdata/tx_service#1  0x00007fc513bf85cc in aws_event_loop_listen_for_io_events (epoll_fd=<optimized out>, events=events@entry=0x7fc5089d3e60, timeout=timeout@entry=100000) at /home/xiexy/github.com/aws/aws/crt/aws-crt-cpp/crt/aws-c-io/source/linux/epoll_event_loop.c:564
eloqdata/tx_service#2  0x00007fc513bf8f47 in aws_event_loop_thread (args=<optimized out>) at /home/xiexy/github.com/aws/aws/crt/aws-crt-cpp/crt/aws-c-io/source/linux/epoll_event_loop.c:614
eloqdata/tx_service#3  0x00007fc51439c232 in thread_fn (arg=0x407fca6d0b0) at /home/xiexy/github.com/aws/aws/crt/aws-crt-cpp/crt/aws-c-common/source/posix/thread.c:177
eloqdata/tx_service#4  0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#5  0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 7 (Thread 0x7fc50a72f6c0 (LWP 1667155) "AwsEventLoop 1"):
#0  0x00007fc525b1e546 in epoll_wait (epfd=15, events=events@entry=0x7fc50a720e60, maxevents=maxevents@entry=100, timeout=timeout@entry=100000) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
eloqdata/tx_service#1  0x00007fc513bf85cc in aws_event_loop_listen_for_io_events (epoll_fd=<optimized out>, events=events@entry=0x7fc50a720e60, timeout=timeout@entry=100000) at /home/xiexy/github.com/aws/aws/crt/aws-crt-cpp/crt/aws-c-io/source/linux/epoll_event_loop.c:564
eloqdata/tx_service#2  0x00007fc513bf8f47 in aws_event_loop_thread (args=<optimized out>) at /home/xiexy/github.com/aws/aws/crt/aws-crt-cpp/crt/aws-c-io/source/linux/epoll_event_loop.c:614
eloqdata/tx_service#3  0x00007fc51439c232 in thread_fn (arg=0x407fca6d970) at /home/xiexy/github.com/aws/aws/crt/aws-crt-cpp/crt/aws-c-common/source/posix/thread.c:177
eloqdata/tx_service#4  0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#5  0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 6 (Thread 0x7fc50c47c6c0 (LWP 1667154) "eloqdoc"):
#0  __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x407fcbec5f0) at ./nptl/futex-internal.c:57
eloqdata/tx_service#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x407fcbec5f0) at ./nptl/futex-internal.c:87
eloqdata/tx_service#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x407fcbec5f0, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
eloqdata/tx_service#3  0x00007fc525a8e5f8 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x407fcbec5a0, cond=0x407fcbec5c8) at ./nptl/pthread_cond_wait.c:503
eloqdata/tx_service#4  ___pthread_cond_wait (cond=0x407fcbec5c8, mutex=0x407fcbec5a0) at ./nptl/pthread_cond_wait.c:627
eloqdata/tx_service#5  0x00007fc51851f727 in std::condition_variable::wait<LogThread(Aws::Utils::Logging::DefaultLogSystem::LogSynchronizationData*, std::shared_ptr<std::basic_ostream<char> >, const Aws::String&, bool)::<lambda()> > (__p=..., __lock=..., this=0x407fcbec5c8) at /usr/include/c++/12/condition_variable:102
eloqdata/tx_service#6  LogThread (syncData=0x407fcbec5a0, logFile=std::shared_ptr<std::basic_ostream<char, std::char_traits<char> >> (use count 1, weak count 0) = {...}, filenamePrefix="/home/xiexy/edisk/service/eloqdata-mongodb/logs/aws_sdk_", rollLog=true) at /home/xiexy/github.com/aws/aws/src/aws-cpp-sdk-core/source/utils/logging/DefaultLogSystem.cpp:36
eloqdata/tx_service#7  0x00007fc51855653f in std::__invoke_impl<void, void (*)(Aws::Utils::Logging::DefaultLogSystem::LogSynchronizationData*, std::shared_ptr<std::basic_ostream<char, std::char_traits<char> > >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool), Aws::Utils::Logging::DefaultLogSystem::LogSynchronizationData*, std::shared_ptr<std::basic_ofstream<char, std::char_traits<char> > >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool> (__f=<optimized out>) at /usr/include/c++/12/bits/invoke.h:61
eloqdata/tx_service#8  std::__invoke<void (*)(Aws::Utils::Logging::DefaultLogSystem::LogSynchronizationData*, std::shared_ptr<std::basic_ostream<char, std::char_traits<char> > >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool), Aws::Utils::Logging::DefaultLogSystem::LogSynchronizationData*, std::shared_ptr<std::basic_ofstream<char, std::char_traits<char> > >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool> (__fn=<optimized out>) at /usr/include/c++/12/bits/invoke.h:96
eloqdata/tx_service#9  std::thread::_Invoker<std::tuple<void (*)(Aws::Utils::Logging::DefaultLogSystem::LogSynchronizationData*, std::shared_ptr<std::basic_ostream<char, std::char_traits<char> > >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool), Aws::Utils::Logging::DefaultLogSystem::LogSynchronizationData*, std::shared_ptr<std::basic_ofstream<char, std::char_traits<char> > >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool> >::_M_invoke<0ul, 1ul, 2ul, 3ul, 4ul> (this=<optimized out>) at /usr/include/c++/12/bits/std_thread.h:279
eloqdata/tx_service#10 std::thread::_Invoker<std::tuple<void (*)(Aws::Utils::Logging::DefaultLogSystem::LogSynchronizationData*, std::shared_ptr<std::basic_ostream<char, std::char_traits<char> > >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool), Aws::Utils::Logging::DefaultLogSystem::LogSynchronizationData*, std::shared_ptr<std::basic_ofstream<char, std::char_traits<char> > >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool> >::operator() (this=<optimized out>) at /usr/include/c++/12/bits/std_thread.h:286
eloqdata/tx_service#11 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(Aws::Utils::Logging::DefaultLogSystem::LogSynchronizationData*, std::shared_ptr<std::basic_ostream<char, std::char_traits<char> > >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool), Aws::Utils::Logging::DefaultLogSystem::LogSynchronizationData*, std::shared_ptr<std::basic_ofstream<char, std::char_traits<char> > >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool> > >::_M_run (this=<optimized out>) at /usr/include/c++/12/bits/std_thread.h:231
eloqdata/tx_service#12 0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#13 0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#14 0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 5 (Thread 0x7fc50e1c96c0 (LWP 1667153) "eloqdoc"):
#0  __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x407fc6a4ba8) at ./nptl/futex-internal.c:57
eloqdata/tx_service#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x407fc6a4ba8) at ./nptl/futex-internal.c:87
eloqdata/tx_service#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x407fc6a4ba8, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
eloqdata/tx_service#3  0x00007fc525a8e5f8 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x407fc6a4bb0, cond=0x407fc6a4b80) at ./nptl/pthread_cond_wait.c:503
eloqdata/tx_service#4  ___pthread_cond_wait (cond=0x407fc6a4b80, mutex=0x407fc6a4bb0) at ./nptl/pthread_cond_wait.c:627
eloqdata/tx_service#5  0x00007fc51b2d508b in std::condition_variable::wait<mongo::MongoSystemHandler::MongoSystemHandler()::<lambda()>::<lambda()> >(std::unique_lock<std::mutex> &, struct {...}) (this=0x407fc6a4b80, __lock=..., __p=...) at /usr/include/c++/12/condition_variable:102
eloqdata/tx_service#6  0x00007fc51b2d47be in operator() (__closure=0x407fc355a18) at src/mongo/db/modules/eloq/src/eloq_kv_engine.cpp:1300
eloqdata/tx_service#7  0x00007fc51b2db841 in std::__invoke_impl<void, mongo::MongoSystemHandler::MongoSystemHandler()::<lambda()> >(std::__invoke_other, struct {...} &&) (__f=...) at /usr/include/c++/12/bits/invoke.h:61
eloqdata/tx_service#8  0x00007fc51b2db650 in std::__invoke<mongo::MongoSystemHandler::MongoSystemHandler()::<lambda()> >(struct {...} &&) (__fn=...) at /usr/include/c++/12/bits/invoke.h:96
eloqdata/tx_service#9  0x00007fc51b2db2a0 in std::thread::_Invoker<std::tuple<mongo::MongoSystemHandler::MongoSystemHandler()::<lambda()> > >::_M_invoke<0>(std::_Index_tuple<0>) (this=0x407fc355a18) at /usr/include/c++/12/bits/std_thread.h:279
eloqdata/tx_service#10 0x00007fc51b2dafb8 in std::thread::_Invoker<std::tuple<mongo::MongoSystemHandler::MongoSystemHandler()::<lambda()> > >::operator()(void) (this=0x407fc355a18) at /usr/include/c++/12/bits/std_thread.h:286
eloqdata/tx_service#11 0x00007fc51b2dad98 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<mongo::MongoSystemHandler::MongoSystemHandler()::<lambda()> > > >::_M_run(void) (this=0x407fc355a10) at /usr/include/c++/12/bits/std_thread.h:231
eloqdata/tx_service#12 0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#13 0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#14 0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 4 (Thread 0x7fc50ff166c0 (LWP 1667152) "Backgro.kSource"):
#0  __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x407fcbec564) at ./nptl/futex-internal.c:57
eloqdata/tx_service#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x407fcbec564) at ./nptl/futex-internal.c:87
eloqdata/tx_service#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x407fcbec564, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
eloqdata/tx_service#3  0x00007fc525a8e5f8 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x407fcbec510, cond=0x407fcbec538) at ./nptl/pthread_cond_wait.c:503
eloqdata/tx_service#4  ___pthread_cond_wait (cond=0x407fcbec538, mutex=0x407fcbec510) at ./nptl/pthread_cond_wait.c:627
eloqdata/tx_service#5  0x00007fc51c776233 in std::condition_variable::wait<mongo::BackgroundThreadClockSource::_startTimerThread()::<lambda()>::<lambda()> >(std::unique_lock<std::mutex> &, struct {...}) (this=0x407fcbec538, __lock=..., __p=...) at /usr/include/c++/12/condition_variable:102
eloqdata/tx_service#6  0x00007fc51c775e27 in operator() (__closure=0x407fc3559b8) at src/mongo/util/background_thread_clock_source.cpp:120
eloqdata/tx_service#7  0x00007fc51c776740 in std::__invoke_impl<void, mongo::BackgroundThreadClockSource::_startTimerThread()::<lambda()> >(std::__invoke_other, struct {...} &&) (__f=...) at /usr/include/c++/12/bits/invoke.h:61
eloqdata/tx_service#8  0x00007fc51c776703 in std::__invoke<mongo::BackgroundThreadClockSource::_startTimerThread()::<lambda()> >(struct {...} &&) (__fn=...) at /usr/include/c++/12/bits/invoke.h:96
eloqdata/tx_service#9  0x00007fc51c7766b0 in std::thread::_Invoker<std::tuple<mongo::BackgroundThreadClockSource::_startTimerThread()::<lambda()> > >::_M_invoke<0>(std::_Index_tuple<0>) (this=0x407fc3559b8) at /usr/include/c++/12/bits/std_thread.h:279
eloqdata/tx_service#10 0x00007fc51c776684 in std::thread::_Invoker<std::tuple<mongo::BackgroundThreadClockSource::_startTimerThread()::<lambda()> > >::operator()(void) (this=0x407fc3559b8) at /usr/include/c++/12/bits/std_thread.h:286
eloqdata/tx_service#11 0x00007fc51c776668 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<mongo::BackgroundThreadClockSource::_startTimerThread()::<lambda()> > > >::_M_run(void) (this=0x407fc3559b0) at /usr/include/c++/12/bits/std_thread.h:231
eloqdata/tx_service#12 0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#13 0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#14 0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 3 (Thread 0x7fc511c636c0 (LWP 1667151) "signalP.gThread"):
#0  0x00007fc525a3d11a in __GI___sigtimedwait (set=set@entry=0x7fc522a5c060 <mongo::(anonymous namespace)::asyncSignals>, info=info@entry=0x7fc511c55170, timeout=timeout@entry=0x0) at ../sysdeps/unix/sysv/linux/sigtimedwait.c:31
eloqdata/tx_service#1  0x00007fc525a3c7ec in __GI___sigwait (set=0x7fc522a5c060 <mongo::(anonymous namespace)::asyncSignals>, sig=0x7fc511c55278) at ../sysdeps/unix/sysv/linux/sigwait.c:28
eloqdata/tx_service#2  0x00007fc522a573e7 in operator() (__closure=0x7fc511c55290) at src/mongo/util/signal_handlers.cpp:176
eloqdata/tx_service#3  0x00007fc522a574c6 in mongo::(anonymous namespace)::signalProcessingThread (rotate=mongo::LogFileStatus::kNeedToRotateLogFile) at src/mongo/util/signal_handlers.cpp:177
eloqdata/tx_service#4  0x00007fc522a5874a in std::__invoke_impl<void, void (*)(mongo::LogFileStatus), mongo::LogFileStatus> (__f=@0x407fcc5cdd0: 0x7fc522a57432 <mongo::(anonymous namespace)::signalProcessingThread(mongo::LogFileStatus)>) at /usr/include/c++/12/bits/invoke.h:61
eloqdata/tx_service#5  0x00007fc522a586ad in std::__invoke<void (*)(mongo::LogFileStatus), mongo::LogFileStatus> (__fn=@0x407fcc5cdd0: 0x7fc522a57432 <mongo::(anonymous namespace)::signalProcessingThread(mongo::LogFileStatus)>) at /usr/include/c++/12/bits/invoke.h:96
eloqdata/tx_service#6  0x00007fc522a5860d in std::thread::_Invoker<std::tuple<void (*)(mongo::LogFileStatus), mongo::LogFileStatus> >::_M_invoke<0ul, 1ul> (this=0x407fcc5cdc8) at /usr/include/c++/12/bits/std_thread.h:279
eloqdata/tx_service#7  0x00007fc522a585c2 in std::thread::_Invoker<std::tuple<void (*)(mongo::LogFileStatus), mongo::LogFileStatus> >::operator() (this=0x407fcc5cdc8) at /usr/include/c++/12/bits/std_thread.h:286
eloqdata/tx_service#8  0x00007fc522a585a2 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(mongo::LogFileStatus), mongo::LogFileStatus> > >::_M_run (this=0x407fcc5cdc0) at /usr/include/c++/12/bits/std_thread.h:231
eloqdata/tx_service#9  0x00007fc525ee6363 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
eloqdata/tx_service#10 0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#11 0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 2 (Thread 0x7fc5139b06c0 (LWP 1667150) "bvar_sampler"):
#0  0x00007fc525adcea1 in __GI___clock_nanosleep (clock_id=clock_id@entry=0, flags=flags@entry=0, req=req@entry=0x7fc5139a22d0, rem=rem@entry=0x0) at ../sysdeps/unix/sysv/linux/clock_nanosleep.c:78
eloqdata/tx_service#1  0x00007fc525ae1c97 in __GI___nanosleep (req=req@entry=0x7fc5139a22d0, rem=rem@entry=0x0) at ../sysdeps/unix/sysv/linux/nanosleep.c:25
eloqdata/tx_service#2  0x00007fc525b134a9 in usleep (useconds=<optimized out>) at ../sysdeps/posix/usleep.c:31
eloqdata/tx_service#3  0x00007fc5263e0b32 in bvar::detail::SamplerCollector::run (this=this@entry=0x407fc0e0d80) at /home/xiexy/github.com/eloqdata/brpc/src/bvar/detail/sampler.cpp:183
eloqdata/tx_service#4  0x00007fc5263e1804 in bvar::detail::SamplerCollector::sampling_thread (arg=0x407fc0e0d80) at /home/xiexy/github.com/eloqdata/brpc/src/bvar/detail/sampler.cpp:113
eloqdata/tx_service#5  0x00007fc525a8f6ba in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
eloqdata/tx_service#6  0x00007fc525b1e120 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 1 (Thread 0x7fc513a0a200 (LWP 1667140) "eloqdoc"):
#0  syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
eloqdata/tx_service#1  0x00007fc5263fe502 in bthread::futex_wait_private (timeout=0x0, expected=0, addr1=0x7fff0a891ce0) at /home/xiexy/github.com/eloqdata/brpc/src/bthread/sys_futex.h:40
eloqdata/tx_service#2  bthread::wait_pthread (pw=..., abstime=<optimized out>, abstime@entry=0x0) at /home/xiexy/github.com/eloqdata/brpc/src/bthread/butex.cpp:153
eloqdata/tx_service#3  0x00007fc5263fe927 in bthread::butex_wait_from_pthread (abstime=0x0, expected_value=1, b=0x407fc72dc00, g=<optimized out>) at /home/xiexy/github.com/eloqdata/brpc/src/bthread/butex.cpp:669
eloqdata/tx_service#4  bthread::butex_wait (arg=arg@entry=0x407fc72dc00, expected_value=expected_value@entry=1, abstime=abstime@entry=0x0) at /home/xiexy/github.com/eloqdata/brpc/src/bthread/butex.cpp:702
eloqdata/tx_service#5  0x00007fc526407d78 in bthread_id_join (id=...) at /home/xiexy/github.com/eloqdata/brpc/src/bthread/id.cpp:529
eloqdata/tx_service#6  0x00007fc5264a3f75 in brpc::Channel::CallMethod (this=0x7fff0a892030, method=0x407fd883940, controller_base=0x7fff0a892170, request=0x7fff0a892000, response=0x7fff0a891f80, done=0x0) at /home/xiexy/github.com/eloqdata/brpc/src/brpc/channel.cpp:581
eloqdata/tx_service#7  0x00007fc517b104c1 in braft::rtb::refresh_leader (group="lg0", timeout_ms=1000) at /home/xiexy/github.com/eloqdata/braft/src/braft/route_table.cpp:237
eloqdata/tx_service#8  0x00007fc51a462932 in txlog::LogAgent::RefreshLeader (this=0x407fcb2c408, log_group_id=0, timeout_ms=1000) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/tx_service/tx-log-protos/log_agent.cpp:170
eloqdata/tx_service#9  0x00007fc51a464c5b in txlog::LogAgent::ReplayLog (this=0x407fcb2c408, cc_node_group_id=0, term=1, source_ip="127.0.0.1", source_port=8083, log_group=-1, start_ts=0, interrupt=std::atomic<bool> = { false }, no_replay=false) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/tx_service/tx-log-protos/log_agent.cpp:512
eloqdata/tx_service#10 0x00007fc51b2e70ac in Eloq::MongoLogAgent::ReplayLog (this=0x407fcb2c400, cc_node_group_id=0, term=1, source_ip="127.0.0.1", source_port=8083, log_group=-1, start_ts=0, interrupt=std::atomic<bool> = { false }) at src/mongo/db/modules/eloq/src/base/eloq_log_agent.h:71
eloqdata/tx_service#11 0x00007fc519e76528 in txservice::Sharder::Init (this=0x7fc51bd24cc0 <txservice::Sharder::Instance(unsigned int, std::unordered_map<unsigned int, std::vector<txservice::NodeConfig, std::allocator<txservice::NodeConfig> >, std::hash<unsigned int>, std::equal_to<unsigned int>, std::allocator<std::pair<unsigned int const, std::vector<txservice::NodeConfig, std::allocator<txservice::NodeConfig> > > > > const*, unsigned long, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const*, std::vector<unsigned short, std::allocator<unsigned short> > const*, txservice::LocalCcShards*, std::unique_ptr<txservice::TxLog, std::default_delete<txservice::TxLog> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const*)::instance_>, node_id=0, ng_id=0, ng_configs=0x7fff0a893a10, config_version=2, txlog_ips=0x7fff0a8939a0, txlog_ports=0x7fff0a8939c0, hm_ip=0x7fff0a893ab0, hm_port=0x7fff0a893922, hm_bin_path=0x7fff0a893ad0, local_shards=0x407fc7d8400, log_agent=std::unique_ptr<txservice::TxLog> = {...}, local_path="local:///home/xiexy/edisk/service/eloqdata-mongodb/data/data", cluster_config_path="/home/xiexy/edisk/service/eloqdata-mongodb/data/data/tx_service/cluster_config", rep_group_cnt=3, enable_brpc_builtin_services=true, fork_host_manager=false) at /home/xiexy/eloqdata-mongo/src/mongo/db/modules/eloq/tx_service/src/sharder.cpp:490
eloqdata/tx_service#12 0x00007fc51b301a61 in txservice::TxService::Start (this=0x407fc7d8400, node_id=0, ng_id=0, ng_configs=0x7fff0a893a10, cluster_config_version=2, txlog_ips=0x7fff0a8939a0, txlog_ports=0x7fff0a8939c0, hm_ip=0x7fff0a893ab0, hm_port=0x7fff0a893922, hm_bin_path=0x7fff0a893ad0, conf=std::map with 12 elements = {...}, log_agent=std::unique_ptr<txservice::TxLog> = {...}, local_path="local:///home/xiexy/edisk/service/eloqdata-mongodb/data/data", cluster_config_path="/home/xiexy/edisk/service/eloqdata-mongodb/data/data/tx_service/cluster_config", enable_brpc_builtin_services=true, fork_host_manager=false) at src/mongo/db/modules/eloq/tx_service/include/tx_service.h:1233
eloqdata/tx_service#13 0x00007fc51b2ce0f5 in mongo::EloqKVEngine::EloqKVEngine (this=0x407fc6a4b00, path="/home/xiexy/edisk/service/eloqdata-mongodb/data/data") at src/mongo/db/modules/eloq/src/eloq_kv_engine.cpp:566
eloqdata/tx_service#14 0x00007fc51bcf2a5d in std::make_unique<mongo::EloqKVEngine, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&> () at /usr/include/c++/12/bits/unique_ptr.h:1065
eloqdata/tx_service#15 0x00007fc51bce0fe6 in mongo::(anonymous namespace)::EloqFactory::create (this=0x407fc3809d0, params=..., lockFile=0x407fc607d58) at src/mongo/db/modules/eloq/src/eloq_init.cpp:48
eloqdata/tx_service#16 0x00007fc525ddb558 in mongo::initializeStorageEngine (service=0x407fc6a0a00, initFlags=mongo::kNone) at src/mongo/db/storage/storage_engine_init.cpp:188
eloqdata/tx_service#17 0x00007fc527ffa481 in mongo::(anonymous namespace)::_initAndListen (listenPort=27017) at src/mongo/db/db.cpp:337
eloqdata/tx_service#18 0x00007fc527ffc8d6 in mongo::(anonymous namespace)::initAndListen (listenPort=27017) at src/mongo/db/db.cpp:689
eloqdata/tx_service#19 0x00007fc527ffe9d6 in mongo::mongoDbMain (argc=3, argv=0x7fff0a894688, envp=0x7fff0a8946a8) at src/mongo/db/db.cpp:1034
eloqdata/tx_service#20 0x0000563285b1ea55 in main (argc=3, argv=0x7fff0a894688, envp=0x7fff0a8946a8) at src/mongo/db/dbmain.cpp:49

[xiexiaoy]

After recompiling the brpc lib, tests can continue.

[xiexiaoy]

I20251120 14:43:41.351217 1793100 tx_operation.cpp:2471] txn: 16197 process acquire_all_intent_op_
I20251120 14:43:42.635396 1793100 tx_operation.cpp:2527] txn: 16197 process prepare_log_op_
I20251120 14:43:42.637625 1793100 tx_operation.cpp:2607] txn: 16197 process post_all_intent_op_
I20251120 14:43:43.757544 1793100 tx_operation.cpp:2646] txn: 16197 process acquire_all_lock_op_
I20251120 14:43:44.725726 1793100 template_cc_map.h:941] AcquireAllCc from txn: 16197, on table: __catalog, key: tpcc.ITEM is blocked.
W20251120 14:43:56.952723 1793100 cc_shard.cpp:873] orphan lock detected, lock holding txn: 16197, txn is initiated by this machine, try to recover
I20251120 14:43:56.952744 1793100 cc_shard.cpp:898] Txn #16197 locks __catalog, lock type: 3
I20251120 14:43:56.952750 1793100 cc_shard.cpp:898] Txn #16197 locks __cluster_config, lock type: 2
W20251120 14:43:56.952770 1793100 cc_shard.cpp:907] orphan lock detected, lock holding txn: 16197, try to recover
I20251120 14:43:56.954348 1793042 log_replay_service.cpp:169] replay service processes a RecoverTx task, txn: 16197
I20251120 14:43:56.955691 1793042 log_replay_service.cpp:979] The tx 16197 is ongoing. Does nothing for recovery.
W20251120 14:44:07.055887 1793100 cc_shard.cpp:873] orphan lock detected, lock holding txn: 16197, txn is initiated by this machine, try to recover
I20251120 14:44:07.055928 1793100 cc_shard.cpp:898] Txn #16197 locks __catalog, lock type: 3
I20251120 14:44:07.055934 1793100 cc_shard.cpp:898] Txn #16197 locks __cluster_config, lock type: 2
W20251120 14:44:07.055940 1793100 cc_shard.cpp:907] orphan lock detected, lock holding txn: 16197, try to recover
I20251120 14:44:07.182628 1793042 log_replay_service.cpp:169] replay service processes a RecoverTx task, txn: 16197
I20251120 14:44:07.183369 1793042 log_replay_service.cpp:979] The tx 16197 is ongoing. Does nothing for recovery.
W20251120 14:44:17.200047 1793100 cc_shard.cpp:873] orphan lock detected, lock holding txn: 16197, txn is initiated by this machine, try to recover
I20251120 14:44:17.200062 1793100 cc_shard.cpp:898] Txn #16197 locks __catalog, lock type: 3
I20251120 14:44:17.200067 1793100 cc_shard.cpp:898] Txn #16197 locks __cluster_config, lock type: 2
W20251120 14:44:17.200073 1793100 cc_shard.cpp:907] orphan lock detected, lock holding txn: 16197, try to recover
I20251120 14:44:17.201812 1793042 log_replay_service.cpp:169] replay service processes a RecoverTx task, txn: 16197
I20251120 14:44:17.203315 1793042 log_replay_service.cpp:979] The tx 16197 is ongoing. Does nothing for recovery.
W20251120 14:44:22.933171 1793100 cc_shard.cpp:873] orphan lock detected, lock holding txn: 16197, txn is initiated by this machine, try to recover
I20251120 14:44:22.933188 1793100 cc_shard.cpp:898] Txn #16197 locks __catalog, lock type: 3
I20251120 14:44:22.933193 1793100 cc_shard.cpp:898] Txn #16197 locks __cluster_config, lock type: 2
W20251120 14:44:22.933198 1793100 cc_shard.cpp:907] orphan lock detected, lock holding txn: 16197, try to recover
I20251120 14:44:23.594799 1793042 log_replay_service.cpp:169] replay service processes a RecoverTx task, txn: 16197
I20251120 14:44:23.595994 1793042 log_replay_service.cpp:979] The tx 16197 is ongoing. Does nothing for recovery.
I20251120 14:44:27.562458 1793100 template_cc_map.h:728] AcquireAllCc from txn: 16197, on table: __catalog, key: tpcc.ITEM is resumed from blocking. acquired_lock: 4, err_code: 0
I20251120 14:44:27.562542 1793100 tx_operation.cpp:3037] txn: 16197 process commit_log_op_
I20251120 14:44:27.563956 1793100 tx_operation.cpp:3108] txn: 16197 process upsert_kv_table_op_
I20251120 14:44:27.591809 1793100 tx_operation.cpp:2886] txn: 16197 process update_sequence_table_op_

A drop collection command blocked on update_sequence_table_op_. AcquireWriteOperation didn't finish.

[xiexiaoy]

Discoverd another bug eloqdata/tx_service#237