Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

libuv dead cycle in cmpxchgi #2462

Closed
wangshangxiu opened this issue Sep 9, 2019 · 6 comments
Closed

libuv dead cycle in cmpxchgi #2462

wangshangxiu opened this issue Sep 9, 2019 · 6 comments
Labels

Comments

@wangshangxiu
Copy link

wangshangxiu commented Sep 9, 2019

Stack:

Thread 1 (Thread 0x7fdc6df578c0 (LWP 15293)):
#0  0x00007fdc6c4c27bf in cmpxchgi (ptr=0x7fdc08073848, oldval=2, newval=0) at ../src/unix/atomic-ops.h:34
#1  0x00007fdc6c4c29e1 in uv__async_spin (handle=0x7fdc080737f0) at ../src/unix/async.c:88
#2  0x00007fdc6c4c2ce2 in uv__async_io (loop=0x7fdc6c6e5ac0 <default_loop_struct>, w=0x7fdc6c6e5c88 <default_loop_struct+456>, events=1) at ../src/unix/async.c:141
#3  0x00007fdc6c4d8716 in uv__io_poll (loop=0x7fdc6c6e5ac0 <default_loop_struct>, timeout=0) at ../src/unix/linux-core.c:384
#4  0x00007fdc6c4c37d0 in uv_run (loop=0x7fdc6c6e5ac0 <default_loop_struct>, mode=UV_RUN_DEFAULT) at ../src/unix/core.c:359
#5  0x00000000004127b3 in CIMFrame::StartFrame (this=0xbf1760) at /home/thz/imServer/cm/imframe.cpp:87
#6  0x00000000004129ec in main (argc=1, argv=0x7ffcd7c8b7f8) at /home/thz/imServer/cm/imframe.cpp:126

top:

top - 10:51:36 up 7 days, 20:14,  6 users,  load average: 1.60, 1.54, 1.42
Tasks: 142 total,   2 running, 140 sleeping,   0 stopped,   0 zombie
%Cpu0  :  1.7 us,  3.0 sy,  0.0 ni, 95.3 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu1  :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu2  :  1.0 us,  3.4 sy,  0.0 ni, 95.3 id,  0.0 wa,  0.0 hi,  0.3 si,  0.0 st
%Cpu3  :  1.0 us,  3.1 sy,  0.0 ni, 95.9 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem :  3881028 total,   797548 free,  1262764 used,  1820716 buff/cache
KiB Swap:  2097148 total,  2057680 free,    39468 used.  2003932 avail Mem 

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND                                                                                                                                                                             
15293 root      20   0 1745592  33992   7860 R 106.3  0.9   4362:22 cm_server                                                                                                                                                                           
15315 root      20   0 3763556  24508   8764 S   5.7  0.6 262:42.03 msg_server                                                                                                                                                                          
20741 root      20   0  117032   1248    924 S   1.3  0.0   0:01.41 netstat                                                                                                                                                                             
15296 root      20   0 1221864  17448   8436 S   1.0  0.4  55:42.84 channel_server                                                                                                                                                                      
15299 root      20   0 1372056  11280   6964 S   0.7  0.3  28:20.86 login_server                                                                                                                                                                        
20748 root      20   0  161964   2248   1576 R   0.7  0.1   0:00.16 top                                                                                                                                                                                 
  810 root      20   0  298712   1712   1332 S   0.3  0.0  10:37.85 vmtoolsd                                                                                                                                                                            
15302 root      20   0  438888  12632   8868 S   0.3  0.3  10:26.04 notify_server                                                                                                                                                                       
15362 root      20   0  504320   9116   7332 S   0.3  0.2  10:19.66 desktop_server                                                                                                                                                                      
    1 root      20   0  191128   2456   1400 S   0.0  0.1   0:14.63 systemd                                                                                                                                                                             
    2 root      20   0       0      0      0 S   0.0  0.0   0:00.89 kthreadd   

Stack is the last stack of cm_server base on libuv , top is cm_server runing status, in fact ,it doesn't run , just in libuv.

@bnoordhuis
Copy link
Member

Can you fill out the issue template? Kernel and libuv version are important to know.

I infer that "dead cycle" means it busy-loops? Can you attach to the process with gdb and post the output of thread apply all backtrace full?

Is there an easy way for me to reproduce the busy loop?

@wangshangxiu
Copy link
Author

wangshangxiu commented Sep 10, 2019

Can you fill out the issue template? Kernel and libuv version are important to know.

I infer that "dead cycle" means it busy-loops? Can you attach to the process with gdb and post the output of thread apply all backtrace full?

Is there an easy way for me to reproduce the busy loop?

The libuv I complied is V1.x , debug version.

Linux localhost.localdomain 3.10.0-862.14.4.el7.x86_64 #1 SMP Wed Sep 26 15:12:11 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

The "busy loop" run in main thread 1, what I do is I hope the socket wirting opration happend in main thread , so I use the uv_asyn_t object to notify loop thread , the uv_asyn_init callback to write data in loop thread . But I create so many uv_asyn_t object in task thread .for every write req , I create on uv_asyn_t and put them back to a queue in loop thread after its callback finished. long time , this happend.

(gdb) thread apply all backtrace full 

Thread 37 (Thread 0x7fdc6686c700 (LWP 15304)):
#0  0x00007fdc6b9b8c73 in select () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fdc67c0a5a5 in apr_sleep () from /usr/lib64/libapr-1.so.0
No symbol table info available.
#2  0x00007fdc6afddf79 in log4cxx::helpers::FileWatchdog::run (data=0xbdd220) at filewatchdog.cpp:76
        pThis = 0xbdd220
        interrupted = 0
#3  0x00007fdc6b03bdfe in log4cxx::helpers::Thread::launcher (thread=0xbe1ba8, data=0xbe1b90) at threadcxx.cpp:100
        package = 0xbe1b90
        tls = <optimized out>
        alive = {alive = 0xbdd270}
        retval = <optimized out>
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 36 (Thread 0x7fdc6606b700 (LWP 15332)):
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
---Type <return> to continue, or q <return> to quit---
#1  0x00007fdc6b988404 in sleep () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007fdc6d9e0888 in CZookeeper::OnThreadRun (this=0xbdc880) at /home/thz/imServer/baseuv/zookeeper.cpp:68
        i = 4
        sPath = {static npos = <optimized out>, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0xbdcde8 "/usdesktop"}}
        it = {_M_node = 0xbdc8c8}
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xbdc9b8) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xbdc9b8
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 35 (Thread 0x7fdc6586a700 (LWP 15333)):
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fdc6b9b9404 in usleep () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007fdc6d9cafa4 in CPacket::OnThreadRun (this=0xbf7cf0) at /home/thz/imServer/baseuv/packet.cpp:125
        i = 0
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xbf7cf0) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xbf7cf0
---Type <return> to continue, or q <return> to quit---
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 34 (Thread 0x7fdc65069700 (LWP 15334)):
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fdc6b9b9404 in usleep () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007fdc6d9cafa4 in CPacket::OnThreadRun (this=0xbfabe0) at /home/thz/imServer/baseuv/packet.cpp:125
        i = 0
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xbfabe0) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xbfabe0
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 33 (Thread 0x7fdc64868700 (LWP 15335)):
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
---Type <return> to continue, or q <return> to quit---
#1  0x00007fdc6b9b9404 in usleep () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007fdc6d9cafa4 in CPacket::OnThreadRun (this=0xbfb590) at /home/thz/imServer/baseuv/packet.cpp:125
        i = 0
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xbfb590) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xbfb590
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 32 (Thread 0x7fdc5ffff700 (LWP 15338)):
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fdc6b9b9404 in usleep () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007fdc6d9cafa4 in CPacket::OnThreadRun (this=0xbfc620) at /home/thz/imServer/baseuv/packet.cpp:125
        i = 0
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xbfc620) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xbfc620
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
---Type <return> to continue, or q <return> to quit---
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 31 (Thread 0x7fdc57fff700 (LWP 15339)):
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fdc6b9b9404 in usleep () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007fdc6d9cafa4 in CPacket::OnThreadRun (this=0xbfcca0) at /home/thz/imServer/baseuv/packet.cpp:125
        i = 0
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xbfcca0) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xbfcca0
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 30 (Thread 0x7fdc5f7fe700 (LWP 15340)):
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fdc6b9b9404 in usleep () from /lib64/libc.so.6
No symbol table info available.
---Type <return> to continue, or q <return> to quit---
#2  0x00007fdc6d9cafa4 in CPacket::OnThreadRun (this=0xbff640) at /home/thz/imServer/baseuv/packet.cpp:125
        i = 0
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xbff640) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xbff640
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 29 (Thread 0x7fdc5effd700 (LWP 15341)):
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fdc6b9b9404 in usleep () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007fdc6d9cafa4 in CPacket::OnThreadRun (this=0xbfbef0) at /home/thz/imServer/baseuv/packet.cpp:125
        i = 0
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xbfbef0) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xbfbef0
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.
---Type <return> to continue, or q <return> to quit---

Thread 28 (Thread 0x7fdc5e5e9700 (LWP 15342)):
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fdc6b9b9404 in usleep () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007fdc6d9cafa4 in CPacket::OnThreadRun (this=0xc01490) at /home/thz/imServer/baseuv/packet.cpp:125
        i = 0
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xc01490) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xc01490
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 27 (Thread 0x7fdc5dde8700 (LWP 15344)):
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fdc6b9b9404 in usleep () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007fdc6d9cafa4 in CPacket::OnThreadRun (this=0xc01ac0) at /home/thz/imServer/baseuv/packet.cpp:125
        i = 0
---Type <return> to continue, or q <return> to quit---
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xc01ac0) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xc01ac0
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 26 (Thread 0x7fdc5cde6700 (LWP 15346)):
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fdc6b9b9404 in usleep () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007fdc6d9cafa4 in CPacket::OnThreadRun (this=0xc04460) at /home/thz/imServer/baseuv/packet.cpp:125
        i = 0
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xc04460) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xc04460
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 25 (Thread 0x7fdc56ffd700 (LWP 15348)):
---Type <return> to continue, or q <return> to quit---
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fdc6b9b9404 in usleep () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007fdc6d9cafa4 in CPacket::OnThreadRun (this=0xc04e80) at /home/thz/imServer/baseuv/packet.cpp:125
        i = 0
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xc04e80) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xc04e80
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 24 (Thread 0x7fdc567fc700 (LWP 15349)):
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fdc6b9b9404 in usleep () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007fdc6d9cafa4 in CPacket::OnThreadRun (this=0xc05c20) at /home/thz/imServer/baseuv/packet.cpp:125
        i = 0
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xc05c20) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xc05c20
---Type <return> to continue, or q <return> to quit---
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 23 (Thread 0x7fdc55ffb700 (LWP 15350)):
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fdc6b9b9404 in usleep () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007fdc6d9cafa4 in CPacket::OnThreadRun (this=0xc06250) at /home/thz/imServer/baseuv/packet.cpp:125
        i = 0
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xc06250) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xc06250
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 22 (Thread 0x7fdc557fa700 (LWP 15351)):
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
---Type <return> to continue, or q <return> to quit---
#1  0x00007fdc6b9b9404 in usleep () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007fdc6d9cafa4 in CPacket::OnThreadRun (this=0xc000f0) at /home/thz/imServer/baseuv/packet.cpp:125
        i = 0
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xc000f0) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xc000f0
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 21 (Thread 0x7fdc54ff9700 (LWP 15352)):
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fdc6b9b9404 in usleep () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007fdc6d9cafa4 in CPacket::OnThreadRun (this=0xc008c0) at /home/thz/imServer/baseuv/packet.cpp:125
        i = 0
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xc008c0) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xc008c0
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
---Type <return> to continue, or q <return> to quit---
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 20 (Thread 0x7fdc547f8700 (LWP 15353)):
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fdc6b9b9404 in usleep () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007fdc6d9cafa4 in CPacket::OnThreadRun (this=0xc0b0b0) at /home/thz/imServer/baseuv/packet.cpp:125
        i = 0
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xc0b0b0) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xc0b0b0
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 19 (Thread 0x7fdc53ff7700 (LWP 15354)):
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fdc6b9b9404 in usleep () from /lib64/libc.so.6
No symbol table info available.
---Type <return> to continue, or q <return> to quit---
#2  0x00007fdc6d9cafa4 in CPacket::OnThreadRun (this=0xc0b6e0) at /home/thz/imServer/baseuv/packet.cpp:125
        i = 0
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xc0b6e0) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xc0b6e0
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 18 (Thread 0x7fdc537f6700 (LWP 15355)):
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fdc6b9b9404 in usleep () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007fdc6d9cafa4 in CPacket::OnThreadRun (this=0xc0e080) at /home/thz/imServer/baseuv/packet.cpp:125
        i = 0
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xc0e080) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xc0e080
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.
---Type <return> to continue, or q <return> to quit---

Thread 17 (Thread 0x7fdc52ff5700 (LWP 15356)):
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fdc6b9b9404 in usleep () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007fdc6d9cafa4 in CPacket::OnThreadRun (this=0xc0eaa0) at /home/thz/imServer/baseuv/packet.cpp:125
        i = 0
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xc0eaa0) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xc0eaa0
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 16 (Thread 0x7fdc527f4700 (LWP 15357)):
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fdc6b9b9404 in usleep () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007fdc6d9cafa4 in CPacket::OnThreadRun (this=0xc0f7f0) at /home/thz/imServer/baseuv/packet.cpp:125
        i = 0
---Type <return> to continue, or q <return> to quit---
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xc0f7f0) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xc0f7f0
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 15 (Thread 0x7fdc51ff3700 (LWP 15358)):
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fdc6b9b9404 in usleep () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007fdc6d9cafa4 in CPacket::OnThreadRun (this=0xc0fe70) at /home/thz/imServer/baseuv/packet.cpp:125
        i = 0
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xc0fe70) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xc0fe70
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 14 (Thread 0x7fdc517f2700 (LWP 15359)):
---Type <return> to continue, or q <return> to quit---
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fdc6b9b9404 in usleep () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007fdc6d9cafa4 in CPacket::OnThreadRun (this=0xc12810) at /home/thz/imServer/baseuv/packet.cpp:125
        i = 0
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xc12810) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xc12810
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 13 (Thread 0x7fdc50ff1700 (LWP 15360)):
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fdc6b9b9404 in usleep () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007fdc6d9cafa4 in CPacket::OnThreadRun (this=0xc13230) at /home/thz/imServer/baseuv/packet.cpp:125
        i = 0
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xc13230) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xc13230
---Type <return> to continue, or q <return> to quit---
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 12 (Thread 0x7fdc507f0700 (LWP 15361)):
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fdc6b9b9404 in usleep () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007fdc6d9cafa4 in CPacket::OnThreadRun (this=0xc13fd0) at /home/thz/imServer/baseuv/packet.cpp:125
        i = 0
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xc13fd0) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xc13fd0
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 11 (Thread 0x7fdc4ffef700 (LWP 15363)):
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
---Type <return> to continue, or q <return> to quit---
#1  0x00007fdc6b9b9404 in usleep () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007fdc6d9cafa4 in CPacket::OnThreadRun (this=0xc14600) at /home/thz/imServer/baseuv/packet.cpp:125
        i = 0
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xc14600) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xc14600
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 10 (Thread 0x7fdc4f7ee700 (LWP 15364)):
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fdc6b9b9404 in usleep () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007fdc6d9cafa4 in CPacket::OnThreadRun (this=0xc16fa0) at /home/thz/imServer/baseuv/packet.cpp:125
        i = 0
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xc16fa0) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xc16fa0
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
---Type <return> to continue, or q <return> to quit---
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 9 (Thread 0x7fdc4efed700 (LWP 15365)):
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fdc6b9b9404 in usleep () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007fdc6d9cafa4 in CPacket::OnThreadRun (this=0xc179c0) at /home/thz/imServer/baseuv/packet.cpp:125
        i = 0
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xc179c0) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xc179c0
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 8 (Thread 0x7fdc4e7ec700 (LWP 15366)):
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fdc6b9b9404 in usleep () from /lib64/libc.so.6
No symbol table info available.
---Type <return> to continue, or q <return> to quit---
#2  0x00007fdc6d9cafa4 in CPacket::OnThreadRun (this=0xc18760) at /home/thz/imServer/baseuv/packet.cpp:125
        i = 0
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xc18760) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xc18760
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 7 (Thread 0x7fdc4dfeb700 (LWP 15367)):
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fdc6b9b9404 in usleep () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007fdc6d9cafa4 in CPacket::OnThreadRun (this=0xc18d90) at /home/thz/imServer/baseuv/packet.cpp:125
        i = 0
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xc18d90) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xc18d90
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.
---Type <return> to continue, or q <return> to quit---

Thread 6 (Thread 0x7fdc4d7ea700 (LWP 15368)):
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fdc6b9b9404 in usleep () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007fdc6d9cafa4 in CPacket::OnThreadRun (this=0xc098c0) at /home/thz/imServer/baseuv/packet.cpp:125
        i = 0
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xc098c0) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xc098c0
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 5 (Thread 0x7fdc4cfe9700 (LWP 15369)):
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fdc6b9b9404 in usleep () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007fdc6d9cafa4 in CPacket::OnThreadRun (this=0xc1de60) at /home/thz/imServer/baseuv/packet.cpp:125
        i = 0
---Type <return> to continue, or q <return> to quit---
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xc1de60) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xc1de60
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 4 (Thread 0x7fdc4c7e8700 (LWP 15370)):
#0  0x00007fdc6b98856d in nanosleep () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fdc6b9b9404 in usleep () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007fdc6d9cafa4 in CPacket::OnThreadRun (this=0xc1ebb0) at /home/thz/imServer/baseuv/packet.cpp:125
        i = 0
#3  0x00007fdc6d9dbaf1 in CThread::StartRoutine (arg=0xc1ebb0) at /home/thz/imServer/baseuv/thread.cpp:31
        pThread = 0xc1ebb0
#4  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 3 (Thread 0x7fdc577fe700 (LWP 20749)):
---Type <return> to continue, or q <return> to quit---
#0  0x00007fdc6b9b6f0d in poll () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fdc6cb64c22 in do_io (v=0x7fdc58001bf0) at src/mt_adaptor.c:387
        tv = {tv_sec = 1, tv_usec = 333000}
        fd = 14
        interest = 2
        timeout = <optimized out>
        maxfd = <optimized out>
        zh = 0x7fdc58001bf0
        fds = {{fd = 12, events = 1, revents = 0}, {fd = 14, events = 1, revents = 0}}
        adaptor_threads = 0x7fdc58002720
        __func__ = "do_io"
#2  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#3  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 2 (Thread 0x7fdc5d5e7700 (LWP 20750)):
#0  0x00007fdc6cd79995 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
No symbol table info available.
#1  0x00007fdc6cb64d6b in do_completion (v=0x7fdc58001bf0) at src/mt_adaptor.c:463
        zh = 0x7fdc58001bf0
---Type <return> to continue, or q <return> to quit---
        __func__ = "do_completion"
#2  0x00007fdc6cd75e25 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#3  0x00007fdc6b9c1bad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 1 (Thread 0x7fdc6df578c0 (LWP 15293)):
#0  0x00007fdc6c4c27bf in cmpxchgi (ptr=0x7fdc08073848, oldval=2, newval=0) at ../src/unix/atomic-ops.h:34
        out = 0
#1  0x00007fdc6c4c29e1 in uv__async_spin (handle=0x7fdc080737f0) at ../src/unix/async.c:88
        rc = 0
#2  0x00007fdc6c4c2ce2 in uv__async_io (loop=0x7fdc6c6e5ac0 <default_loop_struct>, w=0x7fdc6c6e5c88 <default_loop_struct+456>, events=1) at ../src/unix/async.c:141
        buf = "\366&\000\000\000\000\000\000\265\350\235m\334\177\000\000\200\201\310\327\374\177\000\000\310\337@\000\024\000\000\000\240w\264\001\000\000\000\000\340\201\310\327\374\177\000\000\020\202\310\327\374\177\000\000\000\000\000\000\000\000\000\000\020\202\310\327\377\377\377\377\060\000\000\000\060\000\000\000x\202\310\327\374\177\000\000\260\201\310\327\374\177\000\000\000\000\000\000\377\377\377\377\210\243\237\001\000\000\000\000Z\031\255m\334\177\000\000\275\000\000\000\000\000\000\000\343\035\255m\334\177", '\000' <repeats 11 times>, "\202\310\327\374\177\000\000~\244\234m\334\177\000\000\020\202\310\327\374\177\000\000\230\031\000<\334\177\000\000 \202\310\327\374\177\000\000S\247\234m\334\177\000\000\230\031\000<\334\177\000\000"...
        r = 8
        queue = {0x7fdc08073838, 0x7fdc1406cd88}
        q = 0x7fdc08073838
        h = 0x7fdc080737f0
        __PRETTY_FUNCTION__ = "uv__async_io"
#3  0x00007fdc6c4d8716 in uv__io_poll (loop=0x7fdc6c6e5ac0 <default_loop_struct>, timeout=0) at ../src/unix/linux-core.c:384
---Type <return> to continue, or q <return> to quit---
        max_safe_timeout = 1789569
        events = {{events = 1, data = {ptr = 0x9, fd = 9, u32 = 9, u64 = 9}}, {events = 25, data = {ptr = 0x19, fd = 25, u32 = 25, u64 = 25}}, {events = 17, data = {ptr = 0x17, fd = 23, u32 = 23, u64 = 23}}, {
            events = 17, data = {ptr = 0x34, fd = 52, u32 = 52, u64 = 52}}, {events = 17, data = {ptr = 0x21, fd = 33, u32 = 33, u64 = 33}}, {events = 17, data = {ptr = 0x53, fd = 83, u32 = 83, u64 = 83}}, {
            events = 17, data = {ptr = 0x5c, fd = 92, u32 = 92, u64 = 92}}, {events = 25, data = {ptr = 0x3c, fd = 60, u32 = 60, u64 = 60}}, {events = 17, data = {ptr = 0x56, fd = 86, u32 = 86, u64 = 86}}, {
            events = 25, data = {ptr = 0x46, fd = 70, u32 = 70, u64 = 70}}, {events = 17, data = {ptr = 0x69, fd = 105, u32 = 105, u64 = 105}}, {events = 17, data = {ptr = 0x41, fd = 65, u32 = 65, u64 = 65}}, {
            events = 25, data = {ptr = 0x61, fd = 97, u32 = 97, u64 = 97}}, {events = 17, data = {ptr = 0x65, fd = 101, u32 = 101, u64 = 101}}, {events = 25, data = {ptr = 0x36, fd = 54, u32 = 54, u64 = 54}}, {
            events = 17, data = {ptr = 0x5e, fd = 94, u32 = 94, u64 = 94}}, {events = 17, data = {ptr = 0x4c, fd = 76, u32 = 76, u64 = 76}}, {events = 17, data = {ptr = 0x2f, fd = 47, u32 = 47, u64 = 47}}, {
            events = 17, data = {ptr = 0x6c, fd = 108, u32 = 108, u64 = 108}}, {events = 25, data = {ptr = 0x48, fd = 72, u32 = 72, u64 = 72}}, {events = 25, data = {ptr = 0x67, fd = 103, u32 = 103, u64 = 103}}, {
            events = 17, data = {ptr = 0x60, fd = 96, u32 = 96, u64 = 96}}, {events = 17, data = {ptr = 0x70, fd = 112, u32 = 112, u64 = 112}}, {events = 17, data = {ptr = 0x3b, fd = 59, u32 = 59, u64 = 59}}, {
            events = 17, data = {ptr = 0x2e, fd = 46, u32 = 46, u64 = 46}}, {events = 1, data = {ptr = 0x4f, fd = 79, u32 = 79, u64 = 79}}, {events = 1, data = {ptr = 0x2b, fd = 43, u32 = 43, u64 = 43}}, {
            events = 25, data = {ptr = 0x3a, fd = 58, u32 = 58, u64 = 58}}, {events = 17, data = {ptr = 0x2c, fd = 44, u32 = 44, u64 = 44}}, {events = 25, data = {ptr = 0x40, fd = 64, u32 = 64, u64 = 64}}, {
            events = 25, data = {ptr = 0x3f, fd = 63, u32 = 63, u64 = 63}}, {events = 25, data = {ptr = 0x25, fd = 37, u32 = 37, u64 = 37}}, {events = 17, data = {ptr = 0x29, fd = 41, u32 = 41, u64 = 41}}, {
            events = 25, data = {ptr = 0x3e, fd = 62, u32 = 62, u64 = 62}}, {events = 17, data = {ptr = 0x2d, fd = 45, u32 = 45, u64 = 45}}, {events = 17, data = {ptr = 0x42, fd = 66, u32 = 66, u64 = 66}}, {
            events = 17, data = {ptr = 0x43, fd = 67, u32 = 67, u64 = 67}}, {events = 17, data = {ptr = 0x38, fd = 56, u32 = 56, u64 = 56}}, {events = 25, data = {ptr = 0x5a, fd = 90, u32 = 90, u64 = 90}}, {
            events = 17, data = {ptr = 0x49, fd = 73, u32 = 73, u64 = 73}}, {events = 17, data = {ptr = 0x35, fd = 53, u32 = 53, u64 = 53}}, {events = 17, data = {ptr = 0x51, fd = 81, u32 = 81, u64 = 81}}, {
            events = 17, data = {ptr = 0x22, fd = 34, u32 = 34, u64 = 34}}, {events = 25, data = {ptr = 0x1b, fd = 27, u32 = 27, u64 = 27}}, {events = 17, data = {ptr = 0x3d, fd = 61, u32 = 61, u64 = 61}}, {
            events = 17, data = {ptr = 0x23, fd = 35, u32 = 35, u64 = 35}}, {events = 17, data = {ptr = 0x14, fd = 20, u32 = 20, u64 = 20}}, {events = 17, data = {ptr = 0x33, fd = 51, u32 = 51, u64 = 51}}, {
            events = 25, data = {ptr = 0x16, fd = 22, u32 = 22, u64 = 22}}, {events = 17, data = {ptr = 0x37, fd = 55, u32 = 55, 
u64 = 55}}, {events = 25, data = {ptr = 0x1d, fd = 29, u32 = 29, u64 = 29}}, {
            events = 17, data = {ptr = 0x28, fd = 40, u32 = 40, u64 = 40}}, {events = 17, data = {ptr = 0x20, fd = 32, u32 = 32, u64 = 32}}, {events = 17, data = {ptr = 0x45, fd = 69, u32 = 69, u64 = 69}}, {
            events = 17, data = {ptr = 0x6e, fd = 110, u32 = 110, u64 = 110}}, {events = 17, data = {ptr = 0x31, fd = 49, u32 = 49, u64 = 49}}, {events = 25, data = {ptr = 0x1c, fd = 28, u32 = 28, u64 = 28}}, {
            events = 17, data = {ptr = 0x18, fd = 24, u32 = 24, u64 = 24}}, {events = 17, data = {ptr = 0x47, fd = 71, u32 = 71, u64 = 71}}, {events = 25, data = {ptr = 0x24, fd = 36, u32 = 36, u64 = 36}}, {
            events = 17, data = {ptr = 0x58, fd = 88, u32 = 88, u64 = 88}}, {events = 25, data = {ptr = 0x26, fd = 38, u32 = 38, u64 = 38}}, {events = 17, data = {ptr = 0x2a, fd = 42, u32 = 42, u64 = 42}}, {
---Type <return> to continue, or q <return> to quit---
            events = 17, data = {ptr = 0x55, fd = 85, u32 = 85, u64 = 85}}, {events = 25, data = {ptr = 0x5d, fd = 93, u32 = 93, u64 = 93}}, {events = 17, data = {ptr = 0x78, fd = 120, u32 = 120, u64 = 120}}, {
            events = 17, data = {ptr = 0x8d, fd = 141, u32 = 141, u64 = 141}}, {events = 25, data = {ptr = 0x77, fd = 119, u32 = 119, u64 = 119}}, {events = 25, data = {ptr = 0x7a, fd = 122, u32 = 122, 
              u64 = 122}}, {events = 17, data = {ptr = 0x80, fd = 128, u32 = 128, u64 = 128}}, {events = 17, data = {ptr = 0x7b, fd = 123, u32 = 123, u64 = 123}}, {events = 17, data = {ptr = 0x95, fd = 149, 
              u32 = 149, u64 = 149}}, {events = 25, data = {ptr = 0x59, fd = 89, u32 = 89, u64 = 89}}, {events = 17, data = {ptr = 0x62, fd = 98, u32 = 98, u64 = 98}}, {events = 17, data = {ptr = 0x4e, fd = 78, 
              u32 = 78, u64 = 78}}, {events = 25, data = {ptr = 0xcb, fd = 203, u32 = 203, u64 = 203}}, {events = 25, data = {ptr = 0xa8, fd = 168, u32 = 168, u64 = 168}}, {events = 1, data = {ptr = 0x9d, 
              fd = 157, u32 = 157, u64 = 157}}, {events = 1, data = {ptr = 0x359, fd = 857, u32 = 857, u64 = 857}}, {events = 1, data = {ptr = 0x84, fd = 132, u32 = 132, u64 = 132}}, {events = 25, data = {
              ptr = 0x86, fd = 134, u32 = 134, u64 = 134}}, {events = 17, data = {ptr = 0x83, fd = 131, u32 = 131, u64 = 131}}, {events = 17, data = {ptr = 0xa9, fd = 169, u32 = 169, u64 = 169}}, {events = 17, 
            data = {ptr = 0x85, fd = 133, u32 = 133, u64 = 133}}, {events = 17, data = {ptr = 0xa5, fd = 165, u32 = 165, u64 = 165}}, {events = 17, data = {ptr = 0xac, fd = 172, u32 = 172, u64 = 172}}, {
            events = 25, data = {ptr = 0x88, fd = 136, u32 = 136, u64 = 136}}, {events = 1, data = {ptr = 0xc6, fd = 198, u32 = 198, u64 = 198}}, {events = 1, data = {ptr = 0xb7, fd = 183, u32 = 183, u64 = 183}}, {
            events = 17, data = {ptr = 0xb0, fd = 176, u32 = 176, u64 = 176}}, {events = 25, data = {ptr = 0x8c, fd = 140, u32 = 140, u64 = 140}}, {events = 17, data = {ptr = 0x111, fd = 273, u32 = 273, 
              u64 = 273}}, {events = 1, data = {ptr = 0xc2, fd = 194, u32 = 194, u64 = 194}}, {events = 1, data = {ptr = 0xdb, fd = 219, u32 = 219, u64 = 219}}, {events = 1, data = {ptr = 0x118, fd = 280, 
              u32 = 280, u64 = 280}}, {events = 1, data = {ptr = 0x119, fd = 281, u32 = 281, u64 = 281}}, {events = 1, data = {ptr = 0xd2, fd = 210, u32 = 210, u64 = 210}}, {events = 1, data = {ptr = 0xf6, 
              fd = 246, u32 = 246, u64 = 246}}, {events = 1, data = {ptr = 0xfb, fd = 251, u32 = 251, u64 = 251}}, {events = 1, data = {ptr = 0xcd, fd = 205, u32 = 205, u64 = 205}}, {events = 17, data = {
              ptr = 0xef, fd = 239, u32 = 239, u64 = 239}}, {events = 1, data = {ptr = 0xf7, fd = 247, u32 = 247, u64 = 247}}, {events = 1, data = {ptr = 0xf9, fd = 249, u32 = 249, u64 = 249}}, {events = 1, 
            data = {ptr = 0x11f, fd = 287, u32 = 287, u64 = 287}}, {events = 25, data = {ptr = 0x104, fd = 260, u32 = 260, u64 = 260}}, {events = 1, data = {ptr = 0x127, fd = 295, u32 = 295, u64 = 295}}, {
            events = 1, data = {ptr = 0x114, fd = 276, u32 = 276, u64 = 276}}, {events = 25, data = {ptr = 0x130, fd = 304, u32 = 304, u64 = 304}}, {events = 17, data = {ptr = 0x126, fd = 294, u32 = 294, 
              u64 = 294}}, {events = 17, data = {ptr = 0xc5, fd = 197, u32 = 197, u64 = 197}}, {events = 1, data = {ptr = 0x120, fd = 288, u32 = 288, u64 = 288}}, {events = 17, data = {ptr = 0xc7, fd = 199, 
              u32 = 199, u64 = 199}}, {events = 25, data = {ptr = 0x168, fd = 360, u32 = 360, u64 = 360}}, {events = 25, data = {ptr = 0x169, fd = 361, u32 = 361, u64 = 361}}, {events = 17, data = {ptr = 0x150, 
              fd = 336, u32 = 336, u64 = 336}}, {events = 25, data = {ptr = 0x13b, fd = 315, u32 = 315, u64 = 315}}, {events = 25, data = {ptr = 0x157, fd = 343, u32 = 343, u64 = 343}}, {events = 1, data = {
              ptr = 0x155, fd = 341, u32 = 341, u64 = 341}}, {events = 17, data = {ptr = 0x356, fd = 854, u32 = 854, u64 = 854}}, {events = 5, data = {ptr = 0x10, fd = 16, u32 = 16, u64 = 16}}, {events = 1, 
            data = {ptr = 0x12, fd = 18, u32 = 18, u64 = 18}}, {events = 1, data = {ptr = 0x34d, fd = 845, u32 = 845, u64 = 845}}, {events = 1, data = {ptr = 0x293, fd = 659, u32 = 659, u64 = 659}}, {events = 1, 
            data = {ptr = 0x347, fd = 839, u32 = 839, u64 = 839}}, {events = 1, data = {ptr = 0x302, fd = 770, u32 = 770, u64 = 770}}, {events = 25, data = {ptr = 0x21e, fd = 542, u32 = 542, u64 = 542}}, {
---Type <return> to continue, or q <return> to quit---
            events = 1, data = {ptr = 0x367, fd = 871, u32 = 871, u64 = 871}}, {events = 25, data = {ptr = 0x228, fd = 552, u32 = 552, u64 = 552}}, {events = 17, data = {ptr = 0x235, fd = 565, u32 = 565, 
              u64 = 565}}, {events = 25, data = {ptr = 0x249, fd = 585, u32 = 585, u64 = 585}}, {events = 25, data = {ptr = 0x1f9, fd = 505, u32 = 505, u64 = 505}}, {events = 1, data = {ptr = 0xb3, fd = 179, 
              u32 = 179, u64 = 179}}, {events = 25, data = {ptr = 0x21a, fd = 538, u32 = 538, u64 = 538}}, {events = 25, data = {ptr = 0x2dd, fd = 733, u32 = 733, u64 = 733}}, {events = 1, data = {ptr = 0x285, 
              fd = 645, u32 = 645, u64 = 645}}, {events = 1, data = {ptr = 0x286, fd = 646, u32 = 646, u64 = 646}}, {events = 1, data = {ptr = 0x353, fd = 851, u32 = 851, u64 = 851}}, {events = 1, data = {
              ptr = 0x212, fd = 530, u32 = 530, u64 = 530}}, {events = 1, data = {ptr = 0x194, fd = 404, u32 = 404, u64 = 404}}, {events = 1, data = {ptr = 0x1fd, fd = 509, u32 = 509, u64 = 509}}, {events = 1, 
            data = {ptr = 0x14a, fd = 330, u32 = 330, u64 = 330}}, {events = 1, data = {ptr = 0x2d7, fd = 727, u32 = 727, u64 = 727}}, {events = 1, data = {ptr = 0x2bb, fd = 699, u32 = 699, u64 = 699}}, {
            events = 1, data = {ptr = 0x167, fd = 359, u32 = 359, u64 = 359}}, {events = 1, data = {ptr = 0x22b, fd = 555, u32 = 555, u64 = 555}}, {events = 1, data = {ptr = 0x237, fd = 567, u32 = 567, 
              u64 = 567}}, {events = 1, data = {ptr = 0x13f, fd = 319, u32 = 319, u64 = 319}}, {events = 1, data = {ptr = 0x23f, fd = 575, u32 = 575, u64 = 575}}, {events = 1, data = {ptr = 0x37d, fd = 893, 
              u32 = 893, u64 = 893}}, {events = 1, data = {ptr = 0x242, fd = 578, u32 = 578, u64 = 578}}, {events = 1, data = {ptr = 0x1ee, fd = 494, u32 = 494, u64 = 494}}, {events = 1, data = {ptr = 0x1b9, 
              fd = 441, u32 = 441, u64 = 441}}, {events = 1, data = {ptr = 0x184, fd = 388, u32 = 388, u64 = 388}}, {events = 1, data = {ptr = 0x24b, fd = 587, u32 = 587, u64 = 587}}, {events = 1, data = {
              ptr = 0xd1, fd = 209, u32 = 209, u64 = 209}}, {events = 1, data = {ptr = 0x1ed, fd = 493, u32 = 493, u64 = 493}}, {events = 1, data = {ptr = 0x10c, fd = 268, u32 = 268, u64 = 268}}, {events = 1, 
            data = {ptr = 0x30e, fd = 782, u32 = 782, u64 = 782}}, {events = 1, data = {ptr = 0x27b, fd = 635, u32 = 635, u64 = 635}}, {events = 1, data = {ptr = 0x2ea, fd = 746, u32 = 746, u64 = 746}}, {
            events = 1, data = {ptr = 0xe9, fd = 233, u32 = 233, u64 = 233}}, {events = 25, data = {ptr = 0x1bd, fd = 445, u32 = 445, u64 = 445}}, {events = 1, data = {ptr = 0x29e, fd = 670, u32 = 670, 
              u64 = 670}}, {events = 1, data = {ptr = 0x11b, fd = 283, u32 = 283, u64 = 283}}, {events = 1, data = {ptr = 0x1aa, fd = 426, u32 = 426, u64 = 426}}, {events = 1, data = {ptr = 0x183, fd = 387, 
              u32 = 387, u64 = 387}}, {events = 1, data = {ptr = 0x17e, fd = 382, u32 = 382, u64 = 382}}, {events = 25, data = {ptr = 0x17f, fd = 383, u32 = 383, u64 = 383}}, {events = 25, data = {ptr = 0x2cd, 
              fd = 717, u32 = 717, u64 = 717}}, {events = 1, data = {ptr = 0x1f7, fd = 503, u32 = 503, u64 = 503}}, {events = 1, data = {ptr = 0x2f4, fd = 756, u32 = 756, u64 = 756}}, {events = 1, data = {
              ptr = 0x270, fd = 624, u32 = 624, u64 = 624}}, {events = 17, data = {ptr = 0x311, fd = 785, u32 = 785, u64 = 785}}, {events = 1, data = {ptr = 0x28f, fd = 655, u32 = 655, u64 = 655}}, {events = 1, 
            data = {ptr = 0x30d, fd = 781, u32 = 781, u64 = 781}}, {events = 1, data = {ptr = 0x2f5, fd = 757, u32 = 757, u64 = 757}}, {events = 1, data = {ptr = 0x4d, fd = 77, u32 = 77, u64 = 77}}, {events = 17, 
            data = {ptr = 0x30f, fd = 783, u32 = 783, u64 = 783}}, {events = 25, data = {ptr = 0x2ca, fd = 714, u32 = 714, u64 = 714}}, {events = 17, data = {ptr = 0x36a, fd = 874, u32 = 874, u64 = 874}}, {
            events = 1, data = {ptr = 0x312, fd = 786, u32 = 786, u64 = 786}}, {events = 1, data = {ptr = 0x2ed, fd = 749, u32 = 749, u64 = 749}}, {events = 17, data = {ptr = 0x35d, fd = 861, u32 = 861, 
              u64 = 861}}, {events = 25, data = {ptr = 0x2ce, fd = 718, u32 = 718, u64 = 718}}, {events = 25, data = {ptr = 0x307, fd = 775, u32 = 775, u64 = 775}}, {events = 1, data = {ptr = 0x35c, fd = 860, 
              u32 = 860, u64 = 860}}, {events = 25, data = {ptr = 0x363, fd = 867, u32 = 867, u64 = 867}}, {events = 1, data = {ptr = 0x297, fd = 663, u32 = 663, u64 = 663}}, {events = 25, data = {ptr = 0x1b1, 
---Type <return> to continue, or q <return> to quit---
              fd = 433, u32 = 433, u64 = 433}}, {events = 17, data = {ptr = 0x24d, fd = 589, u32 = 589, u64 = 589}}, {events = 1, data = {ptr = 0x2bc, fd = 700, u32 = 700, u64 = 700}}, {events = 17, data = {
              ptr = 0x24c, fd = 588, u32 = 588, u64 = 588}}, {events = 1, data = {ptr = 0x310, fd = 784, u32 = 784, u64 = 784}}, {events = 1, data = {ptr = 0x31f, fd = 799, u32 = 799, u64 = 799}}, {events = 1, 
            data = {ptr = 0x266, fd = 614, u32 = 614, u64 = 614}}, {events = 25, data = {ptr = 0x18a, fd = 394, u32 = 394, u64 = 394}}, {events = 25, data = {ptr = 0x2e6, fd = 742, u32 = 742, u64 = 742}}, {
            events = 17, data = {ptr = 0x2db, fd = 731, u32 = 731, u64 = 731}}, {events = 17, data = {ptr = 0x2fe, fd = 766, u32 = 766, u64 = 766}}, {events = 17, data = {ptr = 0x2d3, fd = 723, u32 = 723, 
              u64 = 723}}...}
        pe = 0x7ffcd7c88620
        e = {events = 1, data = {ptr = 0x13, fd = 19, u32 = 19, u64 = 19}}
        real_timeout = 0
        q = 0x7fdc3c001ee0
        w = 0x7fdc6c6e5c88 <default_loop_struct+456>
        sigset = {__val = {140584674197869, 72057589742960647, 140584674197869, 5, 18446603349780822481, 0, 140584709593697, 32, 140582869532807, 3440, 140584709593780, 3440, 140584709593849, 140584610118770, 
            140582869532686, 140582869532672}}
        psigset = 0x0
        base = 430426874
        have_signals = 0
        nevents = 0
        count = 48
        nfds = 492
        fd = 9
        op = 3
        i = 0
        __PRETTY_FUNCTION__ = "uv__io_poll"
---Type <return> to continue, or q <return> to quit---
#4  0x00007fdc6c4c37d0 in uv_run (loop=0x7fdc6c6e5ac0 <default_loop_struct>, mode=UV_RUN_DEFAULT) at ../src/unix/core.c:359
        timeout = 0
        r = 1
        ran_pending = 1
#5  0x00000000004127b3 in CIMFrame::StartFrame (this=0xbf1760) at /home/thz/imServer/cm/imframe.cpp:87
No locals.
#6  0x00000000004129ec in main (argc=1, argv=0x7ffcd7c8b7f8) at /home/thz/imServer/cm/imframe.cpp:126
        __FUNCTION__ = "main"

@bnoordhuis
Copy link
Member

Okay, thanks. From your description it sounds like libuv runs on thread 1 but you call uv_async_init() from thread 2?

That won't work, only uv_async_send() is thread-safe. You need to call uv_async_init() from thread 1.

@wangshangxiu
Copy link
Author

wangshangxiu commented Sep 10, 2019

Thanks for your patiention .Yes,I use uv_async_init in other thread which uv_loop is not run in. And uv_async_init is not thread-safe,why process doesn't crash?for example,uv_write,uv_close both cause visual errors ,process coredump,if they don't run in uv_run thread.By the way,based on my experiment using the libuv,it seems that it just could work with single io thread mode,all socket stay in one loop. Can I create mutile threads and distribute socket to different loop of different thread?

@bnoordhuis
Copy link
Member

why process doesn't crash?

Because it's undefined behavior. There's no guarantee about what, if anything, will happen.

Can I create mutile threads and distribute socket to different loop of different thread?

Yes, with some effort. See test/benchmark-multi-accept.c for an example, the uv_write2() call in particular.

I'll close this out because it's not a libuv bug. If you have follow-up question, then please post them to the libuv/help repo.

@wangshangxiu
Copy link
Author

wangshangxiu commented Sep 11, 2019 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants