Parallel self-join stalls forever #108
Comments
|
Thank you for your feedback. We also found that there are some problems with the current parallel mechanism, such as hang or coredump when SQL is executed. Because there are many modules involved, we will do a detailed analysis on this in the follow-up, and it is expected that improvements will be made in the next version. |
|
Thanks for the update! |
|
Hi, i try to solve the problem, but i can‘t reproduce the problem. Can you give the steps to reproduce this problem？ |
|
Thanks for checking! |
|
i.e. when just rerun it now, this is the bt from the coordinator when cancelled from within gdb: |
|
@yazun Thanks for your reply! I try to construct use cases, and the execution plan is consistent with the execution plan you pasted, and i also use hash join, but it is not reproduced, so if it's convenient, please tell me the reproduction steps in detail so that we can solve this problem as soon as possible. Look forward to your reply. |
|
Thanks for the hints @beth-database! |
|
@beth-database , this is on the master from our fork, which has been just merged with TBase 2.2/master : attaching to DN: Following the lead: attaching to dn6 process: hope this helps.. |
|
Hello @yazun |
|
Definition of the table is quite simple happily: \d+ dr3_ops_cs36_mv.dr3_common_export
Table "dr3_ops_cs36_mv.dr3_common_export"
Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+-----------------------------+-----------+----------+---------+----------+--------------+-------------
sourceid | bigint | | not null | | plain | |
varitype | text | | not null | | extended | |
created | timestamp without time zone | | | now() | plain | |
Indexes:
"dr3_common_export_pkey" PRIMARY KEY, btree (varitype, sourceid), tablespace "output_tablespace"
"idx_id_dr3_common_export" btree (sourceid), tablespace "final_run_validation_tablespace"
Tablespace: "final_run_validation_tablespace"
Distribute By: SHARD(sourceid)
Location Nodes: ALL DATANODES
Options: autovacuum_analyze_threshold=1000Am traveling today, will try to gstack in look in 3-4hrs. |
|
Hello @yazun |
|
Hello @beth-database, then gstacks |
|
(note that this time it launched the parallel worker on dn5, not dn6) |
|
Hi, @yazun. I'm sorry to interrupt your trip and thanks for your reply. According the stack, there are some doubts: |
|
Hello @beth-database, Checking the 2. |
Attaching to process 9614
[New LWP 9755]
[New LWP 9756]
[New LWP 9757]
[New LWP 9758]
[New LWP 9759]
[New LWP 9760]
[New LWP 9761]
[New LWP 9762]
[New LWP 9763]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
0x00007fadc832a9a3 in select () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install cyrus-sasl-lib-2.1.26-23.el7.x86_64 glibc-2.17-307.el7.1.x86_64 keyutils-libs-1.5.8-3.el7.x86_64 krb5-libs-1.15.1-46.el7.x86_64 libcom_err-1.42.9-17.el7.x86_64 libselinux-2.5-15.el7.x86_64 libxml2-2.9.1-6.el7.4.x86_64 nspr-4.21.0-1.el7.x86_64 nss-3.44.0-7.el7_7.x86_64 nss-softokn-freebl-3.44.0-8.el7_7.x86_64 nss-util-3.44.0-4.el7_7.x86_64 openldap-2.4.44-21.el7_6.x86_64 openssl-libs-1.0.2k-19.el7.x86_64 pcre-8.32-17.el7.x86_64 xz-libs-5.2.2-1.el7.x86_64 zlib-1.2.7-18.el7.x86_64
(gdb) b squeue.c:6978
Breakpoint 1 at 0x7ddfd6: file pgxc/squeue/squeue.c, line 6978.
(gdb) c
Continuing.
[Switching to Thread 0x7fadb9ed1700 (LWP 9759)]
Thread 6 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0db8, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
(gdb) p node
$1 = (ParallelSendNodeControl *) 0x22e0db8
(gdb) p *node
$2 = {nodeId = 8, sock = 16, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 0, buffer = 0x22e11b8, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4646670}
(gdb) p *buffer
$3 = {nodeId = 8, parallelWorkerNum = 0, tuples_put = 2748, tuples_get = 1, ntuples = 996, fast_send = 995, normal_send = 1, send_times = 0, no_data = 0, send_data_len = 0, write_data_len = 65526, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 0 '\000', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 65526, bufTail = 0, bufBorder = 65526, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0,
__spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbbf567c4 "D"}
..
(gdb) disp *buffer
1: *buffer = {nodeId = 4, parallelWorkerNum = 1, tuples_put = 6183, tuples_get = 2, ntuples = 1010, fast_send = 1009, normal_send = 1, send_times = 0, no_data = 0, send_data_len = 0, write_data_len = 65503, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 0 '\000', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 65503, bufTail = 0, bufBorder = 65503, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0,
__spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbbfd6e24 "D"}
(gdb) disp *node
2: *node = {nodeId = 4, sock = 20, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 1, buffer = 0x22e1078, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4647815}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbb6d4700 (LWP 9756)]
Thread 3 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0c08, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 2, parallelWorkerNum = 1, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4648769, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbbfb6c8c ""}
2: *node = {nodeId = 2, sock = 23, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 1, buffer = 0x22e0fd8, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4650440}
(gdb) c
Continuing.
[Switching to Thread 0x7fadb9ed1700 (LWP 9759)]
Thread 6 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0db8, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 8, parallelWorkerNum = 2, tuples_put = 2585, tuples_get = 1, ntuples = 998, fast_send = 997, normal_send = 1, send_times = 0, no_data = 0, send_data_len = 0, write_data_len = 65489, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 0 '\000', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 65489, bufTail = 0, bufBorder = 65489, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0,
__spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc0d7ae4 "D"}
2: *node = {nodeId = 8, sock = 16, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 2, buffer = 0x22e11b8, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4646670}
(gdb) c
Continuing.
[Switching to Thread 0x7fadba6d2700 (LWP 9758)]
Thread 5 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0d28, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 6, parallelWorkerNum = 1, tuples_put = 0, tuples_get = 0, ntuples = 203, fast_send = 203, normal_send = 0, send_times = 1, no_data = 4649096, send_data_len = 12784, write_data_len = 12784, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 12784, bufTail = 12784, bufBorder = 12784, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0,
__kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbbff6fbc "D"}
2: *node = {nodeId = 6, sock = 17, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 1, buffer = 0x22e1118, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4650769}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbbed5700 (LWP 9755)]
Thread 2 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0b78, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 0, parallelWorkerNum = 2, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4650216, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc057484 ""}
2: *node = {nodeId = 0, sock = 21, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 2, buffer = 0x20e6a28, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4651884}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbb6d4700 (LWP 9756)]
Thread 3 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0c08, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 2, parallelWorkerNum = 2, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4648770, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc07761c ""}
2: *node = {nodeId = 2, sock = 23, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 2, buffer = 0x22e0fd8, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4650440}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbaed3700 (LWP 9757)]
Thread 4 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0c98, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 4, parallelWorkerNum = 2, tuples_put = 6517, tuples_get = 2, ntuples = 1013, fast_send = 1012, normal_send = 1, send_times = 0, no_data = 0, send_data_len = 0, write_data_len = 65515, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 0 '\000', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 65515, bufTail = 0, bufBorder = 65515, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0,
__spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc0977b4 "D"}
2: *node = {nodeId = 4, sock = 20, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 2, buffer = 0x22e1078, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4647815}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbbed5700 (LWP 9755)]
Thread 2 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0b78, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 0, parallelWorkerNum = 3, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4650212, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc117e14 ""}
2: *node = {nodeId = 0, sock = 21, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 3, buffer = 0x20e6a28, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4651884}
(gdb) c
Continuing.
Thread 3 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0c08, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 2, parallelWorkerNum = 3, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4648766, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc137fac ""}
2: *node = {nodeId = 2, sock = 23, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 3, buffer = 0x22e0fd8, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4650440}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbaed3700 (LWP 9757)]
Thread 4 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0c98, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 4, parallelWorkerNum = 3, tuples_put = 3952, tuples_get = 2, ntuples = 1006, fast_send = 1005, normal_send = 1, send_times = 0, no_data = 0, send_data_len = 0, write_data_len = 65528, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 0 '\000', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 65528, bufTail = 0, bufBorder = 65528, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0,
__spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc158144 "D"}
2: *node = {nodeId = 4, sock = 20, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 3, buffer = 0x22e1078, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4647815}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbb6d4700 (LWP 9756)]
Thread 3 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0c08, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 2, parallelWorkerNum = 4, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4648768, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc1f893c ""}
2: *node = {nodeId = 2, sock = 23, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 4, buffer = 0x22e0fd8, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4650440}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbbed5700 (LWP 9755)]
Thread 2 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0b78, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 0, parallelWorkerNum = 4, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4650214, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc1d87a4 ""}
2: *node = {nodeId = 0, sock = 21, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 4, buffer = 0x20e6a28, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4651884}
(gdb) c
Continuing.
[Switching to Thread 0x7fadba6d2700 (LWP 9758)]
Thread 5 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0d28, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 6, parallelWorkerNum = 2, tuples_put = 0, tuples_get = 0, ntuples = 194, fast_send = 194, normal_send = 0, send_times = 1, no_data = 4649098, send_data_len = 12063, write_data_len = 12063, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 12063, bufTail = 12063, bufBorder = 12063, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0,
__kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc0b794c "D"}
2: *node = {nodeId = 6, sock = 17, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 2, buffer = 0x22e1118, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4650769}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbaed3700 (LWP 9757)]
Thread 4 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0c98, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 4, parallelWorkerNum = 4, tuples_put = 6447, tuples_get = 2, ntuples = 1014, fast_send = 1013, normal_send = 1, send_times = 0, no_data = 0, send_data_len = 0, write_data_len = 65532, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 0 '\000', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 65532, bufTail = 0, bufBorder = 65532, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0,
__spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc218ad4 "D"}
2: *node = {nodeId = 4, sock = 20, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 4, buffer = 0x22e1078, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4647815}
(gdb)
Continuing.
[Switching to Thread 0x7fadbb6d4700 (LWP 9756)]
Thread 3 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0c08, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 2, parallelWorkerNum = 5, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4648769, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc2b92cc ""}
2: *node = {nodeId = 2, sock = 23, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 5, buffer = 0x22e0fd8, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4650440}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbbed5700 (LWP 9755)]
Thread 2 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0b78, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 0, parallelWorkerNum = 5, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4650214, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc299134 ""}
2: *node = {nodeId = 0, sock = 21, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 5, buffer = 0x20e6a28, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4651884}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbaed3700 (LWP 9757)]
Thread 4 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0c98, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 4, parallelWorkerNum = 5, tuples_put = 5884, tuples_get = 1, ntuples = 1010, fast_send = 1009, normal_send = 1, send_times = 0, no_data = 0, send_data_len = 0, write_data_len = 65519, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 0 '\000', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 65519, bufTail = 0, bufBorder = 65519, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0,
__spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc2d9464 "D"}
2: *node = {nodeId = 4, sock = 20, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 5, buffer = 0x22e1078, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4647815}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbb6d4700 (LWP 9756)]
Thread 3 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0c50, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 3, parallelWorkerNum = 0, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4648767, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbbf063c8 ""}
2: *node = {nodeId = 3, sock = 22, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 0, buffer = 0x22e1028, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4650440}
(gdb) c
Continuing.
[Switching to Thread 0x7fadba6d2700 (LWP 9758)]
Thread 5 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0d28, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 6, parallelWorkerNum = 3, tuples_put = 0, tuples_get = 0, ntuples = 99, fast_send = 99, normal_send = 0, send_times = 1, no_data = 4649094, send_data_len = 6002, write_data_len = 6002, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 6002, bufTail = 6002, bufBorder = 6002, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0,
__kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc1782dc "D"}
2: *node = {nodeId = 6, sock = 17, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 3, buffer = 0x22e1118, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4650769}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbbed5700 (LWP 9755)]
Thread 2 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0bc0, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 1, parallelWorkerNum = 0, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4650213, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbbee6230 ""}
2: *node = {nodeId = 1, sock = 15, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 0, buffer = 0x22e0f88, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4651885}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbb6d4700 (LWP 9756)]
Thread 3 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0c50, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 3, parallelWorkerNum = 1, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4648769, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbbfc6d58 ""}
2: *node = {nodeId = 3, sock = 22, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 1, buffer = 0x22e1028, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4650440}
(gdb)
Continuing.
[Switching to Thread 0x7fadbaed3700 (LWP 9757)]
Thread 4 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0ce0, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 5, parallelWorkerNum = 0, tuples_put = 0, tuples_get = 0, ntuples = 237, fast_send = 237, normal_send = 0, send_times = 1, no_data = 4646142, send_data_len = 15882, write_data_len = 15882, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 15882, bufTail = 15882, bufBorder = 15882, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0,
__kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbbf26560 "D"}
2: *node = {nodeId = 5, sock = 19, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 0, buffer = 0x22e10c8, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4647815}
(gdb) c
Continuing.
Thread 2 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0bc0, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 1, parallelWorkerNum = 1, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4650214, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbbfa6bc0 ""}
2: *node = {nodeId = 1, sock = 15, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 1, buffer = 0x22e0f88, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4651885}
(gdb) c
Continuing.
[Switching to Thread 0x7fadba6d2700 (LWP 9758)]
Thread 5 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0d28, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 6, parallelWorkerNum = 4, tuples_put = 0, tuples_get = 0, ntuples = 191, fast_send = 191, normal_send = 0, send_times = 1, no_data = 4649096, send_data_len = 11945, write_data_len = 11945, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 11945, bufTail = 11945, bufBorder = 11945, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0,
__kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc238c6c "D"}
2: *node = {nodeId = 6, sock = 17, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 4, buffer = 0x22e1118, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4650769}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbb6d4700 (LWP 9756)]
Thread 3 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0c50, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 3, parallelWorkerNum = 2, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4648770, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc0876e8 ""}
2: *node = {nodeId = 3, sock = 22, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 2, buffer = 0x22e1028, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4650440}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbaed3700 (LWP 9757)]
Thread 4 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0ce0, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 5, parallelWorkerNum = 1, tuples_put = 0, tuples_get = 0, ntuples = 259, fast_send = 259, normal_send = 0, send_times = 1, no_data = 4646143, send_data_len = 17438, write_data_len = 17438, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 17438, bufTail = 17438, bufBorder = 17438, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0,
__kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbbfe6ef0 "D"}
2: *node = {nodeId = 5, sock = 19, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 1, buffer = 0x22e10c8, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4647815}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbbed5700 (LWP 9755)]
Thread 2 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0bc0, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 1, parallelWorkerNum = 2, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4650216, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc067550 ""}
2: *node = {nodeId = 1, sock = 15, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 2, buffer = 0x22e0f88, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4651885}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbb6d4700 (LWP 9756)]
Thread 3 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0c50, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 3, parallelWorkerNum = 3, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4648766, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc148078 ""}
2: *node = {nodeId = 3, sock = 22, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 3, buffer = 0x22e1028, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4650440}
(gdb) c
Continuing.
[Switching to Thread 0x7fadba6d2700 (LWP 9758)]
Thread 5 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0d28, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 6, parallelWorkerNum = 5, tuples_put = 0, tuples_get = 0, ntuples = 211, fast_send = 211, normal_send = 0, send_times = 1, no_data = 4649096, send_data_len = 13233, write_data_len = 13233, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 13233, bufTail = 13233, bufBorder = 13233, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0,
__kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc2f95fc "D"}
2: *node = {nodeId = 6, sock = 17, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 5, buffer = 0x22e1118, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4650769}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbbed5700 (LWP 9755)]
Thread 2 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0bc0, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 1, parallelWorkerNum = 3, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4650212, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc127ee0 ""}
2: *node = {nodeId = 1, sock = 15, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 3, buffer = 0x22e0f88, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4651885}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbaed3700 (LWP 9757)]
Thread 4 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0ce0, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 5, parallelWorkerNum = 2, tuples_put = 0, tuples_get = 0, ntuples = 276, fast_send = 276, normal_send = 0, send_times = 1, no_data = 4646144, send_data_len = 18590, write_data_len = 18590, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 18590, bufTail = 18590, bufBorder = 18590, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0,
__kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc0a7880 "D"}
2: *node = {nodeId = 5, sock = 19, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 2, buffer = 0x22e10c8, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4647815}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbb6d4700 (LWP 9756)]
Thread 3 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0c50, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 3, parallelWorkerNum = 4, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4648768, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc208a08 ""}
2: *node = {nodeId = 3, sock = 22, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 4, buffer = 0x22e1028, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4650440}
(gdb) c
Continuing.
[Switching to Thread 0x7fadba6d2700 (LWP 9758)]
Thread 5 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0d70, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 7, parallelWorkerNum = 0, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4649096, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbbf466f8 ""}
2: *node = {nodeId = 7, sock = 14, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 0, buffer = 0x22e1168, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4650769}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbbed5700 (LWP 9755)]
Thread 2 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0bc0, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 1, parallelWorkerNum = 4, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4650214, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc1e8870 ""}
2: *node = {nodeId = 1, sock = 15, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 4, buffer = 0x22e0f88, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4651885}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbb6d4700 (LWP 9756)]
Thread 3 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0c50, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 3, parallelWorkerNum = 5, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4648769, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc2c9398 ""}
2: *node = {nodeId = 3, sock = 22, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 5, buffer = 0x22e1028, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4650440}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbaed3700 (LWP 9757)]
Thread 4 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0ce0, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 5, parallelWorkerNum = 3, tuples_put = 0, tuples_get = 0, ntuples = 274, fast_send = 274, normal_send = 0, send_times = 1, no_data = 4646141, send_data_len = 18430, write_data_len = 18430, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 18430, bufTail = 18430, bufBorder = 18430, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0,
__kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc168210 "D"}
2: *node = {nodeId = 5, sock = 19, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 3, buffer = 0x22e10c8, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4647815}
(gdb) c
Continuing.
Thread 2 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0bc0, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 1, parallelWorkerNum = 5, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4650214, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc2a9200 ""}
2: *node = {nodeId = 1, sock = 15, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 5, buffer = 0x22e0f88, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4651885}
(gdb) c
Continuing.
[Switching to Thread 0x7fadba6d2700 (LWP 9758)]
Thread 5 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0d70, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 7, parallelWorkerNum = 1, tuples_put = 0, tuples_get = 0, ntuples = 2, fast_send = 2, normal_send = 0, send_times = 1, no_data = 4649096, send_data_len = 139, write_data_len = 139, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 139, bufTail = 139, bufBorder = 139, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0,
__spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc007088 "D"}
2: *node = {nodeId = 7, sock = 14, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 1, buffer = 0x22e1168, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4650769}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbaed3700 (LWP 9757)]
Thread 4 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0ce0, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 5, parallelWorkerNum = 4, tuples_put = 0, tuples_get = 0, ntuples = 312, fast_send = 312, normal_send = 0, send_times = 1, no_data = 4646143, send_data_len = 20972, write_data_len = 20972, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 20972, bufTail = 20972, bufBorder = 20972, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0,
__kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc228ba0 "D"}
2: *node = {nodeId = 5, sock = 19, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 4, buffer = 0x22e10c8, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4647815}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbb6d4700 (LWP 9756)]
Thread 3 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0c08, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 2, parallelWorkerNum = 0, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4648768, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbbef62fc ""}
2: *node = {nodeId = 2, sock = 23, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 0, buffer = 0x22e0fd8, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4650441}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbbed5700 (LWP 9755)]
Thread 2 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0b78, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 0, parallelWorkerNum = 0, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4650214, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbbed6164 ""}
2: *node = {nodeId = 0, sock = 21, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 0, buffer = 0x20e6a28, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4651885}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbaed3700 (LWP 9757)]
Thread 4 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0ce0, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 5, parallelWorkerNum = 5, tuples_put = 0, tuples_get = 0, ntuples = 253, fast_send = 253, normal_send = 0, send_times = 1, no_data = 4646143, send_data_len = 16985, write_data_len = 16985, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 16985, bufTail = 16985, bufBorder = 16985, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0,
__kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc2e9530 "D"}
2: *node = {nodeId = 5, sock = 19, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 5, buffer = 0x22e10c8, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4647815}
(gdb) c
Continuing.
[Switching to Thread 0x7fadba6d2700 (LWP 9758)]
Thread 5 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0d70, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 7, parallelWorkerNum = 2, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4649099, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc0c7a18 ""}
2: *node = {nodeId = 7, sock = 14, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 2, buffer = 0x22e1168, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4650769}
(gdb) c
Continuing.
[Switching to Thread 0x7fadb9ed1700 (LWP 9759)]
Thread 6 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0db8, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 8, parallelWorkerNum = 3, tuples_put = 2564, tuples_get = 1, ntuples = 993, fast_send = 992, normal_send = 1, send_times = 0, no_data = 0, send_data_len = 0, write_data_len = 65488, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 0 '\000', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 65488, bufTail = 0, bufBorder = 65488, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0,
__spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc198474 "D"}
2: *node = {nodeId = 8, sock = 16, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 3, buffer = 0x22e11b8, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4646670}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbb6d4700 (LWP 9756)]
Thread 3 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0c08, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 2, parallelWorkerNum = 1, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4648770, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbbfb6c8c ""}
2: *node = {nodeId = 2, sock = 23, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 1, buffer = 0x22e0fd8, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4650441}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbbed5700 (LWP 9755)]
Thread 2 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0b78, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 0, parallelWorkerNum = 1, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4650215, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbbf96af4 ""}
2: *node = {nodeId = 0, sock = 21, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 1, buffer = 0x20e6a28, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4651885}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbaed3700 (LWP 9757)]
Thread 4 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0c98, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 4, parallelWorkerNum = 0, tuples_put = 3845, tuples_get = 2, ntuples = 1004, fast_send = 1003, normal_send = 1, send_times = 0, no_data = 0, send_data_len = 0, write_data_len = 65525, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 0 '\000', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 65525, bufTail = 0, bufBorder = 65525, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0,
__spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbbf16494 "D"}
2: *node = {nodeId = 4, sock = 20, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 0, buffer = 0x22e1078, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4647816}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbb6d4700 (LWP 9756)]
Thread 3 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0c08, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 2, parallelWorkerNum = 2, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4648771, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc07761c ""}
2: *node = {nodeId = 2, sock = 23, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 2, buffer = 0x22e0fd8, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4650441}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbbed5700 (LWP 9755)]
Thread 2 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0b78, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 0, parallelWorkerNum = 2, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4650217, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc057484 ""}
2: *node = {nodeId = 0, sock = 21, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 2, buffer = 0x20e6a28, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4651885}
(gdb) c
Continuing.
[Switching to Thread 0x7fadba6d2700 (LWP 9758)]
Thread 5 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0d70, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 7, parallelWorkerNum = 3, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4649095, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc1883a8 ""}
2: *node = {nodeId = 7, sock = 14, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 3, buffer = 0x22e1168, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4650769}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbb6d4700 (LWP 9756)]
Thread 3 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0c08, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 2, parallelWorkerNum = 3, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4648767, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc137fac ""}
2: *node = {nodeId = 2, sock = 23, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 3, buffer = 0x22e0fd8, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4650441}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbaed3700 (LWP 9757)]
Thread 4 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0c98, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 4, parallelWorkerNum = 1, tuples_put = 6183, tuples_get = 1, ntuples = 1010, fast_send = 1009, normal_send = 1, send_times = 0, no_data = 0, send_data_len = 0, write_data_len = 65503, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 0 '\000', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 65503, bufTail = 0, bufBorder = 65503, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0,
__spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbbfd6e24 "D"}
2: *node = {nodeId = 4, sock = 20, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 1, buffer = 0x22e1078, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4647816}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbbed5700 (LWP 9755)]
Thread 2 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0b78, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 0, parallelWorkerNum = 3, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4650213, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc117e14 ""}
2: *node = {nodeId = 0, sock = 21, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 3, buffer = 0x20e6a28, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4651885}
(gdb) c
Continuing.
[Switching to Thread 0x7fadba6d2700 (LWP 9758)]
Thread 5 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0d70, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 7, parallelWorkerNum = 4, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4649097, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc248d38 ""}
2: *node = {nodeId = 7, sock = 14, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 4, buffer = 0x22e1168, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4650769}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbb6d4700 (LWP 9756)]
Thread 3 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0c08, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 2, parallelWorkerNum = 4, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4648769, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc1f893c ""}
2: *node = {nodeId = 2, sock = 23, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 4, buffer = 0x22e0fd8, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4650441}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbaed3700 (LWP 9757)]
Thread 4 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0c98, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 4, parallelWorkerNum = 2, tuples_put = 6517, tuples_get = 2, ntuples = 1013, fast_send = 1012, normal_send = 1, send_times = 0, no_data = 0, send_data_len = 0, write_data_len = 65515, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 0 '\000', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 65515, bufTail = 0, bufBorder = 65515, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0,
__spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc0977b4 "D"}
2: *node = {nodeId = 4, sock = 20, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 2, buffer = 0x22e1078, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4647816}
(gdb) c
Continuing.
[Switching to Thread 0x7fadbbed5700 (LWP 9755)]
Thread 2 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0b78, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 0, parallelWorkerNum = 4, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4650215, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc1d87a4 ""}
2: *node = {nodeId = 0, sock = 21, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 4, buffer = 0x20e6a28, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4651885}
(gdb) c
Continuing.
[Switching to Thread 0x7fadba6d2700 (LWP 9758)]
Thread 5 "postgres" hit Breakpoint 1, SendNodeData (node=node@entry=0x22e0d70, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6978 if (buffer->status == DataPumpSndStatus_set_socket)
1: *buffer = {nodeId = 7, parallelWorkerNum = 5, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4649097, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0,
__elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc3096c8 ""}
2: *node = {nodeId = 7, sock = 14, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 5, buffer = 0x22e1168, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4650769}
(gdb) |
(gdb) info threads
Id Target Id Frame
1 Thread 0x7fadc9fcc840 (LWP 9614) "postgres" 0x00007fadc832a9a3 in select () from /lib64/libc.so.6
2 Thread 0x7fadbbed5700 (LWP 9755) "postgres" SendNodeData (node=node@entry=0x22e0b78, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
* 3 Thread 0x7fadbb6d4700 (LWP 9756) "postgres" SendNodeData (node=node@entry=0x22e0c08, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
4 Thread 0x7fadbaed3700 (LWP 9757) "postgres" 0x00000000007ddfdb in SendNodeData (node=node@entry=0x22e0c98, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
5 Thread 0x7fadba6d2700 (LWP 9758) "postgres" SendNodeData (node=node@entry=0x22e0d70, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
6 Thread 0x7fadb9ed1700 (LWP 9759) "postgres" SendNodeData (node=node@entry=0x22e0db8, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
7 Thread 0x7fadb96d0700 (LWP 9760) "postgres" SendNodeData (node=node@entry=0x22e0e48, last_send=last_send@entry=0 '\000') at pgxc/squeue/squeue.c:6978
8 Thread 0x7fadb8ecf700 (LWP 9761) "postgres" 0x00007fadc832a9a3 in select () from /lib64/libc.so.6
9 Thread 0x7fadb86ce700 (LWP 9762) "postgres" 0x00007fadc832a9a3 in select () from /lib64/libc.so.6
10 Thread 0x7fadb7ecd700 (LWP 9763) "postgres" 0x00007fadc9bc99dd in accept () from /lib64/libpthread.so.0 |
|
or printing them all at once: thread apply 2 3 4 5 6 7 p *buffer
Thread 2 (Thread 0x7fadbbed5700 (LWP 9755)):
$10 = {nodeId = 0, parallelWorkerNum = 4, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4650215, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc1d87a4 ""}
Thread 3 (Thread 0x7fadbb6d4700 (LWP 9756)):
$11 = {nodeId = 2, parallelWorkerNum = 5, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4648770, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc2b92cc ""}
Thread 4 (Thread 0x7fadbaed3700 (LWP 9757)):
$12 = {nodeId = 4, parallelWorkerNum = 3, tuples_put = 3952, tuples_get = 1, ntuples = 1006, fast_send = 1005, normal_send = 1, send_times = 0, no_data = 0, send_data_len = 0, write_data_len = 65528, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 0 '\000', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 65528, bufTail = 0, bufBorder = 65528, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc158144 "D"}
Thread 5 (Thread 0x7fadba6d2700 (LWP 9758)):
$13 = {nodeId = 7, parallelWorkerNum = 5, tuples_put = 0, tuples_get = 0, ntuples = 0, fast_send = 0, normal_send = 0, send_times = 0, no_data = 4649097, send_data_len = 0, write_data_len = 0, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 0, bufTail = 0, bufBorder = 0, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc3096c8 ""}
Thread 6 (Thread 0x7fadb9ed1700 (LWP 9759)):
$14 = {nodeId = 8, parallelWorkerNum = 3, tuples_put = 2564, tuples_get = 2, ntuples = 993, fast_send = 992, normal_send = 1, send_times = 0, no_data = 0, send_data_len = 0, write_data_len = 65488, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 0 '\000', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 65488, bufTail = 0, bufBorder = 65488, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc198474 "D"}
Thread 7 (Thread 0x7fadb96d0700 (LWP 9760)):
$15 = {nodeId = 10, parallelWorkerNum = 1, tuples_put = 0, tuples_get = 0, ntuples = 42, fast_send = 42, normal_send = 0, send_times = 1, no_data = 4648909, send_data_len = 2521, write_data_len = 2521, long_tuple = 0 '\000', status = DataPumpSndStatus_set_socket, stuck = 0 '\000', last_send = 1 '\001', bufLock = 0 '\000', bufFull = 0 '\000', bufHead = 2521, bufTail = 2521, bufBorder = 2521, sendSem = {m_cnt = 0, m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, m_cond = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}}, bufLength = 65536, buffer = 0x7fadbc0372ec "D"}
thread apply 2 3 4 5 6 7 p *node
Thread 2 (Thread 0x7fadbbed5700 (LWP 9755)):
$16 = {nodeId = 0, sock = 21, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 4, buffer = 0x20e6a28, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4651885}
Thread 3 (Thread 0x7fadbb6d4700 (LWP 9756)):
$17 = {nodeId = 2, sock = 23, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 5, buffer = 0x22e0fd8, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4650441}
Thread 4 (Thread 0x7fadbaed3700 (LWP 9757)):
$18 = {nodeId = 4, sock = 20, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 3, buffer = 0x22e1078, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4647816}
Thread 5 (Thread 0x7fadba6d2700 (LWP 9758)):
$19 = {nodeId = 7, sock = 14, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 5, buffer = 0x22e1168, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4650769}
Thread 6 (Thread 0x7fadb9ed1700 (LWP 9759)):
$20 = {nodeId = 8, sock = 16, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 3, buffer = 0x22e11b8, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4646670}
Thread 7 (Thread 0x7fadb96d0700 (LWP 9760)):
$21 = {nodeId = 10, sock = 13, numParallelWorkers = 6, status = DataPumpSndStatus_set_socket, lock = 0 '\000', errorno = 0, current_buffer = 1, buffer = 0x22e1258, last_offset = 0, remaining_length = 0, ntuples = 0, sleep_count = 0, send_timies = 4650585} |
|
Hi, @yazun thanks for feedback. |
|
excellent! |
|
Just to be sure: |
|
also, for a 12 node cluster, should we set |
|
also, is |
|
setting |
|
It's not really true for 'sender_thread_buffer_size = sender_thread_batch_size * sender_thread_num' to getting best result. For each sender thread, it has some buffers for parallel workers, and for each buffer, as long as the amount of data reaches the batch size threshold, the sender thread will send data. Sender threads do not affect each other. |
|
"also, for a 12 node cluster, should we set sender_thread_num to 12 for an optimal parallel squeue operations?" |
|
"also, is pgxl_remote_fetch_size related in any similar way to the data pump thread buffer?" |
|
Thanks a lot of clarifications @beth-database. |
JennyJennyChen
pushed a commit
that referenced
this issue
Jan 19, 2022
JennyJennyChen
added a commit
that referenced
this issue
Jan 19, 2022
#108 Co-authored-by: bethding <bethding@tencent.com>
JennyJennyChen
added a commit
that referenced
this issue
Jan 19, 2022
JennyJennyChen
pushed a commit
that referenced
this issue
Jan 19, 2022
JennyJennyChen
pushed a commit
that referenced
this issue
Jan 19, 2022
JennyJennyChen
pushed a commit
that referenced
this issue
Jan 19, 2022
|
@yazun you can download the latest code to try . thanks for your replies, and thanks @beth-database |
|
Will test and will get back to you this week! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
We noticed no errors this time, but the query that takes a second while run with parallelism disabled stalls forever when parallel workers are used.
This plan stalls for hours:
Disabling parallelism:
The text was updated successfully, but these errors were encountered: