Check query_set != nullptr failed: query set must not be null when register task #7677
Description
Bug Report
Please answer these questions before submitting your issue. Thanks!
1. Minimal reproduce step (Required)
ch workload
sql raise error
2. What did you expect to see? (Required)
no error
3. What did you see instead (Required)
[2023/06/19 04:58:37.992 +08:00] [ERROR] [local_mpp_coordinator.go:372] ["mpp dispatch response meet error"] [error="DB::TiFlashException: MPP query is already aborted, aborted reason: From MPP<query:<query_ts:1687121917969790065, local_query_id:801, server_id:3970601, start_ts:442268888050171946>,task_id:7>: DB::Exception: Check query_set != nullptr failed: query set must not be null when register task"] [timestamp=442268888050171946] [task=15] [task-mpp-version=1] [error-mpp-version=0]
[2023/06/19 04:58:37.992 +08:00] [ERROR] [local_mpp_coordinator.go:372] ["mpp dispatch response meet error"] [error="DB::TiFlashException: MPP query is already aborted, aborted reason: From MPP<query:<query_ts:1687121917969790065, local_query_id:801, server_id:3970601, start_ts:442268888050171946>,task_id:7>: DB::Exception: Check query_set != nullptr failed: query set must not be null when register task"] [timestamp=442268888050171946] [task=13] [task-mpp-version=1] [error-mpp-version=0]
[2023/06/19 04:58:37.992 +08:00] [ERROR] [local_mpp_coordinator.go:372] ["mpp dispatch response meet error"] [error="DB::Exception: Check query_set != nullptr failed: query set must not be null when register task"] [timestamp=442268888050171946] [task=7] [task-mpp-version=1] [error-mpp-version=0]
[2023/06/19 04:58:37.992 +08:00] [ERROR] [local_mpp_coordinator.go:372] ["mpp dispatch response meet error"] [error="DB::TiFlashException: MPP query is already aborted, aborted reason: From MPP<query:<query_ts:1687121917969790065, local_query_id:801, server_id:3970601, start_ts:442268888050171946>,task_id:7>: DB::Exception: Check query_set != nullptr failed: query set must not be null when register task"] [timestamp=442268888050171946] [task=3] [task-mpp-version=1] [error-mpp-version=0]
[2023/06/19 04:58:37.997 +08:00] [INFO] [conn.go:1199] ["command dispatched failed"] [conn=8731443937518026761] [connInfo="id:8731443937518026761, addr:10.200.10.72:58384 status:10, collation:utf8mb4_general_ci, user:root"] [command=Query] [status="inTxn:0, autocommit:1"] [sql="\nselect\t s_name, count(*) as numwait\nfrom\t supplier, order_line l1, orders, stock, nation\nwhere\t ol_o_id = o_id\n and ol_w_id = o_w_id\n and ol_d_id = o_d_id\n and ol_w_id = s_w_id\n and ol_i_id = s_i_id\n and mod((s_w_id * s_i_id),10000) = s_suppkey\n and l1.ol_delivery_d > o_entry_d\n and not exists (select *\n from\torder_line l2\n where l2.ol_o_id = l1.ol_o_id\n and l2.ol_w_id = l1.ol_w_id\n and l2.ol_d_id = l1.ol_d_id\n and l2.ol_delivery_d > l1.ol_delivery_d)\n and s_nationkey = n_nationkey\n and n_name = 'CHINA'\ngroup by s_name\norder by numwait desc, s_name;\n"] [txn_mode=PESSIMISTIC] [timestamp=442268888050171946] [err="DB::TiFlashException: MPP query is already aborted, aborted reason: From MPP<query:<query_ts:1687121917969790065, local_query_id:801, server_id:3970601, start_ts:442268888050171946>,task_id:7>: DB::Exception: Check query_set != nullptr failed: query set must not be null when register task\ngithub.com/pingcap/tidb/executor/internal/mpp.(*localMppCoordinator).handleDispatchReq\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/internal/mpp/local_mpp_coordinator.go:373\ngithub.com/pingcap/tidb/executor/internal/mpp.(*localMppCoordinator).dispatchAll.func2\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/internal/mpp/local_mpp_coordinator.go:300\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1598"]
[root@tidb-1-0 ~]# grep "not be null" -A 10 -B 10 /tiup/deploy/tidb-4000/log/tidb.log
[2023/06/19 04:58:37.987 +08:00] [INFO] [local_mpp_coordinator.go:172] ["Dispatch mpp task"] [timestamp=442268888050171946] [ID=7] [QueryTs=1687121917969790065] [LocalQueryId=801] [ServerID=3970601] [address=tiflash-2-peer:3930] [plan="RightHashJoin{Recv(3, 4, )->Recv(5, 6, )}(tpcc.supplier.s_suppkey,Column#62)(tpcc.stock.s_w_id,tpcc.order_line.ol_w_id)(tpcc.stock.s_i_id,tpcc.order_line.ol_i_id)->Send(13, 14, )"] [mpp-version=1] [exchange-compression-mode=FAST]
[2023/06/19 04:58:37.987 +08:00] [INFO] [local_mpp_coordinator.go:172] ["Dispatch mpp task"] [timestamp=442268888050171946] [ID=8] [QueryTs=1687121917969790065] [LocalQueryId=801] [ServerID=3970601] [address=tiflash-1-peer:3930] [plan="RightHashJoin{Recv(3, 4, )->Recv(5, 6, )}(tpcc.supplier.s_suppkey,Column#62)(tpcc.stock.s_w_id,tpcc.order_line.ol_w_id)(tpcc.stock.s_i_id,tpcc.order_line.ol_i_id)->Send(13, 14, )"] [mpp-version=1] [exchange-compression-mode=FAST]
[2023/06/19 04:58:37.987 +08:00] [INFO] [local_mpp_coordinator.go:172] ["Dispatch mpp task"] [timestamp=442268888050171946] [ID=9] [QueryTs=1687121917969790065] [LocalQueryId=801] [ServerID=3970601] [address=tiflash-2-peer:3930] [plan="Table(orders)->Sel([not(isnull(tpcc.orders.o_entry_d))])->Send(13, 14, )"] [mpp-version=1] [exchange-compression-mode=FAST]
[2023/06/19 04:58:37.987 +08:00] [INFO] [local_mpp_coordinator.go:172] ["Dispatch mpp task"] [timestamp=442268888050171946] [ID=10] [QueryTs=1687121917969790065] [LocalQueryId=801] [ServerID=3970601] [address=tiflash-1-peer:3930] [plan="Table(orders)->Sel([not(isnull(tpcc.orders.o_entry_d))])->Send(13, 14, )"] [mpp-version=1] [exchange-compression-mode=FAST]
[2023/06/19 04:58:37.987 +08:00] [INFO] [local_mpp_coordinator.go:172] ["Dispatch mpp task"] [timestamp=442268888050171946] [ID=11] [QueryTs=1687121917969790065] [LocalQueryId=801] [ServerID=3970601] [address=tiflash-1-peer:3930] [plan="Table(order_line)->Send(13, 14, )"] [mpp-version=1] [exchange-compression-mode=FAST]
[2023/06/19 04:58:37.987 +08:00] [INFO] [local_mpp_coordinator.go:172] ["Dispatch mpp task"] [timestamp=442268888050171946] [ID=12] [QueryTs=1687121917969790065] [LocalQueryId=801] [ServerID=3970601] [address=tiflash-2-peer:3930] [plan="Table(order_line)->Send(13, 14, )"] [mpp-version=1] [exchange-compression-mode=FAST]
[2023/06/19 04:58:37.987 +08:00] [INFO] [local_mpp_coordinator.go:172] ["Dispatch mpp task"] [timestamp=442268888050171946] [ID=13] [QueryTs=1687121917969790065] [LocalQueryId=801] [ServerID=3970601] [address=tiflash-2-peer:3930] [plan="LeftHashJoin{LeftHashJoin{Recv(7, 8, )->Recv(9, 10, )}(tpcc.order_line.ol_o_id,tpcc.orders.o_id)(tpcc.order_line.ol_w_id,tpcc.orders.o_w_id)(tpcc.order_line.ol_d_id,tpcc.orders.o_d_id)->Projection->Recv(11, 12, )}(tpcc.order_line.ol_o_id,tpcc.order_line.ol_o_id)(tpcc.order_line.ol_w_id,tpcc.order_line.ol_w_id)(tpcc.order_line.ol_d_id,tpcc.order_line.ol_d_id)->HashAgg->Send(15, 16, )"] [mpp-version=1] [exchange-compression-mode=FAST]
[2023/06/19 04:58:37.987 +08:00] [INFO] [local_mpp_coordinator.go:172] ["Dispatch mpp task"] [timestamp=442268888050171946] [ID=14] [QueryTs=1687121917969790065] [LocalQueryId=801] [ServerID=3970601] [address=tiflash-1-peer:3930] [plan="LeftHashJoin{LeftHashJoin{Recv(7, 8, )->Recv(9, 10, )}(tpcc.order_line.ol_o_id,tpcc.orders.o_id)(tpcc.order_line.ol_w_id,tpcc.orders.o_w_id)(tpcc.order_line.ol_d_id,tpcc.orders.o_d_id)->Projection->Recv(11, 12, )}(tpcc.order_line.ol_o_id,tpcc.order_line.ol_o_id)(tpcc.order_line.ol_w_id,tpcc.order_line.ol_w_id)(tpcc.order_line.ol_d_id,tpcc.order_line.ol_d_id)->HashAgg->Send(15, 16, )"] [mpp-version=1] [exchange-compression-mode=FAST]
[2023/06/19 04:58:37.987 +08:00] [INFO] [local_mpp_coordinator.go:172] ["Dispatch mpp task"] [timestamp=442268888050171946] [ID=15] [QueryTs=1687121917969790065] [LocalQueryId=801] [ServerID=3970601] [address=tiflash-2-peer:3930] [plan="Recv(13, 14, )->HashAgg->Projection->Projection->Send(-1, )"] [mpp-version=1] [exchange-compression-mode=NONE]
[2023/06/19 04:58:37.987 +08:00] [INFO] [local_mpp_coordinator.go:172] ["Dispatch mpp task"] [timestamp=442268888050171946] [ID=16] [QueryTs=1687121917969790065] [LocalQueryId=801] [ServerID=3970601] [address=tiflash-1-peer:3930] [plan="Recv(13, 14, )->HashAgg->Projection->Projection->Send(-1, )"] [mpp-version=1] [exchange-compression-mode=NONE]
[2023/06/19 04:58:37.992 +08:00] [ERROR] [local_mpp_coordinator.go:372] ["mpp dispatch response meet error"] [error="DB::TiFlashException: MPP query is already aborted, aborted reason: From MPP<query:<query_ts:1687121917969790065, local_query_id:801, server_id:3970601, start_ts:442268888050171946>,task_id:7>: DB::Exception: Check query_set != nullptr failed: query set must not be null when register task"] [timestamp=442268888050171946] [task=15] [task-mpp-version=1] [error-mpp-version=0]
[2023/06/19 04:58:37.992 +08:00] [ERROR] [local_mpp_coordinator.go:372] ["mpp dispatch response meet error"] [error="DB::TiFlashException: MPP query is already aborted, aborted reason: From MPP<query:<query_ts:1687121917969790065, local_query_id:801, server_id:3970601, start_ts:442268888050171946>,task_id:7>: DB::Exception: Check query_set != nullptr failed: query set must not be null when register task"] [timestamp=442268888050171946] [task=13] [task-mpp-version=1] [error-mpp-version=0]
[2023/06/19 04:58:37.992 +08:00] [ERROR] [local_mpp_coordinator.go:372] ["mpp dispatch response meet error"] [error="DB::Exception: Check query_set != nullptr failed: query set must not be null when register task"] [timestamp=442268888050171946] [task=7] [task-mpp-version=1] [error-mpp-version=0]
[2023/06/19 04:58:37.992 +08:00] [WARN] [mpp.go:142] ["mpp dispatch meet io error"] [error="rpc error: code = Canceled desc = context canceled"] [timestamp=442268888050171946] [task=9] [mpp-version=1]
[2023/06/19 04:58:37.992 +08:00] [ERROR] [local_mpp_coordinator.go:362] ["mpp dispatch meet error"] [error="rpc error: code = Canceled desc = context canceled"] [timestamp=442268888050171946] [task=9] [mpp-version=1]
[2023/06/19 04:58:37.992 +08:00] [WARN] [mpp.go:142] ["mpp dispatch meet io error"] [error="rpc error: code = Canceled desc = context canceled"] [timestamp=442268888050171946] [task=11] [mpp-version=1]
[2023/06/19 04:58:37.992 +08:00] [ERROR] [local_mpp_coordinator.go:362] ["mpp dispatch meet error"] [error="rpc error: code = Canceled desc = context canceled"] [timestamp=442268888050171946] [task=11] [mpp-version=1]
[2023/06/19 04:58:37.992 +08:00] [ERROR] [local_mpp_coordinator.go:372] ["mpp dispatch response meet error"] [error="DB::TiFlashException: MPP query is already aborted, aborted reason: From MPP<query:<query_ts:1687121917969790065, local_query_id:801, server_id:3970601, start_ts:442268888050171946>,task_id:7>: DB::Exception: Check query_set != nullptr failed: query set must not be null when register task"] [timestamp=442268888050171946] [task=3] [task-mpp-version=1] [error-mpp-version=0]
[2023/06/19 04:58:37.992 +08:00] [WARN] [mpp.go:142] ["mpp dispatch meet io error"] [error="rpc error: code = Canceled desc = context canceled"] [timestamp=442268888050171946] [task=12] [mpp-version=1]
[2023/06/19 04:58:37.992 +08:00] [ERROR] [local_mpp_coordinator.go:362] ["mpp dispatch meet error"] [error="rpc error: code = Canceled desc = context canceled"] [timestamp=442268888050171946] [task=12] [mpp-version=1]
[2023/06/19 04:58:37.992 +08:00] [WARN] [mpp.go:261] ["establish mpp connection meet error and cannot retry"] [error="rpc error: code = Canceled desc = context canceled"] [timestamp=442268888050171946] [task=16] [mpp-version=1]
[2023/06/19 04:58:37.993 +08:00] [WARN] [mpp.go:142] ["mpp dispatch meet io error"] [error="rpc error: code = Canceled desc = context canceled"] [timestamp=442268888050171946] [task=6] [mpp-version=1]
[2023/06/19 04:58:37.993 +08:00] [ERROR] [local_mpp_coordinator.go:362] ["mpp dispatch meet error"] [error="rpc error: code = Canceled desc = context canceled"] [timestamp=442268888050171946] [task=6] [mpp-version=1]
[2023/06/19 04:58:37.997 +08:00] [INFO] [conn.go:1199] ["command dispatched failed"] [conn=8731443937518026761] [connInfo="id:8731443937518026761, addr:10.200.10.72:58384 status:10, collation:utf8mb4_general_ci, user:root"] [command=Query] [status="inTxn:0, autocommit:1"] [sql="\nselect\t s_name, count(*) as numwait\nfrom\t supplier, order_line l1, orders, stock, nation\nwhere\t ol_o_id = o_id\n and ol_w_id = o_w_id\n and ol_d_id = o_d_id\n and ol_w_id = s_w_id\n and ol_i_id = s_i_id\n and mod((s_w_id * s_i_id),10000) = s_suppkey\n and l1.ol_delivery_d > o_entry_d\n and not exists (select *\n from\torder_line l2\n where l2.ol_o_id = l1.ol_o_id\n and l2.ol_w_id = l1.ol_w_id\n and l2.ol_d_id = l1.ol_d_id\n and l2.ol_delivery_d > l1.ol_delivery_d)\n and s_nationkey = n_nationkey\n and n_name = 'CHINA'\ngroup by s_name\norder by numwait desc, s_name;\n"] [txn_mode=PESSIMISTIC] [timestamp=442268888050171946] [err="DB::TiFlashException: MPP query is already aborted, aborted reason: From MPP<query:<query_ts:1687121917969790065, local_query_id:801, server_id:3970601, start_ts:442268888050171946>,task_id:7>: DB::Exception: Check query_set != nullptr failed: query set must not be null when register task\ngithub.com/pingcap/tidb/executor/internal/mpp.(*localMppCoordinator).handleDispatchReq\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/internal/mpp/local_mpp_coordinator.go:373\ngithub.com/pingcap/tidb/executor/internal/mpp.(*localMppCoordinator).dispatchAll.func2\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/internal/mpp/local_mpp_coordinator.go:300\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1598"]
4. What is your TiFlash version? (Required)
master