[NPE] Insert failed in cluster mode.

[cserwen]

Describe the bug

• Use https://github.com/thulab/iotdb-benchmark 0.13 to test 0.14 cluster mode. But write failed because of following logs.

2022-09-26 14:37:26,937 ERROR cn.edu.tsinghua.iotdb.benchmark.iotdb013.IoTDBClusterSession:173 - insert tablet failed 
org.apache.iotdb.rpc.StatementExecutionException: 411: no detailed failure reason in QueryStateMachine
        at org.apache.iotdb.rpc.RpcUtils.verifySuccess(RpcUtils.java:94)
        at org.apache.iotdb.rpc.RpcUtils.verifySuccessWithRedirection(RpcUtils.java:115)
        at org.apache.iotdb.session.SessionConnection.insertTablet(SessionConnection.java:589)
        at org.apache.iotdb.session.Session.insertTablet(Session.java:1573)
        at org.apache.iotdb.session.pool.SessionPool.insertTablet(SessionPool.java:446)
        at org.apache.iotdb.session.pool.SessionPool.insertTablet(SessionPool.java:421)
        at cn.edu.tsinghua.iotdb.benchmark.iotdb013.IoTDBClusterSession.lambda$insertOneBatchByTablet$1(IoTDBClusterSession.java:171)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

To Reproduce
Steps to reproduce the behavior:

• Server nodes:
  
• Server config:

# config node
internal_address=xx.xx.xx.159
target_config_nodes=xx.xx.xx.159:22277
schema_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus
data_region_consensus_protocol_class=org.apache.iotdb.consensus.multileader.MultiLeaderConsensus
schema_replication_factor=3
data_replication_factor=3

# data node-1
rpc_address=xx.xx.xx.159
internal_address=xx.xx.xx.159
target_config_nodes=xx.xx.xx.159:22277

# data node-2
rpc_address=xx.xx.xx.160
internal_address=xx.xx.xx.160
target_config_nodes=xx.xx.xx.159:22277

# data node-3
rpc_address=xx.xx.xx.161
internal_address=xx.xx.xx.161
target_config_nodes=xx.xx.xx.159:22277

# data node-4
rpc_address=xx.xx.xx.162
internal_address=xx.xx.xx.162
target_config_nodes=xx.xx.xx.159:22277

• Client

# iot benchmark config
DB_SWITCH=IoTDB-013-SESSION_BY_TABLET
HOST=xx.xx.xx.159,xx.xx.xx.160,xx.xx.xx.161,xx.xx.xx.162
PORT=6667,6667,6667,6667
BENCHMARK_CLUSTER=true
IS_ALL_NODES_VISIBLE=true

Then exec the command

sh benchmark.sh

Expected behavior

• Success to write records.

Related logs

• Client

org.apache.iotdb.rpc.StatementExecutionException: 411: no detailed failure reason in QueryStateMachine
        at org.apache.iotdb.rpc.RpcUtils.verifySuccess(RpcUtils.java:94)
        at org.apache.iotdb.rpc.RpcUtils.verifySuccessWithRedirection(RpcUtils.java:115)
        at org.apache.iotdb.session.SessionConnection.insertTablet(SessionConnection.java:589)
        at org.apache.iotdb.session.Session.insertTablet(Session.java:1573)
        at org.apache.iotdb.session.pool.SessionPool.insertTablet(SessionPool.java:446)
        at org.apache.iotdb.session.pool.SessionPool.insertTablet(SessionPool.java:421)
        at cn.edu.tsinghua.iotdb.benchmark.iotdb013.IoTDBClusterSession.lambda$insertOneBatchByTablet$1(IoTDBClusterSession.java:171)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Server

2022-09-26 14:37:26,292 [20220926_063726_03693_2-140] ERROR o.a.i.d.m.p.s.FragmentInstanceDispatcherImpl:129 - cannot dispatch FI for write operation 
java.lang.NullPointerException: null
        at org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.dispatchRemote(FragmentInstanceDispatcherImpl.java:179)
        at org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.dispatchOneInstance(FragmentInstanceDispatcherImpl.java:145)
        at org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.dispatchWriteSync(FragmentInstanceDispatcherImpl.java:125)
        at org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.dispatch(FragmentInstanceDispatcherImpl.java:96)
        at org.apache.iotdb.db.mpp.plan.scheduler.ClusterScheduler.start(ClusterScheduler.java:101)
        at org.apache.iotdb.db.mpp.plan.execution.QueryExecution.schedule(QueryExecution.java:255)
        at org.apache.iotdb.db.mpp.plan.execution.QueryExecution.start(QueryExecution.java:182)
        at org.apache.iotdb.db.mpp.plan.Coordinator.execute(Coordinator.java:146)
        at org.apache.iotdb.db.mpp.plan.Coordinator.execute(Coordinator.java:160)
        at org.apache.iotdb.db.service.thrift.impl.ClientRPCServiceImpl.insertTablet(ClientRPCServiceImpl.java:995)
        at org.apache.iotdb.service.rpc.thrift.IClientRPCService$Processor$insertTablet.getResult(IClientRPCService.java:3512)
        at org.apache.iotdb.service.rpc.thrift.IClientRPCService$Processor$insertTablet.getResult(IClientRPCService.java:3492)
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38)
        at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Desktop (please complete the following information):

• OS: centos
• Version
  • server: 0.14.0-preview3
  • client: 0.13

Additional context
Add any other context about the problem here.

[cserwen]

Looks similar to issue #7327

[github-actions]

Hi, this is your first issue in IoTDB project. Thanks for your report. Welcome to join the community!