fix #13837 the Invalid log : worker worker have not received the hear…

[lxorc]

fix #13837 the Invalid log : worker worker have not received the hearbet

[ruanwenjun]

We don't need to add this check, this case might be due to we registering an empty node? Could you please describe how to reproduce this.

[lxorc]

We don't need to add this check, this case might be due to we registering an empty node? Could you please describe how to reproduce this.

Yes, by default a worker node will be loaded in registryClient.subscribe(REGISTRY_DOLPHINSCHEDULER_WORKERS, new WorkerDataListener());

registryClient.subscribe(REGISTRY_DOLPHINSCHEDULER_WORKERS, new WorkerDataListener()); 
String[] parts = path.split("/");  // REGISTRY_DOLPHINSCHEDULER_WORKERS = /nodes/worker
final String workerAddress = parts[parts.length - 1];  // workerAddress  = worker
if (type == Type.UPDATE) // the Type.UPDATE is  Node /nodes/worker update ,   case NODE_UPDATED: type(Type.UPDATE);
syncSingleWorkerNodeInfo(workerAddress, JSONUtils.parseObject(data, WorkerHeartBeat.class));  //  ???
workerNodeInfo.put(workerAddress, info);  // put "worker" key into the workerNodeInfo

The effect is that the addr of getHostWeight will be the worker and heartBeat == null.

so

        if (heartBeat == null) {
            logger.warn("worker {} in work group {} have not received the heartbeat", addr, workerGroup);
            return Optional.empty();
        }

[ruanwenjun]

We don't need to add this check, this case might be due to we registering an empty node? Could you please describe how to reproduce this.

Yes, by default a worker node will be loaded in registryClient.subscribe(REGISTRY_DOLPHINSCHEDULER_WORKERS, new WorkerDataListener());

registryClient.subscribe(REGISTRY_DOLPHINSCHEDULER_WORKERS, new WorkerDataListener()); 
String[] parts = path.split("/");  // REGISTRY_DOLPHINSCHEDULER_WORKERS = /nodes/worker
final String workerAddress = parts[parts.length - 1];  // workerAddress  = worker
if (type == Type.UPDATE) // the Type.UPDATE is  Node /nodes/worker update ,   case NODE_UPDATED: type(Type.UPDATE);
syncSingleWorkerNodeInfo(workerAddress, JSONUtils.parseObject(data, WorkerHeartBeat.class));  //  ???
workerNodeInfo.put(workerAddress, info);  // put "worker" key into the workerNodeInfo

The effect is that the addr of getHostWeight will be the worker and heartBeat == null.

so

        if (heartBeat == null) {
            logger.warn("worker {} in work group {} have not received the heartbeat", addr, workerGroup);
            return Optional.empty();
        }

So we only need to subscribe the event of REGISTRY_DOLPHINSCHEDULER_WORKERS's child path

[lxorc]

We don't need to add this check, this case might be due to we registering an empty node? Could you please describe how to reproduce this.

Yes, by default a worker node will be loaded in registryClient.subscribe(REGISTRY_DOLPHINSCHEDULER_WORKERS, new WorkerDataListener());

registryClient.subscribe(REGISTRY_DOLPHINSCHEDULER_WORKERS, new WorkerDataListener()); 
String[] parts = path.split("/");  // REGISTRY_DOLPHINSCHEDULER_WORKERS = /nodes/worker
final String workerAddress = parts[parts.length - 1];  // workerAddress  = worker
if (type == Type.UPDATE) // the Type.UPDATE is  Node /nodes/worker update ,   case NODE_UPDATED: type(Type.UPDATE);
syncSingleWorkerNodeInfo(workerAddress, JSONUtils.parseObject(data, WorkerHeartBeat.class));  //  ???
workerNodeInfo.put(workerAddress, info);  // put "worker" key into the workerNodeInfo

The effect is that the addr of getHostWeight will be the worker and heartBeat == null.
so

        if (heartBeat == null) {
            logger.warn("worker {} in work group {} have not received the heartbeat", addr, workerGroup);
            return Optional.empty();
        }

So we only need to subscribe the event of REGISTRY_DOLPHINSCHEDULER_WORKERS's child path

Please help to review my code to see if there are other problems. If there is a better implementation, please reply to me. And I have fixed it in my production environment and it works fine.

[ruanwenjun]

We don't need to add this check, this case might be due to we registering an empty node? Could you please describe how to reproduce this.

Yes, by default a worker node will be loaded in registryClient.subscribe(REGISTRY_DOLPHINSCHEDULER_WORKERS, new WorkerDataListener());

registryClient.subscribe(REGISTRY_DOLPHINSCHEDULER_WORKERS, new WorkerDataListener()); 
String[] parts = path.split("/");  // REGISTRY_DOLPHINSCHEDULER_WORKERS = /nodes/worker
final String workerAddress = parts[parts.length - 1];  // workerAddress  = worker
if (type == Type.UPDATE) // the Type.UPDATE is  Node /nodes/worker update ,   case NODE_UPDATED: type(Type.UPDATE);
syncSingleWorkerNodeInfo(workerAddress, JSONUtils.parseObject(data, WorkerHeartBeat.class));  //  ???
workerNodeInfo.put(workerAddress, info);  // put "worker" key into the workerNodeInfo

The effect is that the addr of getHostWeight will be the worker and heartBeat == null.
so

        if (heartBeat == null) {
            logger.warn("worker {} in work group {} have not received the heartbeat", addr, workerGroup);
            return Optional.empty();
        }

So we only need to subscribe the event of REGISTRY_DOLPHINSCHEDULER_WORKERS's child path

Please help to review my code to see if there are other problems. If there is a better implementation, please reply to me. And I have fixed it in my production environment and it works fine.

Does master path will have this problem? could you please change the master together?

[ruanwenjun]

Please fix the code style by mvn spotless:apply

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity for 120 days. It will be closed in 7 days if no further activity occurs.

[github-actions]

This pull request has been closed because it has not had recent activity. You could reopen it if you try to continue your work, and anyone who are interested in it are encouraged to continue work on this pull request.