Add process listeners to XML RPC server instance

[luator]

I recently noticed a bug in the roslaunch API: When I pass a ProcessListener to ROSLaunchParent, it should be called whenever a node that was launched with it dies. However, this was not working for nodes that are launched on a remote machine. See corresponding question on ROS Ansers.

I digged into the code and found out that the callback calls are already forwarded to the server, but on the server side there are multiple objects holding a list of ProcessListeners that is not properly synchronized, i.e. the instance that receives the callback call from the remote machine does not have the listeners that were passed to the ROSLaunchParent.

The change I propose fixes this (at least it is working well for me now). However, as the code is quite complicated and I don't have a complete oversight of what is going on there, I would be grateful if someone with a better understanding could check this to make sure there are no undesired side effects.

The PR is based on kinetic-devel as I currently do not have a setup to test with lunar.

[dirk-thomas]

Can you please provide a reproducible example how you have tested the patch.

[luator]

This contains a package with a minimal working example: test_remote_monitoring.tar.gz

How to use

Adapt the launch file to run the stuff on a remote machine, then start it with

rosrun test_remote_monitoring monitor.py _launch_file:=$(rospack find test_remote_monitoring)/launch/node.launch

The monitor node runs the launch file (starting two nodes node1 and node2) and registers a ProcessListener that prints a warning whenever a node died. The launch file starts two dummy nodes, node1 and node2.

Now kill the nodes one after another by calling:

rosnode kill node1
rosnode kill node2

Output

When launching locally

...
[node1-1] process has finished cleanly
log file: ...
[WARN] [rosout]: node1-1 died with code 0
[node2-2] process has finished cleanly
log file: ...
[WARN] [rosout]: node2-2 died with code 0
all processes on machine have died, roslaunch will exit

--> Warning is printed for each node.

When launching on remote machine

...
[remote-hostname-0]: [node1-1] process has finished cleanly
log file: ...
[remote-hostname-0]: [node2-2] process has finished cleanly
log file: ...
[remote-hostname-0]: all processes on machine have died, roslaunch will exit
[remote-hostname-0] process has died
[WARN] [rosout]: remote-hostname-0 died with code None
remote[remote-hostname-0]: unable to contact [remote-hostname] to shutdown cleanly. The remote roslaunch may have exited already.
all processes on machine have died, roslaunch will exit

--> Warning is not printed when the first node is killed. Only after the second one is killed, there is one for the whole launch file. None for the individual nodes, though.

When launching on remote machine with fix of this PR

...
[remote-hostname-0]: [node1-1] process has finished cleanly
log file: ...
[WARN] [rosout]: node1-1 died with code 0
[remote-hostname-0]: [node2-2] process has finished cleanly
log file: ...
[WARN] [rosout]: node2-2 died with code 0
[remote-hostname-0]: all processes on machine have died, roslaunch will exit
[remote-hostname-0] process has died
[WARN] [rosout]: remote-hostname-0 died with code None
remote[remote-hostname-0]: unable to contact [remote-hostname] to shutdown cleanly. The remote roslaunch may have exited already.
all processes on machine have died, roslaunch will exit

--> Now there are warnings for the individual nodes as soon as they are killed.

[dirk-thomas]

Thank you for the patch. I will close this in favor of #1319 which only difference is that it targets the lunar-devel branch.