Add support for running multiple CommsClient nodes #772
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds proper support for having multiple CommsClients in different ROS nodes (or even multiple clients in one ROS node).
Solves: #638, #604 (yes, multicast!) and #270.
It is a rather large PR. I must admit much larger than I imagined when I started to work on it (given the fact that our half-sufficient workaround was just one file; but that workaround still did not work when nodes got restarted). Yet I still think this is the (more or less) minimal set of changes to provide proper support for multiple clients at least on the ROS side. There can still be only a single CommsClient with
_useIgnition=true
. But that's not a big problem because that client is a part of the bridge (or sim) container, so it's easy to make sure there is only one. On the other hand, solution containers using_useIgnition=false
will now get much bigger freedom in spawning commsclients.I even added support for calling
Unbind()
, although it seems to me that it's not that useful in SubT challenge (but hey, what if!).What this PR brings
subt::communication_client::CommsClient
(even in one ROS node, e.g. several nodelets can have their own commsclients). Each of these clients has its own separate list of bound endpoints which get unregistered when this client goes out of scope.CommsClient::Unbind()
. This will stop calling the callback bound in the correspondingBind()
call.respawn="true"
and they will still work even after a crash.What this PR breaks
This PR of course breaks a lot of APIs/ABIs. I tried to keep the changes for downstream users to a minimum, and a proof that I was successful in that is that there's literally no "downstream" code in osrf/subt that would need any changes (BaseStation/Controller plugins, tests, subt_example_node). Here's a view of the changes from a few different perspectives.
Users of
subt::communication_client::CommsClient(..., _useIgnition=false)
Bind()
call which now returns a list of registered endpoint IDs instead of plain bool. I added extra backwards compatibility feature by overloadingoperator!
for the result type, so that people can still easily testif (!client->Bind(...)) ...
. I couldn't figure out how to support also a positive test likeif(client->Bind(...))...
. People who use just the CommsClient and did not check the return value of Bind() should be completely unaffected by this PR (except gaining all of the bonuses it brings).CommsClient
has two more optional parameters._listenBeacons
specifies whether this client should automatically bind to the beacon port and fill the Neighbors() array_rosNh
is an optional ROS nodehandle to use for the created/subscribed ROS topics/services (e.g. in case users want to remap some of the topics)Direct users of the ROS API:
subt_ros/srv
have changed. They're now using client/endpoint IDs instead of plain names for unregistration, unbind and as return values. The Bind.srv also requires a related client ID instead of address. It should be pretty easy to maintain a mapping between client/endpoint IDs and their corresponding names./robot
, but are instead published on topic/robot/comms
. This is the key part that allows multiple CommsClients to coexist.Users of
subt::communication_client::CommsClient(..., _useIgnition=true)
_useIgnition=false
, there is nothing changed but the return type ofBind()
.Direct users of the Ignition Transport API:
Broker::OnAddrRegistration
,Broker::OnAddrUnregistration
andBroker::OnEndpointRegistration
for their new signatures. AllUInt32
fields stand for some ID (clientId or endpointId).Validation
To be more sure this PR actually does what it says, I added lots of unit/integration tests. They can probably still be extended, but I think they capture the basic functionality well. As another measure to verify validity of the changes, I ran a subt_comms_test with two subt_example robots, and everything went fine. I was even able to restart each of the "solution" launch files without restarting the simulator and the test still went ok (but it needed subt_comms_test to be restarted because it lost track of the connections).
As a last point, I want to run a full systems test with our solution containers and a full 6-robot team. I will exchange our custom multi-client workaround for the changes in this PR and observe the behavior of the system. Our system normally runs two commsclients on the teambase - on for communicating with other robots, and one for communication with the basestation. So this test should practically verify that a multi-client setup works. I will also try to kill some of the communication nodes to see whether restarting them is possible and they are fully operational afterwards.