Creating multiple spawners can cause issues finding the CM

[fmauch]

Describe the bug
When using multiple controller spawners started at once, it can happen that one or more don't find the controller_manager node.

While chasing a flaky test on our driver package we realized that it sometimes happens that a specific controller spawner hangs in Waiting for '/controller_manager' node to exist, which seems weird as in the same launch file we start multiple spawners and the others spawn fine, so the CM node definitively does exist.

Sure, reducing the number of spawners using the controller list feature could potentially improve things, but we would still need two spawners: one for active and one for inactive controllers and, as far as I understand internally the mechanisms for node discovery are similar to having individual spawners, anyway.

To Reproduce
I've tried to narrow it down and created a small package that shows the issue: https://github.com/fmauch/ros2_node_exploration_test. It is independent of the controller_manager, as I wanted to see whether the problem persists without anything of ros2_control around it.

In there there are two launch files that create a /talker node and multiple nodes that check for its existence

• The node_finder.launch.py file uses the same discovery method as the controller spawner. It starts 20 instances of the node_finder and rarely all of them succeed in finding the /talker node.
• The node_finder_timer.launch.py uses a spin function and a timer to check for /talker. In the launch file I start 50 instances of node_finder_timer and so far I haven't been able to produce a failing instance.

Expected behavior
Spawners should always find existing nodes correctly

Environment (please complete the following information):

• Version Rolling (master)

Additional context
I could potentially see two ways of resolving this:

1. Reporting this upstream. However, I fear the reaction may bee "You're using it wrong, since you never spin in your node". I might be wrong, though.
2. Change the spawner to actually use a spin mechanism. I'm happy to create a PR if this is the desired way forward.

[fmauch]

As discussed, I'll create a PR modifying the spawner :-)