Hardware tests with nimbro network make the router lag

[MartinSegaert]

Hello,
We tried a hardware test with two UAVs using nimbro with the MpcTracker and Se3Controller. We simply make both UAVs hover. We connect with one ground station PC to both UAVs in two terminals by ssh-ing to both. We found out that the WI-FI router (TP-Link Archer C50 - AC1200) has a lot of difficulties handling the nimbro network and the tmux sessions of both UAVs via ssh. By first using the default configuration of the nimbro file (shown below), the test works fine. The 2 UAVs hover, it is possible to navigate smoothly in the tmux session of both UAVs and the simultaneous landing of both UAVs works.

However, if we do one of the listed below (individually, starting from the default configuration), the tmux sessions start to lag or even completely freeze. Sometimes the WI-FI router completely stops working and has to be turned OFF and ON. This makes us lose control of the UAVs and they have to be manually landed with the transmitter.

• Echo rostopic control_manager/mpc_tracker/predicted_trajectory to one UAV from the other one (which gives a lot of printing in the tmux session)
• Increase the nimbro rate of ‘’mpc_tracker/predicted_trajectory too much’’ (It works for 2Hz and 5Hz. At 10 Hz it worked two times but failed one time, and at 20 Hz it fails completly)
• Printing 6 ROS_INFO_THROTTLEs in the MpcTracker with a period of 0.5 seconds

It seems that to make it work, almost nothing should be printed in the tmux sessions and the amount of information that can be communicated with the nimbro network should not be more than the equivalent of ‘’mpc_tracker/predicted_trajectory’’ at a rate of 5 Hz (and ideally only at a rate of 2 Hz) + the other topics in the nimbro file.

We want to make hardware tests with our controller and tracker with 2 UAVs communicating the following:
• UAV1 to UAV2
- 2x mrs_msgs::PositionCommand
- 1x mrs_msgs::BoolStamped
• UAV2 to UAV1 :
- 2x mrs_msgs::PositionCommand
- 2x mrs_msgs::UAVState
- 1x mrs_msgs::BoolStamped

In simulation (on two different computers using the nimbro network with the router) it works fine with a rate of 20 Hz. But it doesn’t in hardware. Is it because it is too much for the router when it has to ssh both tmux sessions as well ?
Our next step is to try to remove as much prints in the controller tab as possible and find at which maximal communication rate it works. As the ‘’mpc_tracker/predicted_trajectory’’ contains a horizon of 40 xyz-points, what we would like to communicate is roughly (more than) two times less data content. As ‘’mpc_tracker/predicted_trajectory’’ worked with 5 Hz. In our case, a communication rate of 10 Hz should be possible.

My questions are :

• Is it correct to ssh to both UAVs in two terminals ? As both UAVs need to be started (+ load trajectories can't be done with nimbro) we don't see another way to do it. Is there maybe a way to only show some of the tmux tabs to reduce the ssh communication ?
• Is it normal that the tmux sessions freeze when we do one of the things I mentioned? Can we do something about it except reduce the communication rate or the number of prints in the tmux sessions?
• How much bandwidth of the router does the ssh-ing use compared to the nimbro communication and how important is it to not print in the tmux sessions?
• How can we maximize the nimbro rate of our messages? Is it better to wrap different messages in one big message instead of sending them separatly? Is there something else we can do?
• With the limitation of the router we face, I don't understand how it would be possible to do hardware tests with more than 2 UAVs. Did you already do that ? And if yes, what was done to resolve our issue ?

Thanks for your help,

Martin

[MartinSegaert]

After having removed as much printing as possible (in our own codes) in the tmux sessions (Removed unnecessary prints + Period of prints on 10 seconds), the communication of the desired messages is possible at a rate of 20 Hz. However, the freezing of the sshing/router still happens on some occasions, but a higher communication rate does not seem to lead to more frequent freezes. The frequency of occurrence of these freezes are acceptable, it is just important to be ready to take over manually in case it happens.

It was also noted that the quality of communication is lower at the start, but also when arming the UAVs and during the takeoff. During flight, it works fine.

If you have any recommendation to improve the communication further, don't hesitate to mention it.

Sincerely,

Martin

[MartinSegaert]

After extra tests were done, it turns out my solution was not so robust after all.
To solve this, I have avoided almost any communication before take-off has finished and the in-flight controller is activated.
Initially I started communication before that.