How to use and extend the MRS software platform?

[bconvens]

Hello MRS team,

Note: @klaxalk asked me to place my questions here since he will go on holiday.

As I told some of you during the MRS summer school, I (and some of my colleagues) would really like to use (and possibly extend) your system for research on constrained and distributed control of aerial swarms.

Currently, we have two identical quadrotor builds composed of:

• F330 frame kit
• Motors: Racerstar Racing Edition 2216 BR2216 1400KV
• Props: Gemfan 8045 8X4.5 8 Inch Carbon Nylon Propeller EPP
• ESCs: Racerstar RS30A Lite 30A Blheli_S BB1
• Battery 3S: ZOP Power 11.1V 2800mAh 3S 60C Lipo Battery XT60 Plug
• Transmitter: Flysky FS-i6X i6X 10CH 2.4GHz AFHDS 2A RC
• Receiver: FS-iA10B
• Companion computer = flight controller (in my setup): raspi 3b+ and navio2 shield
• GPS: navio2 GPS/GNSS antenna
• No additional sensors (yet). Only the minimal necessary to make it fly and do position control.

I can teleoperate the drone using ardupilot (running on the raspbian OS) and I can calibrate/configure the settings using QGroundControl.
I did not yet test position control with gps, but would like to test it with your framework.

The questions I have:

1. If possible without expecting "big issues/risks of incompatibilities" and too much "lost time", it would be great if your system (and the ardupilot or px4 autopilot) can run onboard a cheap raspberry pi (model 3 or 4)
1.1 What is the quickest way to test if there is enough CPU/RAM on the raspi to run your software and the ardupilot or px4 autopilot?
1.2 Would it be enough if I install Ubuntu 18 server on the pi and try doing your 1 drone simulation tests?
2. CONFIG A: On the raspi I currently have raspbian OS, the specific version preconfigured by emlid for ardupilot on navio2. In your setup, you are running PX4 on pixhawk hardware. You have an intel nuc as a companion computer with ubuntu 18 and in my setup I use the raspi as both autopilot and companion computer.
2.1 Did you install the ubuntu 18 full desktop version or server version of the NUC?
2.2 Both ardupilot and PX4 seem to be -but I have no experience with this- compatible with MAVROS. So assuming I only use a single raspberry pi and navio2, which OS should be installed on the pi (raspbian by emlid, ubuntu server, ubuntu mate, ...)?
2.3 Should I still use ardupilot or should I use PX4 as you do? PX4 on raspi seems experimental/discontinued, assuming this is not an issue, which build to choose: native or cross-compiling? You don't seem to have a make working for this target yet.
3. CONFIG B: If it would be easier or if I need more compute power at some point,
3.1 would I still be able to use this raspi and navio2 as flight controller (running PX4 or ardupilot) and add another companion computer (e.g. an intel nuc or raspi, ...)?
3.2 Which OS to install then on the companion and on the raspi?
3.3 So should I do something similar as explained in your wiki and connect the navio2 UART port with an FTDI serial to USB converter to any onboard computer we want to use that can run your system?
4. Assuming I have a system that can be configured to run your software using CONFIG A and/or CONFIG B. Which steps should I prepare in simulation and follow during the real experiment to test safely some outdoor flight (e.g. running your MPC tracker) using gps with a single drone (my custom build)?
5. What changes if I want to test a basic example for 2 drones using normal gps?
6. If I understood correctly, you are also using RTK gps:
6.1 Is it correct that you mainly use this if you want to know precisely the distance between 2 drones? With normal gps there are relative position errors of the order of 1 meter, right?
6.2Which specific RTK gps do you use? Does it work well?
7. As a final backup strategy it would be great if you could provide a list with the drone hardware (e.g. F450) such that we can buy the components and are 100% sure everything will be compatible.

Many thanks for all your help and suggestions!
Best,
Bryan

[matemat13]

1 The system should run as-is on a Raspberry Pi or most of the other common ARM-based platforms (e.g. Jetson TX, Xavier NX or the Odroid).
1.1 Run the simulation environment on a separate computer, connected through Ethernet LAN to the Raspberry Pi/whatever. Run the MRS self-localization and control pipeline on the device to be tested for CPU/RAM. The px4 autopilot runs on a separate embedded microcontroller (we use the PixHawk autopilot HW), so it's not limited by computational resources of the onboard PC. According to our experience, the system should run out of the box onboard a Raspberry Pi or equivalent platform.
1.2 Enough for what? If you want to test the CPU/RAM load, bear in mind that the simulation itself is significantly more computationally demanding, then the MRS self-localization and control pipeline. Therefore if you run the whole simulation on the Raspberry Pi, you'd be testing the computer as a simulation platform and not as an onboard computer platform, which is probably not what you want. If you just want to run the MRS pipeline, you can get off with a much weaker CPU than for a whole simulation.

2 In general, I'd say just try to install our system on a freshly flashed Raspberry Pi with Ubuntu 18.04 and see what works and what doesn't. It should work. If you encounter any specific problems, try to solve them one by one (pull requests welcome) or contact us in case you need something from our side.
2.1 We use the normal desktop version of Ubuntu, because we usually utilize GUI for debugging purposes etc. (our full Linux setup is available here: https://github.com/klaxalk/linux-setup). However, a minimal installation should be sufficient. Worse case, you might encounter some missing packages, which will have to be manually installed, if we forgot about some in the installation scripts.
2.2 I also have no experience with ardupilot, but AFAIK, it should be compatible with MAVROS. We use px4. Regarding the OS, we're using Ubuntu 18.04, so I'd start with that, if possible. The flavor (lubuntu/xubuntu/kubuntu) is irrelevant, as we don't rely on the window manager anyways (in fact, we use a separate window manager for work - the i3wm, but the MRS pipeline is completely separate from a WM).
2.3 See response to 2.

3 Instead of this solution, I'd recommend using the PixHawk in place of the Raspberry Pi. This is the solution we are using currently and have experience with. We've not tested using Raspberry Pi or any other embedded computer in place of the PixHawk.
3.1 I don't see why not, but we can't really help you with this solution as we have no experience with it.
3.2 I guess whichever suits you the most. On the computer where you intend to run our pipeline, I'd recommend using Ubuntu 18.04, since that's what we are using.
3.3 That seems like a possible solution if you insist on using navio2 as the flight controller instead of PixHawk.

4 I refer you to our documentation: https://ctu-mrs.github.io. There is a specific section about simulation with a detailed tutorial on how to set up a basic simulation experiment, which is a good starting point to see that everything is working. Then you should add your specific HW platform into the simulation environment and test the system with it. You'll most probably have to set the correct weight, dimensions, motor layout and motor constants. @klaxalk may provide better details regarding this procedure. After your platform is flying in simulations, you can try going to real-world experiments.

5 You add another drone ;) On a more serious note - you should have the drones on a common LAN over WiFi and enable the collision avoidance to... well... avoid collisions. Although perhaps your question was more specific? If so, please rephrase.

6 Yes.
6.1 With normal GPS, there are potentially much larger position errors. It wildly depends on specific conditions (satellite visibility, EM interference, quality of GPS etc.). Nowadays, we're using RTK GPS mostly as a ground-truth for evaluation of various localization and detection algorithms.
6.2 We were using the Tersus GNSS RTK, but we don't have very good experience with it. It takes forever for the GPS to acquire RTK fix and communication between the base station and RTK modules is unreliable. We've been testing a new solution - @DanHert may know more about that.

7 I'll look for some list. Again, @DanHert may have a better idea.

Hope this was helpful :) Feel free to ask more questions in case anything is still unclear!

[bconvens]

Thanks a lot for the responses @matemat13 !

I'll keep you posted if at some point this alternative raspberry Pi + navio2 solution would be working.
But first, I prefer to walk on a safer road and learn how to use the exact same hardware setup as you are using.

I have one main follow-up question concerning the transition from simulation to real-hardware tests.
Assume I have built the same F450 drones as you have (so using intel NUC) and the mrs-uav-system is installed on the NUCs.
Assume that the multiple NUCs and a ground station are connected via router over a common LAN over WiFi.
Up to now I only ran some basic simulation tests e.g. three drones gps by running those simulation Tmux sessions. Is there any equivalent basic tmux session for outdoor swarm use you could point me to? I assume from the ground PC you are telling each NUC somehow to only run a specific part of the code. That part is not clear how you do that.

Thanks!

[DanHert]

Hello, regarding the tmux sessions that run on the drones, we have examples here:
https://github.com/ctu-mrs/uav_core/tree/master/tmux_scripts
We use bare tmux scripts for real drones, not tmuxinator.
You simply SSH into the drone over wifi, and run the tmux script there.
You can also set your computer in a way to run the required tmux session right away on computer startup.