Unmanned Aerial Underwater Vehicle

Double Critic Deep Reinforcement Learning for Mapless Navigation of Unmmaned Aerial Vehicles.

Summary

We presents a novel deep reinforcement learning-based system for goal-oriented mapless navigation for Unmanned Aerial Vehicles (UAVs). In the context of aerial vehicles, image-based sensing approaches are commonly used, which demands high processing power hardware which can be heavy and difficult to embed into UAVs, mainly for small UAVs. With our proposed approach there is no need for image processing power. Instead, we propose a simple learning system that uses only a few sparse range data from a distance sensor to train the learning agent. We based our approaches on two state-of-art double critic Deep-RL techniques: Twin Delayed Deep Deterministic Policy Gradient (TD3) and Soft Actor-Critic (SAC). We show that our approaches manage to outperform an approach based on the Deep Deterministic Policy Gradient (DDPG) technique and the BUG2 algorithm. Also, our new Deep-RL structure based on Recurrent Neural Networks (RNNs) outperforms the current structure used to perform mapless navigation of mobile robots. Overall, we conclude that Deep-RL approaches based on double critic with Recurrent Neural Networks (RNNs) are better suited to perform mapless navigation and obstacle avoidance of UAVs.

Setup

All of requirements is show in the badgets above, but if you want to install all of them, enter the repository and execute the following line of code:

pip3 install -r requirements.txt

Before cloning the repository we need to configure your workspace. To do this proceed with the following commands in your terminal:

mkdir -p ~/hydrone/src

cd ~/hydrone/

catkin_make

Now that the workspace is already configured just enter the src folder and clone the repository, finally compile the project. To do this proceed with the following commands in your terminal:

cd ~/hydrone/src/

git clone https://github.com/ricardoGrando/hydrone_deep_rl_jint --recursive

cd ~/hydrone/

catkin_make

We now need to configure your terminal to accept the commands directed to our hydrone workspace. For this you can simply copy the line of code below to your .bashrc (or .zshrc if you use zsh instead of bash) or put the code directly into your terminal. Note that if you choose the second option, every time you open a new terminal you will have to give the following command again.

For bash:

source ~/hydrone/devel/setup.bash

For zsh:

source ~/hydrone/devel/setup.zsh

Okay, now your Hydrone is ready to run!
To do this, just execute the following command:

roslaunch hydrone_deep_rl_jint hydrone.launch

Media

We have the official simulation video posted on youtube, to access it just click on the following hyperlink: Video

If you liked this repository, please don't forget to starred it!