This repository contains the code for the work described in Semantically-enhanced Variational Autoencoder for Deep Collision Prediction. At its current state, this repository is expected to be used with the code in ORACLE for inference.
The work is explained in the following videos:
- Overview: https://www.youtube.com/watch?v=Ni4VywUQCPw
- Detailed Explanation: https://www.youtube.com/watch?v=yoO5MqSPfKw
- Field Experiments: https://www.youtube.com/watch?v=9NZvVPvUrPo
Recommended: Anaconda or Miniconda
Clone the repository:
git clone git@github.com:ntnu-arl/sevae.gitTo install the repository, run the following commands:
cd sevae
pip3 install -e .The folders contain the following:
- networks: Contains the VAE network, and the loss functions for training the VAE
- datasets: Contain scripts that utilize pytorch's dataset class to read from a tfrecord file
- utils: Contains utility scripts for creating the tfrecord files, and other utilities
- weights: Contains the weights for the VAE
- inference: Contains the scripts for running the VAE node, and the scripts for interfacing with the VAE node with ROS-based simulators
- baselines: Contains the scripts for running the baseline compression methods (FFT and Wavelets)
To run the seVAE node for inference to obtain the latent space, run the following command:
cd sevae/inference/src
python3 vae_node.py --sim=True
The file train_seVAE.py contains the code used to train the Semantically-enhanced Variational Autoencoder for Deep Collision Prediction as described in this paper. Currently this is not supported with datasets but support for training yourselves will be added soon!
If you use this work in your research, please cite the following paper:
@misc{kulkarni2023semanticallyenhanced,
title={Semantically-enhanced Deep Collision Prediction for Autonomous Navigation using Aerial Robots},
author={Mihir Kulkarni and Huan Nguyen and Kostas Alexis},
year={2023},
eprint={2307.11522},
archivePrefix={arXiv},
primaryClass={cs.RO}
}