METU Racing Driverless Perception pipeline

This is the official Github repository for the perception pipeline of the METU Racing's Driverless project. Some of the primary goals of this pipeline is solving for object detection through visual and LIDAR data.

Guidelines for using this repository (Important!)

• For a quickstart on git, check out using git.
• For understanding the general workflow of our repository, please refer to this document on getting started with the Perception pipeline.
• For rules on contributing to the repository, please first read this guide on contributing to git repositories.
• For a reference on writing proper commit messages, check out Conventional commits.

Notes

• Export the environment variable DRVLSS_PERCEPTION_PATH for easier reference to relative paths in our program.
• bag_to_images.py is an external tool that helps extract images from rosbags. It can (and should) be used to extract datasets from rosbags we come across (like EUFS or AMZ Driverless' datasets) and prepare a dataset.
• For training on Colab, check out the Jupyter notebook for training yolov4
• For creating Darknet configs & additional files, check out AlexeyAB's Darknet repo

How to predict

We rely on 2 submodules for running our YOLOv4 predictions: Darknet and Tianxiaomo/Pytorch-YOLOv4. Here are the use cases for these modules:

• Darknet for general training on datasets. We convert our dataset to the Darknet format and train. This can be done on any local machine, but for convenience and performance, should be done on cloud systems like Google Colab and/or Paperspace.
  • As an additional note, Paperspace does not generally allow for Darknet training, but Colab does; please check the section for training on Google Colab for more details.
• Tianxiaomo YOLOv4 for the base implementation of Darknet on Python. We use the code as a draft for our own program and compile with CUDA and OpenCV. We use the webcam code for live predictions, but do note that running the prediction code on most machines is slow.

Training our dataset on Google Colab

The whole process of training our dataset can be done on Google Colab, without needing additional hardware or subscriptions etc. The process can be split into two steps: preparing the dataset and training.

1. For preparing our dataset, follow the steps provided by Darknet's github page. The main steps to follow are creating the yolo-*.cfg file, obj.data file and obj.names file. Furthermore, the dataset needs to be prepared in a special way that each training, validation and testing subset needs to be split into its own subfolder, and each image must be accompanied with a text file with the same name containing the annotation data of the image. Finally, for each subset, there must be a text file specifying the absolute paths of the image files in the subset. These can be made as diverse as possible as creating different permutations of data may be used for cross-validation.
2. After the dataset is prepared, it must be uploaded to Google Drive for the training process. The model training folder must contain the appropriate folders for an easier workflow, specifically: a cfg folder for containing the yolo-*.cfg files, a data folder for containing our training data, our files obj.data, obj.names, train-*.txt, valid-*.txt, test-*.txt, and a backup folder for storing our pre-weights and weights files. After such a directory hierarchy is enforced, one can follow the steps provided in our Colab train notebook to train the model. Roughly, this notebook sets up the Drive directory, compiles Darknet for usage and starts training the model. For continued training, the Colab window must remain opened and monitored periodically to prevent timing out. The final output is a *.weights file, stored in the backup folder.
3. After training is complete, we only need to retrieve the yolo-*.cfg file, the obj.data, obj.names files and the yolo-*.weights files to plug into predict.py.

Environment variables

Our codebase has a lot of elements, and for the sake of preserving generality, we have to refrain from using absolute paths and/or relative paths inside our codebase. Instead, we should use environment variables so that we can keep our code portable. Here are some environment variables that we should set:

• DRVLSS_PERCEPTION_PATH: The root directory for the perception pipeline. Some of our code requires that this variable be set to run.

TODO

• Remove save to file features from the code and add it as a final command in predict.py

The data in use

Currently, all data in use is from rosbags as we have no other source for acquiring datasets/test materials. More specifically, the following are the data we have started using:

• The Sanderson car park dataset from the Edinburgh University FS team.
• The AMZ Driverless 2017 dataset from the AMZ Driverless github page.

Resources

The following items are resources we chose to follow during the development process. This list will be updated periodically.

• AMZ Driverless: The Full Autonomous Racing System
• The Complete Self-Driving Car Course - Applied Deep Learning
• OpenCV Python Tutorial For Beginners
• Robotics: Perception
• Self-Driving Cars Specialization
• Introduction to Computer Vision
• Getting Started with LIDAR
• Labeling Ground Truth for Object Detection
• Deep Learning with Python
• Hands-on Machine Learning with Scikit-Learn and Tensorflow