Crash Course on Autonomous Racing

Here are some of the most essential topics that are recommended for getting a racing system ready, most of which are from F1Tenth. This is a good platform for introducing topics in autonomous racing. Some in-depth topics may not be possible to cover in this README, but can be freely expolored!

Recommended to learn on-the-go, alongside system-specific learning/research. Every topic is highly useful for your benefit, but learn what you need for your own purpose. Challenging, but productive (and interesting) way to appreciate the design of autonomous systems, and to gain the most applicable skills. Especially good to learn if you are not in your final year of study, or want to learn outside of your studies.

This document will get updated regularly, so you can star this repo, or fork it. Some extra content may be added in addition to just this README file.

Before you start:

• You can install Ubuntu 18.04 LTS as a dual-boot OS following the Tutorial (BACKUP YOUR MAIN OS AND LOCAL DRIVES IN CASE OF DATA LOSS DUE TO INSTALLATION ERRORS, EVEN IF RARE) - Most reliable way once you have the OS installed
• You can run everything in a Docker container with Ubuntu 18.04 and ros:melodic-robot-bionic (reference to Dockerfile and Tutorial) - You may need to give Docker permissions after installation (using Docker engine in Ubuntu). Once you have Docker installed, try out our quick-start guide for running the F1Tenth Simulator with Docker.
• Ubuntu Virtual Machine (like VirtualBox) on your main OS is fine also, but may run slightly slower than dual-boot. Example guide here.
• Guide to ROS Melodic installation (prerequisite for F1Tenth Sim is ros-melodic-desktop, but full version is recommended, in case you want to more yourself).
• Get familiar with Linear Algebra and Probability on-the-go, as it's very common in the theory the more in-depth you go
• Check out the AMZ team resources for extra stuff like example systems, datasets, and references!
• ALSO: Start off from F1Tenth Gym for easier setup and for trying out the labs!

Stuck on anything? No worries!:

• Ask away by creating an issue here, for asking for help with any aspect of the material (theory, setup, debugging).
• Reach out to anyone and you'll get all the help possible!

ROS Tips

• This cheat sheet shows very comprehenisve command line commands for ROS, as well as common commands here

Introduction to Autonomous Racing

1. (F1Tenth) Lecture 1 - Introduces F1Tenth platform, autonomous racing concepts, and simulator (Ignore UPenn-specific logistics)
2. (F1Tenth) Tutorial 1 - Introduces the learning-based F1Tenth simulator (should run on most computers, backup main OS before making new Ubuntu installation). Sim installation on Ubuntu and more
3. (F1Tenth) Lab 1 - Introduces the Robot Operating System (ROS) framework, good for learning main features (we will use ROS Melodic as the distro) (Further tutorials on ROS in their wiki)
4. (YouTube) The ethical dilemma of self-driving cars - Patrick Lin
5. (YouTube) First Formula Student Germany (Driverless) 2017 Moments
6. (Tool) Learn X in Y minutes (for any language of your choice, we recommend Python, C/C++)
7. (Tutorial) Using ROS with C++ and Python Nodes - Also includes tutorials for C++ and Python individually. This is for extra reference

Core Concepts

1. (F1Tenth) Lecture 2 - Introduces safety systems, and AV sensors
2. (F1Tenth) Tutorial 2 - Introduces the addition of system planners in F1Tenth Simulator
3. (F1Tenth) Lab 2 - Introduces the programming of an Automatic Braking System in F1Tenth Sim. Important safety feature of car
4. (F1Tenth) Lecture 3 - Introduces coordinates, reference frames, rigid body transformations, and tf2_ros
5. (YouTube) Controlling Self Driving Cars - Brief introduction to controller design (PID)
6. (F1Tenth) Lecture 4 - Laplace Domain Dynamics & PID
7. (F1Tenth) Lab 3 - Wall Follower Controller Intro - Use a PID controller to race the car along the walls of the track! (Brian Douglas on MATLAB has a useful intro to PIDs series, along with in-depth control systems topics on his own channel)
8. (YouTube) AMZ Driverless presents Learning-Based Model Predictive Control to overcome model inaccuracies! (quick 3 min research presentation)

Reactive Racing Methods

1. (F1Tenth) Lecture 6 - Follow the Gap Method and variants (details, pros and cons of non-planning methods are introduced)
2. (F1Tenth) Lab 4 - Have a go at designing and tuning your own Follow the Gap method!
3. (YouTube) Reactive racing success in head-to-head

Mapping and Localization Intro

1. (F1Tenth) Lecture 9 & 10 - Details of the ideas, and some algorithms that can be used
2. (F1Tenth) Lecture 11 - Particle filter localization, and in-depth details of algorithm
3. (F1Tenth) Lecture 12 - Introduces modern SLAM using Google Cartographer as part of a mapping and localization system, key breakthroughs in efficiency, and guide on installing it
4. (F1Tenth) Lab 5 - Create your own Scan Matching system (group work is recommended from this lab onwards, provides a rigorous and effective way to learn about optimizing an algorithm like this, but challenging)
5. (YouTube) AMZ Driverless on Redundant Perception and State Estimation - How robust perception plays a key role in Localization
6. (Book/Repo) Theory and implementation of Kalman/Bayesian Filters (useful for localization)

Planning Methods

1. (F1Tenth) Lecture 13 - Introduction to Pure Pursuit racing and waypoint following
2. (F1Tenth) Lab 6 - Use Google Cartographer SLAM and Pure Pursuit planning to race your car!
3. (F1Tenth) Lecture 14 - Path Planning/Search methods, and RRT/RRT* for local planning
4. (Tool) Uniform Grid Path Planner simulations by Lai Yan Kai - Great for visualizing step-by-step calculations in any configuration for a few global planning algorithms
5. (F1Tenth) Lab 7 - Design your own motion planner system! Learn about different practical considerations of path planning, and RRT/RRT*
6. (YouTube) MATLAB Intro to Autonomous Path Planning
7. (F1Tenth) Lecture 20 - Raceline Optimization (introduces methods for solving the quickest route for any track, or planning-based control problem)

Control

1. (YouTube) Intro to Fourier Transforms and Laplace Transforms - a baseline topic that you will see often in analyzing control problems, and a great, concise playlist for introducing the maths, and the purpose of these representations. More in-depth topics on the Brian Douglas channel
2. (YouTube) Model Predictive Control (MPC) by MATLAB - comprehensive introduction and guide on what MPC is about
3. (F1Tenth) Lecture 21 - MPC, alternative introduction, with further references for F1Tenth racing

Vision

1. (F1Tenth) Lecture 17 - Introduction to vision model and pose estimation
2. (F1Tenth) Lecture 18 - Multi-view geometry, and YOLO object detection
3. (YouTube) Stanford CS231n by Andrej Karpathy - rigorous coverage of computer vision, specifically Convolutional Neural Networks for Visual Recognition. Long, detailed lecture series. Pre-requisite to Deep Learning is Machine Learning
4. PyTorch Tutorial - currently more common framework for Deep Learning researchers (also Lightning)
5. Keras quick guide - built on TensorFlow framework, research work sources may also reference TF1.0, TF2.0, Keras (different API changes throughout TF versions, but should not be varying for core machine learning functionality, and similar to PyTorch in quite a few ways)

Reinforcement Learning (RL)

1. Introduction to Deep Learning - Here's a link to the full MIT Deep Learning course. It's really easy to grasp and they have a bunch of interesting labs for you to work on.
2. OpenAI Gym docs/tutorial - This is the most widely used framework for RL experiment benchmarking
3. Spinning Up by OpenAI - Rigorous guide to RL and deep RL, useful for new researchers
4. (YouTube) Solving Rubik's Cube with a single robot hand - very challenging for even people to do one-handed
5. (YouTube) David Silver's DeepMind intro to RL - lecture series that delivers holistic view on RL, very good for understanding but not the most practical for implementation of programs.
6. Alternative RL course with Colab-ready notebooks