A curated collection of papers and research materials that students need to be aware of when they are getting started with research in the lab. Students are not expected to read all of them, only the resources that are most related to their work. They are categorized here in terms of topics:
UofT Robotics Institute Seminar Series
Vector Institute Visitor Talks
Control meets learning seminar series
Fields Institute ML Seminars 21'-22'
Fields Institute ML Seminars 20'-21'
Week 5, Motion Planning, from Florian's CSC477
Steve Lavalle's book is the go-to reference in this field.
Sidd Srinivasa's talk
Logic Geometric Programming and differentiable modes
Week 3, PID Control, from Florian's CSC477
Week 4, LQR, from Florian's CSC477
Lecture 2 on LQR, iterative LQR, and model-based RL, from Florian's grad course
An introduction to trajectory optimization
Underactuated robotics course at MIT, 2020 version, from Russ Tedrake. Also check out his online book on the same topic.
All of Steve Brunton's videos on learning for control are interesting, but the following playlists in particular are worth understanding:
- Koopman Analysis, which turns nonlinear dynamics problems into linear.
- Data-Driven Dynamical Systems, which presents techniques for learning dynamical systems from data
- Data-Driven Control, which presents techniques for learning controllers from data, using learned dynamical systems.
Jean Jacques Slotine's grad course on nonlinear control. It covers stability and convergence of nonlinear systems, adaptive control, system identification, and when can you approximate a nonlinear system by a linear system.
Nikolai Matni's grad course on learning and control
Advanced dynamics course from Zac Manchester
AA 203: Optimal and Learning-Based Control
Control meets learning seminar series and the associated Youtube channel
gradSim (differentiable physics and rendering) for system identification
Robotic Manipulation, Russ Tedrake
Topics in Advanced Robotic Manipulation, Jeannette Bohg
Advanced Robotic Manipulation, Oussama Khatib
Learning for Adaptive and Reactive Robot Control: A Dynamical Systems Approach, by Aude Billard, Sina Mirrazavi, Nadia Figueroa
Florian's imitation learning seminar course. Look at the slides.
Yisong Yue's imitation learning talk
RL course at UCL, by David Silver. Check out the slides and youtube videos. This course is good for discrete RL in games like chess and Go, so definitely not tailored to robotics.
Deep RL course at Berkeley, by Sergey Levine. Check out the youtube videos. This course is good for both discrete and continuous state and action RL, so it is applicable to robotics.
What is a good state representation/encoding for RL? See these papers and their related works to get started:
- https://arxiv.org/abs/2207.08229 (state encoding should be predictive of the next action in a multi-step inverse dynamics model)
- https://arxiv.org/abs/2212.14511 (state encoding should be predictive of rewards)
- https://arxiv.org/abs/1811.04551 (state encoding should be predictive of next states, rewards, and able to reconstruct observations)
Exploration strategies in deep RL
Monotonic improvement in model-based RL: see https://arxiv.org/abs/1807.03858 and https://arxiv.org/abs/1805.10755
Lessons from AlphaZero for Optimal, Model-Predictive, and Adaptive Control (Bertsekas)
These courses are about having controllers/policies/planning algorithms that get better over time, as they solve more problems. These two courses are unique in the world and very much bleeding edge.
David Duvenaud's seminar course. Combining Monte Carlo Tree Search with neural networks, learning from expensive algorithms.
Yisong Yue's seminar course. Learning for branch and bound optimizers, learning A* heuristics. Lots of good stuff here.
Scaling probabilistically safe imitation learning, Scott Niekum, online talk, which can also be found here.
Safe Learning MPC, by Melanie Zeilinger, online talk
Safety-critical continuous and discrete-continuous systems, by Aaron Ames, online talk
Backwards reachability for control, via the Hamilton-Jacobi-Bellman equation survey paper and tutorial. Note that these approaches do not scale in more than 10-15 dimensions.
Introduction to reachability for linear systems, a tutorial.
How should a robot assess risk? Towards an axiomatic theory of risk in robotics, by Anirudha Majumdar. paper
A survey paper on safe RL
Constrained Policy Optimization paper
Conservative Safety Critics for Exploration
Roger Grosse's seminar course on uncertainty modeling and active learning
VIME: Variational Information Maximizing Exploration
All of these course are about physics-based animation, how to simulate realistic motion, and how to handle contacts, deformable objects.
To understand some of the material in these courses you will need to understand classical physics and mechanics. A good resource is
- Classical physics by V. Balakrishnan at IIT Madras.
DiffTaichi: Differentiable Programming for Physical Simulation
Awesome multibody dynamics simulation
The frontier of simulation-based inference
BayesSim: adaptive domain randomization via probabilistic inference for robotics simulators
Domain randomization blog post
3D Gaussian Splatting for Real-Time Radiance Field Rendering
Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer
Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning
Mitsuba2: A Retargetable Forward and Inverse Renderer paper
Neural scene representation and rendering
SCALOR: Generative World Models with Scalable Object Representations
Generative Hierarchical Models for Parts, Objects, and Scenes
SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decomposition
Animesh Garg's graduate seminar course on 3D and Geometric Deep Learning
Joint 3D Proposal Generation and Object Detection from View Aggregation
Multi-View 3D Object Detection Network for Autonomous Driving
PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows
PointNetVLAD: Deep Point Cloud Based Retrieval for Large-Scale Place Recognition
SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks
One-Shot Informed Robotic Visual Search in the Wild. See the related works section.
A Metric Learning Reality Check
A Theoretical Analysis of Contrastive Unsupervised Representation Learning
SCALOR: Generative World Models with Scalable Object Representations
State Estimation for Robotics, by Tim Barfoot, Professor, University of Toronto
Bayesian Filtering and Smoothing, by Simo Sarkka, Professor, Aalto University
Factor Graphs for Robotic Perception, by Profs. Frank Dellaert and Michael Kaess
Variational Inference: a Review for Statisticians
Tutorials on Variational Autoencoders (VAEs): https://arxiv.org/abs/1606.05908 and https://jaan.io/what-is-variational-autoencoder-vae-tutorial/
[]
The art of writing effectively, by Larry McEnerney, Director of the University of Chicago's Writing Program
Writing beyond the Academy, by Larry McEnerney, Director of the University of Chicago's Writing Program
Eight lessons learned in two years of PhD, by Muhammad Khalifa, PhD student, U. Michigan
Principles for productive group meetings, by Jacob Steinhardt, Assistant Professor, UC Berkeley
How I read research papers, by Aaditya Ramdas, Assistant Professor, CMU
Principles for a PhD program, by Seong Joon Oh, group leader, U. of Tuebingen
How to achieve success in a machine learning PhD, by Patrick Kidger, PhD student, Oxford.