README.md

Awesome-Imitation-Learning:

A curated list of awesome imitation learning (including inverse reinforcement learning and behavior cloning) resources, inspired by awesome-php.

Contribution

Please feel free to send me pull request or email (kriswu8021@gmail.com) to add links.

Table of Contents

• Papers
• Tutorials and Talks
• Blogs

Papers

General settings

• How Resilient Are Imitation Learning Methods to Sub-optimal Experts?, Gavenski et al., BRACIS 2023

• IQ-Learn: Inverse soft-Q Learning for Imitation, D. Garg et al., NeurIPS 2021

• Learning from Imperfect Demonstrations from Agents with Varying Dynamics, Z. Cao et al., ICRA 2021

• Robust Imitation Learning from Noisy Demonstrations, V. Tangkaratt et al., AISTATS 2021

• Generative Adversarial Imitation Learning with Neural Networks: Global Optimality and Convergence Rate, Y. Zhang et al., ICML 2020

• Provable Representation Learning for Imitation Learning via Bi-level Optimization, S. Arora et al., ICML 2020

• Domain Adaptive Imitation Learning, K. Kim et al., ICML 2020

• VILD: Variational Imitation Learning with Diverse-quality Demonstrations, V. Tangkaratt et al., ICML 2020

• Imitation Learning from Imperfect Demonstration, Y. Wu et al., ICML 2019

• A Divergence Minimization Perspective on Imitation Learning Methods, S. Ghasemipour et al., CoRL 2019

• Sample-Efficient Imitation Learning via Generative Adversarial Nets, L. Blonde et al., AISTATS 2019

• Sample Efficient Imitation Learning for Continuous Control, F. Sasaki et al., ICLR 2019

• Random Expert Distillation: Imitation Learning via Expert Policy Support Estimation, R. Wang et al., ICML 2019

• Uncertainty-Aware Data Aggregation for Deep Imitation Learning, Y. Cui et al., ICRA 2019

• Goal-conditioned Imitation Learning, Y. Ding et al., ICML Workshop 2019

• Adversarial Imitation Learning from Incomplete Demonstrations, M. Sun et al., 2019

• Generative Adversarial Self-Imitation Learning, J. Oh et al., 2019

• Wasserstein Adversarial Imitation Learning, H. Xiao et al., 2019

• Learning Plannable Representations with Causal InfoGAN, T. Kurutach et al., NeurIPS 2018

• Self-Imitation Learning, J. Oh et al., ICML 2018

• Deep Q-learning from Demonstrations, T. Hester et al., AAAI 2018

• An Algorithmic Perspective on Imitation Learning, T. Osa et al., 2018

• Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning, I. Kostrikov et al., 2018

• Universal Planning Networks, A. Srinivas et al., 2018

• Learning to Search via Retrospective Imitation, J. Song et al., 2018

• Third-Person Imitation Learning, B. Stadie et al., ICLR 2017

• RAIL: Risk-Averse Imitation Learning, A. Santara et al., NIPS 2017

• Generative Adversarial Imitation Learning, J. Ho et al., NIPS 2016

Applications

• Model Imitation for Model-Based Reinforcement Learning, Y. Wu et al., 2019

• Better-than-Demonstrator Imitation Learning via Automatically-Ranked Demonstrations, D. Brown et al., CoRL 2019

• Task-Relevant Adversarial Imitation Learning, K. Zolna et al., 2019

• Multi-Task Hierarchical Imitation Learning for Home Automation, R. Fox et al., 2019

• Imitation Learning for Human Pose Prediction, B. Wang et al., 2019

• Making Efficient Use of Demonstrations to Solve Hard Exploration Problems, C. Gulcehre et al., 2019

• Imitation Learning from Video by Leveraging Proprioception, F. Torabi et al., IJCAI 2019

• Adversarial Imitation Learning from Incomplete Demonstrations, M. Sun et al., 2019

• End-to-end Driving via Conditional Imitation Learning, F. Codevilla et al., ICRA 2018

• R2P2: A ReparameteRized Pushforward Policy for Diverse, Precise Generative Path Forecasting, N. Rhinehart et al., ECCV 2018 [blog]

• End-to-End Learning Driver Policy using Moments Deep Neural Network, D. Qian et al., ROBIO 2018

• Learning Montezuma’s Revenge from a Single Demonstration, T. Salimans., et al., 2018

• ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst, M. Bansal et al., 2018

• Video Imitation GAN: Learning control policies by imitating raw videos using generative adversarial reward estimation, S. Chaudhury et al., 2018

• Query-Efficient Imitation Learning for End-to-End Autonomous Driving, J. Zhang et al., 2016

Survey papers

• Imitation Learning: Progress, Taxonomies and Challenges, Zheng et al., 2022

• Deep Reinforcement Learning: An Overview, Y. Li, 2018

• A Brief Survey of Deep Reinforcement Learning, K. Arulkumaran et al., 2017

• Imitation Learning : A Survey of Learning Methods, A. Hussein et al.

Robotics and Vision

• Graph-Structured Visual Imitation, M. Sieb et al., CoRL 2019

• On-Policy Robot Imitation Learning from a Converging Supervisor, A. Balakrishna et al., CoRL 2019

• Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Reward, M. Vecerik et al., 2017

Cold-start methods

• Zero-Shot Visual Imitation, D. Pathak et al., ICLR 2018

• One-Shot Hierarchical Imitation Learning of Compound Visuomotor Tasks, T. Yu et al., 2018

• One-Shot Imitation Learning, Y. Duan et al., NIPS 2017

Learning multi-modal behaviors

• Learning a Multi-Modal Policy via Imitating Demonstrations with Mixed Behaviors, F Hsiao et al., 2019

• Watch, Try, Learn: Meta-Learning from Demonstrations and Reward. Imitation learning, A. Zhou et al., 2019

• Shared Multi-Task Imitation Learning for Indoor Self-Navigation, J. Xu et al., 2018

• Robust Imitation of Diverse Behaviors, Z. Wang et al., NIPS 2017

• Multi-Modal Imitation Learning from Unstructured Demonstrations using Generative Adversarial Nets, K. Hausman et al., NIPS 2017

• InfoGAIL: Interpretable Imitation Learning from Visual Demonstrations, Y. Li et al., NIPS 2017

Hierarchical approaches

• Learning Compound Tasks without Task-specific Knowledge via Imitation and Self-supervised Learning, S. Lee et al., ICML 2020

• CompILE: Compositional Imitation Learning and Execution, T. Kipf et al., ICML 2019

• Directed-Info GAIL: Learning Hierarchical Policies from Unsegmented Demonstrations using Directed Information, M. Sharma et al., ICLR 2019

• Hierarchical Imitation and Reinforcement Learning, H. Le et al., ICML 2018

• OptionGAN: Learning Joint Reward-Policy Options using Generative Adversarial Inverse Reinforcement Learning, P. Henderson et al., AAAI 2018

Learning from human preference

• Safe Imitation Learning via Fast Bayesian Reward Inference from Preferences, D. Brown et al., ICML 2020

• A Low-Cost Ethics Shaping Approach for Designing Reinforcement Learning Agents, Y. Wu et al., AAAI 2018

• Deep Reinforcement Learning from Human Preferences, P. Christiano et al., NIPS 2017

Learning from observations

• Self-Supervised Adversarial Imitation Learning M. Juarez et al., IJCNN 2023

• MobILE: Model-Based Imitation Learning From Observation Alone, Kidambi et al., NeurIPS 2021

• Off-Policy Imitation Learning from Observations, Zhu et al., NeurIPS 2020

• Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreement, C. Yang et al., NeurIPS 2019

• To Follow or not to Follow: Selective Imitation Learning from Observations, Y. Lee et al., CoRL 2019

• Provably Efficient Imitation Learning from Observation Alone, W. Sun et al., ICML 2019

• To follow or not to follow: Selective Imitation Learning from Observations, Y. Lee et al.

• Recent Advances in Imitation Learning from Observation, F. Torabi et al., IJCAI 2019

• Adversarial Imitation Learning from State-only Demonstrations, F. Torabi et al., AAMAS 2019

• Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation, Y. Liu et al., 2018

• Observational Learning by Reinforcement Learning, D. Borsa et al., 2017

Model-based approaches

• Safe end-to-end imitation learning for model predictive control, K. Lee et al., ICRA 2019

• Deep Imitative Models for Flexible Inference, Planning, and Control, N. Rhinehart et al., 2019 [blog]

• Model-based imitation learning from state trajectories, S. Chaudhury et al., 2018

• End-to-End Differentiable Adversarial Imitation Learning, N. Baram et al., ICML 2017

Behavior cloning

• Imitating Unknown Policies via Exploration, G. Nathan et al., BMVC 2020

• Augmented Behavioral Cloning from Observation, M. Juarez et al., IJCNN 2020

• Truly Batch Apprenticeship Learning with Deep Successor Features, D. Lee et al., 2019

• SQIL: Imitation Learning via Regularized Behavioral Cloning, S. Reddy et al., 2019

• Behavioral Cloning from Observation, F. Torabi et al., IJCAI 2018

• Causal Confusion in Imitation Learning, P. Haan et al., NeurIPS 2018

Imitation with rewards

• Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning, A. Gupta et al., CoRL 2019

• Integration of Imitation Learning using GAIL and Reinforcement Learning using Task-achievement Rewards via Probabilistic Generative Model, A. Kinose et al., 2019

• Reinforced Imitation in Heterogeneous Action Space, K. Zolna et al., 2019

• Reinforcement and Imitation Learning for Diverse Visuomotor Skills, Y. Zhu et al., RSS 2018

• Policy Optimization with Demonstrations, B. Kang et al., ICML 2018

• Reinforcement Learning from Imperfect Demonstrations, Y. Gao et al., ICML Workshop 2018

• Pre-training with Non-expert Human Demonstration for Deep Reinforcement Learning, G. Cruz Jr et al., 2018

• Sparse Reward Based Manipulator Motion Planning by Using High Speed Learning from Demonstrations, G. Zuo et al., ROBIO 2018

Multi-agent systems

• Independent Generative Adversarial Self-Imitation Learning in Cooperative Multiagent Systems, X. Hao et al., AAMAS 2019

• PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settings, N. Rhinehart et al., 2019 [blog]

Inverse reinforcement learning

• Intrinsic Reward Driven Imitation Learning via Generative Model, X. Yu et al., ICML 2020

• Inferring Task Goals and Constraints using Bayesian Nonparametric Inverse Reinforcement Learning, D. Park et al., CoRL 2019

• Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observations, D. Brown et al., ICML 2019

• Learning Reward Functions by Integrating Human Demonstrations and Preferences, M. Palan et al., 2019

• Learning Robust Rewards with Adversarial Inverse Reinforcement Learning, J. Fu et al., 2018

• Model-Free Deep Inverse Reinforcement Learning by Logistic Regression, E. Uchibe, 2018

• Compatible Reward Inverse Reinforcement Learning, A. Metelli et al., NIPS 2017

• A Connection Between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models, C. Finn et al., NIPS Workshop 2016

• Maximum Entropy Inverse Reinforcement Learning, B. Ziebart et al., AAAI 2008

POMDP

• Learning Belief Representations for Imitation Learning in POMDPs, T. Gangwani et al., 2019

Planning

• Dyna-AIL : Adversarial Imitation Learning by Planning, V. Saxena et al., 2019

Representation learning

• Visual Adversarial Imitation Learning using Variational Models, R. Rafailov et al., NeuRIPS 2021
• An Empirical Investigation of Representation Learning for Imitation, X. Chen et al., NeuRIPS 2021
• Self-Supervised Disentangled Representation Learning for Third-Person Imitation Learning, J. Shang et al., IROS 2021
• The Surprising Effectiveness of Representation Learning for Visual Imitation, J. Pari et al., 2021
• Provable Representation Learning for Imitation Learning via Bi-level Optimization, S. Arora et al., ICML 2020
• Causal Confusion in Imitation Learning, P. Haan et al, NeuRIPS 2019

Tutorials and talks

• 2018 ICML (Slides)
• Imitation learning basic (National Taiwan University)
• New Frontiers in Imitation Learning (2017)
• Unity Course

Blogs

• Introduction to Imitation Learning

Materials

• Imitation Learning
• CMU Imitation Learning
• Deep Reinforcement Learning via Imitation Learning, S. Levine

Licenses

License

To the extent possible under law, Yueh-Hua Wu has waived all copyright and related or neighboring rights to this work.