Robust-RL-Papers

📚Must-read Papers on Robust Reinforcement Learning

---

"Here are some other paper lists you might be interested in:

💡 Reinforcement-Learning-Papers: Related papers for reinforcement learning, including classic papers and latest papers in top conferences.

🔬 Reinforcement-Learning-Papers: List of Top-tier Conference Papers on Reinforcement Learning (RL)，including: NeurIPS, ICML, AAAI, IJCAI, AAMAS, ICLR, ICRA, etc.

We sincerely invite you to dive into these collections of papers and resources, each offering a distinct journey of exploration and discovery. 🥳”

🔔 News

📜Content

• Robust RL Papers
  • Foundation
  • Transition and Reward Robust Design
  • Disturbance Robust Design
  • Action Robust Design
  • Observation Robust Design
  • Generalization to New or Unseen Environments
  • Addressing Uncertainty In Model Parameters
  • Geometric Analysis
  • Maximum Entropy

Robust RL Papers

---

Foundation

1. Robust Reinforcement Learning

   Jun Morimoto, Kenji Doya. PDF, Advances in Neural Information Processing Systems, vol. 13, MIT Press, 2000

2. Robust Markov Decision Processes

   Wolfram Wiesemann, Daniel Kuhn, Berç Rustem. DOI, Mathematics of Operations Research, 38(1):153-183, 2012

3. Robust Markov Decision Processes: Beyond Rectangularity

   Vineet Goyal, Julien Grand-Clément. DOI, Mathematics of Operations Research, 48(1):203-226, 2022.

4. Robust Reinforcement Learning: A Review of Foundations and Recent Advances

   Johannes Moos, Kai Hansel, Hesham Abdulsamad, Sebastian Stark, David Clever, Jan Peters. DOI, Machine Learning and Knowledge Extraction, 4(1):276-315, 2022.

---

Transition and Reward Robust Design(Dealing with distributional shift in the environment dynamics)

1. Policy Gradient for Rectangular Robust Markov Decision Processes

   Navdeep Kumar, Esther Derman, Matthieu Geist, Kfir Y. Levy, Shie Mannor. PDF, Advances in Neural Information Processing Systems, vol. 36, pp. 59477-59501, Curran Associates, Inc., 2023

2. Twice Regularized Markov Decision Processes: The Equivalence between Robustness and Regularization

   Esther Derman, Yevgeniy Men, Matthieu Geist, Shie Mannor. PDF, arXiv:2303.06654 [cs.LG], 2023

3. Beyond Discounted Returns: Robust Markov Decision Processes with Average and Blackwell Optimality

   Julien Grand-Clement, Marek Petrik, Nicolas Vieille. PDF, arXiv:2312.03618 [math.OC], 2023

4. A General Framework for Learning-Based Distributionally Robust MPC of Markov Jump Systems

   Michel Schuurmans, Panagiotis Patrinos. DOI, IEEE Transactions on Automatic Control, vol. 68, issue 5, pp. 2950–2965, Institute of Electrical and Electronics Engineers, 2023

5. Double Pessimism is Provably Efficient for Distributionally Robust Offline Reinforcement Learning: Generic Algorithm and Robust Partial Coverage

   Jose Blanchet, Miao Lu, Tong Zhang, Han Zhong. PDF, arXiv:2305.09659 [cs.LG], 2023

6. Memory Gym: Towards Endless Tasks to Benchmark Memory Capabilities of Agents

   Marco Pleines, Matthias Pallasch, Frank Zimmer, Mike Preuss. PDF, arXiv:2309.17207 [cs.LG], 2024

---

Disturbance Robust Design

1. Characterising the Robustness of Reinforcement Learning for Continuous Control using Disturbance Injection

   Catherine Glossop, Jacopo Panerati, Amrit Krishnan, Zhaocong Yuan, Angela P. Schoellig. PDF, 2022

---

Action Robust Design

1. Tactics of Robust Deep Reinforcement Learning with Randomized Smoothing

   Chung-En Sun, Sicun Gao, Tsui-Wei Weng. PDF, 2024

---

Observation Robust Design

1. Corruption-Robust Algorithms with Uncertainty Weighting for Nonlinear Contextual Bandits and Markov Decision Processes

   Chenlu Ye, Wei Xiong, Quanquan Gu, Tong Zhang. PDF, Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:39834-39863, 2023.

---

Generalization to New or Unseen Environments

1. Learning Invariant Representations for Reinforcement Learning without Reconstruction

   Amy Zhang, Rowan McAllister, Roberto Calandra, Yarin Gal, Sergey Levine. PDF arXiv:2006.10742, 2020

---

Addressing Uncertainty in Model Parameters

1. Robust Reinforcement Learning using Offline Data

   Kishan Panaganti, Zaiyan Xu, Dileep Kalathil, Mohammad Ghavamzadeh. PDF, Advances in Neural Information Processing Systems, 2022.

---

Geometric Analysis

1. The Geometry of Robust Value Functions

   Kaixin Wang, Navdeep Kumar, Kuangqi Zhou, Bryan Hooi, Jiashi Feng, Shie Mannor. PDF, arXiv:2201.12929 [cs.LG], 2022

---

Maximum Entropy

1. Maximum Entropy RL (Provably) Solves Some Robust RL Problems

   Benjamin Eysenbach, Sergey Levine. PDF, arXiv:2103.06257 [cs.LG], 2021