This repository contains the code relative to the "reproducibility challenge" project of the Reinforcement Learning PhD course offered by Politecnico di Milano. The selected paper for the project is: Generative Adversarial Imitation Learning by Jonathan Ho and Stefano Ermon.
Politecnico di Milano - Reinforcement Learning course information
The code ("main.ipynb") is meant to work on Google Colab (last check: 28th June 2024). The setup of the Jupyter Notebook is subdivided in the following sections:
-
Install packages: it installs the necessary packages (including MuJoCo) and loads the RL_project folder (which you can download from this GitHub repository) from your Google Drive account. A script from the imitation Python library is corrected
-
General functions: definition of Python functions useful throughout the code
The main code is then subdivided in the following sections:
- Creating the environment for both the expert policy and the imitation algorithms: creations of the environments (must run)
- Train the expert algorithm: uses TRPO to train an expert policy and a random policy (optional, if you already have run this, it can be skipped! Load the previous results in the Google Colab file manager and go to number 3.)
- Imitation learning: defines the imitation learning algorithm implemented, i.e., Behavioral Cloning and GAIL. (must run)
- Compare the GAIL and Behavioral Cloning to the expert: train and compare the imitation algorithms to the expert and the random policy (must run)
- Plot curves to visualize the comparison: generates the normalized curves present in the GAIL paper. If you already have the .csv results, you can directly load them in the Google Colab file manager