This repository contains code for Diagnosing Bottlenecks in Deep Q-learning Algorithms by Justin Fu*, Aviral Kumar*, Matthew Soh, Sergey Levine.
- Tabular/discrete environments useful for debugging deep RL algorithms.
- FQI and Q-iteration solvers.
For those who prefer experimenting in Jupyter Notebooks, our algorithm prototyping notebook is available here as a Colab notebook. This notebook contains a gridworld implementation, along with FQI and plotting code.
pip install -r requirements.txt sudo apt-get install python-dev
Compile Cython environments (this must be run from the repo root directory)
Run tests (this must be run from the repo root directory)
Experiment scripts are located in the
scripts folder. Each script runs a sweep over environments, and various hyperparameter settings across multiple seeds.
Plotting code is also located in the
scripts folder. Each plotting script takes as argument the log directory for one of the experiment scripts
python plot_exact_fqi.py <path-to-exact-fqi-logs>