Skip to content
Code for Diagnosing Bottlenecks in Deep Q-learning. Contains implementations of tabular environments plus solvers.
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
debugq Initial upload Mar 9, 2019
docker Initial upload Mar 9, 2019
rlutil Initial upload Mar 9, 2019
scripts Initial upload Mar 9, 2019
.gitignore Initial upload Mar 9, 2019
LICENSE Initial upload Mar 9, 2019
Makefile Initial upload Mar 9, 2019
README.md Update README.md May 14, 2019
requirements.txt
setup.py Initial upload Mar 9, 2019

README.md

diagnosing_qlearning

Diagnosing Q-learning

This repository contains code for Diagnosing Bottlenecks in Deep Q-learning Algorithms by Justin Fu*, Aviral Kumar*, Matthew Soh, Sergey Levine.

This includes:

  • Tabular/discrete environments useful for debugging deep RL algorithms.
  • FQI and Q-iteration solvers.

Colab Notebook

For those who prefer experimenting in Jupyter Notebooks, our algorithm prototyping notebook is available here as a Colab notebook. This notebook contains a gridworld implementation, along with FQI and plotting code.

Setup

Install dependencies

pip install -r requirements.txt
sudo apt-get install python-dev

Compile Cython environments (this must be run from the repo root directory)

make build

Run tests (this must be run from the repo root directory)

make test

Running Experiments

Experiment scripts are located in the scripts folder. Each script runs a sweep over environments, and various hyperparameter settings across multiple seeds.

For example,

python scripts/run_weighted_exact_fqi.py

Plotting

Plotting code is also located in the scripts folder. Each plotting script takes as argument the log directory for one of the experiment scripts

For example

python plot_exact_fqi.py <path-to-exact-fqi-logs>
You can’t perform that action at this time.