Skip to content
No description, website, or topics provided.
Python Shell
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
RAA-DuelingDQN
RAA-TD3
learning_curves
poster
LICENSE
README.md

README.md

Regularized Anderson Acceleration for Off-Policy Deep Reinforcement Learning

Regularized Anderson Acceleration (RAA) is a general acceleration framework for off-policy deep reinforcement learning. The algorithm is based on the paper Regularized Anderson Acceleration for Off-Policy Deep Reinforcement Learning presented at NeurIPS 2019.

This implementation uses PyTorch and Python 3.6. Method is tested on MuJoCo continuous control tasks and Atari 2600 in OpenAI Gym v2.

Usage

The paper results can be reproduced by running:

For RAA-DuelingDQN

./RAA-DuelingDQN/run_atari.sh

For RAA-TD3

./RAA-TD3/run_mujoco.sh

Hyper-parameters can be modified with different arguments to main.py. We include an implementation of DuelingDQN (/RAA-DuelingDQN/src/dqn.py) for easy comparison of hyper-parameters with RAA-DuelingDQN and an implementation of TD3 (/RAA-TD3/src/TD3.py) for easy comparison of hyper-parameters with RAA-TD3.

Results

Learning curves found in the paper are found under /learning_curves. Some experimental data and saved models are found under /RAA-DuelingDQN/logs and /RAA-TD3/logs. Numerical results can be found in the paper, or from the learning curves.

Reference

@article{wenjie2019regularized,
  title={Regularized Anderson Acceleration for Off-Policy Deep Reinforcement Learning},
  author={Wenjie Shi, Shiji Song, Hui Wu, Ya-Chu Hsu, Cheng Wu, Gao Huang},
  booktitle={Advances In Neural Information Processing Systems},
  year={2019}
}
You can’t perform that action at this time.