PyAgents: a library of DRL algorithms

Agents

Currently, the following algorithms have been implemented:

• REINFORCE: Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning

• A2C: Asynchronous Methods for Deep Reinforcement Learning

• PPO: Proximal Policy Optimization

• DQN: Human level control through deep reinforcement learning (Mnih et al., 2015)

  • Double DQN: Deep Reinforcement Learning with Double Q-learning (van Hasselt et al., 2015)
  • Dueling DQN: Dueling Network Architectures for Deep Reinforcement Learning (Wang et al., 2015)
  • NoisyNets: Noisy Networks for Exploration

• C51: A Distributional Perspective on Reinforcement Learning

• QRDQN: Distributional Reinforcement Learning with Quantile Regression

• DDPG: Continuous control with deep reinforcement learning

• SAC: Soft Actor-Critic Algorithms and Applications

• PER: Prioritized Experience Replay (Schaul et al., 2016)

Usage

A demo script train.py is provided to train the implemented agents on some classical RL environments. Hyperparameters can be customized by using gin configuration files, as described below.

python train.py [-h] [-a AGENT] [-e ENV] [-n NUM_ENVS] [-c CONFIG_DIR] 
               [-o OUTPUT_DIR] [-tv TEST_VER] 
               [-s SEED] [--video | --no-video]

Command line arguments:

Argument	Description
-h, --help	Shows help message
-a AGENT, --agent AGENT	Algorithm to train (see choices above)
-e ENV, --env ENV	Environment to use (see available ones in train.py)
-n NUM_ENVS, --num-envs NUM_ENVS	Number of parallel training envs (vectorized environments)
-c CONFIG_DIR, --config-dir CONFIG_DIR	Path to directory containing .gin config files (see gins for examples)
-o OUTPUT_DIR, --output-dir OUTPUT_DIR	Either directory to save models (training) or directory to load models from (testing)
-tv TEST_VER, --test-ver TEST_VER	If provided, load and test version tv of the agent in directory OUTPUT_DIR
-s SEED, --seed SEED	(default: 42) random seed
--video, --no-video	(default: False) if True, record testing video every now and then