Caution 1: Code is under active development. Breaking changes are probable.

Caution 2: Documentation is lagging behind the code development.

Digideep

Introduction

Digideep provides a framework for deep reinforcement learning research. Digideep's focus is on code MODULARITY and REUSABILITY.

Specifications:

• Compatible with OpenAI Gym and Deepmind dm_control.
• PyTorch for modeling neural networks.
• Three RL methods already implemented: DDPG, SAC, and PPO.

See documentation at https://digideep.readthedocs.io/en/latest/.

Usage

Installation

Follow instructions.

Running

• Start a training session based on a parameter file. Default parameter files are stored in digideep/params. Example:

# Running PPO on the 'PongNoFrameskip-v4' environment
python -m digideep.main --params digideep.params.atari_ppo

# Run with TensorBoard
python -m digideep.main --params digideep.params.atari_ppo --tensorboard

• Change a parameter in parameter file from command-line:

# Starting PPO training on 'DMCBenchCheetahRun-v0', instead.
python -m digideep.main --params digideep.params.mujoco_ppo --cpanel '{"model_name":"DMCBenchCheetahRun-v0"}'

Any parameter specified in the cpanel section in the parameter file can be altered through command-line.

• Playing a trained policy from a checkpoint. Example:

python -m digideep.main --play --load-checkpoint "<path_to_checkpoint>"

• Visualizing an environment:

python -m digideep.environment.play --model "Pendulum-v0"

• Listing all available environments using a filter. Example:

python -m digideep.environment.play --list-include ".*"

See usage notes for more detailed usage information.

Sample Results

# Running "SAC" on the default "Pendulum" environment:
python -m digideep.main --params digideep.params.sac_params --tensorboard

# Running "PPO" on "PongNoFrameskip-v4" environment:
python -m digideep.main --params digideep.params.atari_ppo --tensorboard

# Running `PPO` on dm_control's `DMBenchCheetahRun-v0` environment:
python -m digideep.main --params digideep.params.mujoco_ppo --cpanel '{"model_name":"DMBenchCheetahRun-v0", "from_module":"digideep.environment.dmc2gym"}' --tensorboard

Learning Graph	Trained Policy

Changelog

• 2019-05-20: Added Soft Actor-Critic (SAC). Added full support for Dict observation spaces.
• 2019-03-04: Digideep was launched.

Contributions

Contributions are welcome. If you would like to contribute to Digideep consider Pull Requests and Issues pages of the project.

Citation

Please use the following BibTeX entry to cite this repository in your publications:

@misc{digideep19,
  author = {Sharif, Mohammadreza},
  title = {Digideep: A DeepRL pipeline for developers},
  year = {2019},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/sharif1093/digideep}},
}

License

BSD 2-clause.

Acknowledgement

I would like to appreciate authors of OpenAI baselines, pytorch-a2c-ppo-acktr, RL-Adventure-2, and RLkit projects.