Warmup for Neural Networks

Status: Active (under active development, breaking changes may occur)

This is my personal neural network repository written in PyTorch. Currently nothing fancy, nothing complex. Reinforcement learning side of this repo will be more active in the future.

DL - Deep Learning

Following architectures are implemented in DL package:

• Feed forward network
• Convolutional neural network
• Recurrent neural network

RL - Reinforcement Learning

Following architectures are implemented in RL package:

• Vanilla Policy Gradient (VPG)
• Proximal Policy Optimization (PPO)
• Deep Q Network (DQN)
• World models (WM)

Utilities

• MPI support
• Replay buffer
• Watch agent's internal value estimation as it interacts with env
• Prioritized replay buffer
• Agent comm module

Installation

NN-Warmup requires Python3, PyTorch, OpenAI Gym and OpenMPI. Mujoco physics engine is optional but can be installed with mujoco-py.

Install Python

Install Python using Anaconda:

conda create -n warmup python=3.7
source activate warmup

Install OpenMPI

Ubuntu

sudo apt update && sudo apt install libopenmpi-dev

Mac OS X

brew install openmpi

Install NN-warmup

git clone https://github.com/yigitdemirag/NN-warmup.git
cd NN-warmup
pip install -e .

Check Your Install

To see if you've successfully installed NN-warmup, try running PPO in the OpenAI Gym's LunarLander-v2 environment with:

python -m RL.run ppo --hid "[32,32]" --env LunarLander-v2 --exp_name initialtest --epochs 50

After it finishes training, watch a video of the trained policy with:

python -m RL.run test_policy data/initialtest/initialtest_s0

And plot the results with:

python -m RL.run plot data/initialtest/initialtest_s0

Disclaimer

• The structure of Reinforcement Learning part is highly inspired from Spinning Up in Deep RL of OpenAI.
• MPI support is taken from firedup Repo.