Introduction

This repository provides typical and pupolar reinforcement learning algorithms using Python and Pytorch, including Q-learning, Sarsa, DQN (Double-DQN,Dueling-DQN, DQN with Prioritized Experience Replay), DDPG and TD3.

Some simple environments are provided to test the performance of RL algorithms which are as follow:

• maze and tic-tac for testing Q-learning and Sarsa
• pendulum for testing DQN
• robot for testing DDPG and TD3

Prerequisites

python 3.6
Pytorch 1.0
gym

Implementation

• Q-learning and Sarsa are implemented as described in Reinforcement Learning: An Introduction (Chapter6, P154-P157)

• DQN and its advanced modification are implemented based on paper: (The neural networks approximator is modified into a more simple structure)

  • DQN: Playing Atari with Deep Reinforcement Learning
  • DQN with Prioritized Experience Replay:Prioritized Experience Replay
  • Dueling-DQN: Dueling Network Architectures for Deep Reinforcement Learning
  • Double-DQN: Deep reinforcement learning with double Q-Learning

• DDPG and TD3 are implemented strictly based on:

  • DDPG: Continuous control with deep reinforcement learning
  • TD3: Addressing Function Approximation Error in Actor-Critic Methods

Usage

• maze: python3 run_maze_RL.py --algorithm Q-learning # or Sarsa
• tic-tac: python3 run_tictac_RL.py --algorithm Q-learning # or Sarsa
• pendulum: python3 run_pendulum_DRL.py --double True --prioritized True --dueling True # True or False to select DQN algorithms
• robot: python3 run_robot_DRL.py --algorithm TD3 # or DDPG

Acknowledgment

Some environments used in this repository are referenced from 莫烦 Python.