Project_RL

This is a group project for the Reinforcement learning course offered in M.Sc Artificial Intelligence at the University of Amsterdam.

Contributors

• David Speck
• Masoumeh Bakhtiariziabari
• Ruth Wijma
• Victor Zuanazzi

Dependencies

This project uses python 3. To install dependencies run:

pip install -r requirements.txt

Problem Statement

There are different types of experience replay, e.g. prioritized experience replay and hindsight experience replay. Compare two or more types of experience replay. Does the ‘winner’ depend on the type of environment?

Experience Replays

We mainly experimented with three experience replays techniques which are:

- Naive Experience Replay 
- Prioritized Experience Replay
    + rank base
    + proportion base
- Combined Experience Replay
- Adaptive Experience Replay 
  + Adaptive ER
  + Adaptive CER
  + Adaptive rank PER

Hyperparameters

hyperparameters were chosen based on referenced papers

Parameter	Value
Fixed Buffer size (Number of frames) [1]	10^4
Batch Size	64
Learning rate [1]	5e-4
max_step (Maximum episode duration) [1]	1000
(Priority) [2]	0.6
(IS) [2]	0.4 -> 1
(Discount Factor)	0.8 for Catpole 0.99 for MountainCar and LunarLander
(Softupdate interpolation parameter)	0.001 for Catpole 0.1 for MountainCar and LunarLander

• Code
• Poster presentation