Skip to content

kwea123/hindsight_experience_replay

Repository files navigation

Hindsight Experience Replay (HER)

This repository contains a tensorflow HER implementation and a bit flipping environment as described in OpenAI's paper

The implementation includes :

  1. In Hindsight Experience Replay.ipynb :
    1. A DQN and a DDQN agent (which also work on other traditional gym environments)
    2. A bit flipping environment
    3. Pre-trained models for 30-bits, 40-bits and 50-bits flipping environments
  2. In ChaseEnv_DDPG.ipynb :
    1. A DDPG agent
    2. A ChaseEnv environment, where a chaser is initialized at a random position in a 2d plane and has to reach a goal in another random position within a certain threshold.

Benchmarks

  • 100% success rate for 30 and 40-bits environments
  • 95% success rate for 50-bits environment (average on 100 tests)
  • 90% success rate for size=5 ChaseEnv (average on 100 tests)

Customize

Check the "Training" cell to adjust training parameters and enable/disable HER.

TODO

  • Optimize the way to concatenate transitions
  • Parallelize training
  • Train on bit length > 30
  • Implement DDPG

Extra

Here is a link to a robot arm reach environment created in Unity, trained with ML-Agents.

This environment is trained using DDPG with and without HER, and the comparison is plotted. DDPG+HER performs better.