A neural approach to OpenAI Gym tasks using Relative Entropy Policy Search
Clone or download
Pull request Compare This branch is 1 commit ahead of stefanknegt:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
policies
results
run
utils
values
.gitignore
README.md
controller.py
requirements.txt

README.md

Relative Entropy Policy Search

REPS by Peters et al., 2010 is an approach to tackle the optimisation bias and information loss often associated with continuous control Reinforcement Learning tasks by introducing an upper bound by which both the policy and value functions are allowed to change. Extending the approach by van Hoof et al., 2017, this research project by Knegt, Kuric, Müller-Eberstein and Scheffers aims to extend REPS to deep neural network models which perform tasks within OpenAI Gym.

Requirements

This project is written in Python3 and requires the dependencies listed in the requirements.txt file. These include:

Please follow each respective installation guide if necessary.

Running Gym Tasks

Scripts to run gym tasks for Pendulum-v0, Swimmer-v2, HalfCheetah-v2 as well as Reacher-v2, Hopper-v2 you can use the apprpriate scripts in the run/ directory.

$ python3 run/pendulum.py

The models of each iteration are saved as .pth files and the evaluation results should be rendered every two iterations.

Results

Rendered results of the best performing model configurations for Pendulum-v0, Swimmer-v2 and HalfCheetah-v2 can be found on YouTube:

Further plots and result numbers can be found within the results/ directory.

References

Peters, J., Mulling, K., and Altün, Y. Relative entropy policy search. In Fox, M. and Poole, D. (eds.), Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence (AAAI 2010), pp. 1607–1612. AAAI Press, 2010.

van Hoof, H., Neumann, G., and Peters, J. Non-parametric policy search with limited information loss. Journal of Machine Learning Research, 18(73):1–46, 2017.

Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., and Zaremba, W. Openai G ym. CoRR, abs/1606.01540, 2016.