Skip to content
Lightweight and scalable framework for Reinforcement Learning
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.
apps sync with internal repo Feb 1, 2019
launch sync with internal repo Feb 1, 2019
pytools sync with internal repo Feb 1, 2019
.gitignore sync back to internal repo. Changes: 1) Several more options moved to… Aug 7, 2018


This repository provides ostensibly high-performance c++ implementations of RACER and other popular RL algorithms including PPO, DQN, DPG, ACER, and NAF.
Smarties was largely written during the development of ReF-ER and RACER, and was an honest attempt to maximize code reusability, modularity, and extensibility.
Development should be considered an on-going process, however RACER and DPG were extensively tested and analyzed on the mujoco-based robotic benchmark problems of OpenAI gym and the DeepMind Control Suite.

Smarties requires gcc version 6.1 or greater, a thread-safe (at least MPI_THREAD_SERIALIZED) implementation of MPI, and a serial BLAS implementation with CBLAS interface. Furthermore, in order to test on the benchmark problems, OpenAI gym or the DeepMind Control Suite with python>=3.5.

The makefiles folder is poorly documented and requires some experience with gnu makefiles to navigate. However, compilation on Mac OS should work out of the box (assuming all dependencies are installed).

It should be quite easy for any user to extend this repository by adding their own environment definitions written either with python or c++. Some examples are provided in the apps folder.

The folder launch contains the launch scripts, some description on how to use them, and the description of the output files. Some tools to postprocess the outputs are in the folder pytools.

To cite this repository, reference the paper:

title={Remember and Forget for Experience Replay},
author={Novati, Guido and Koumoutsakos, Petros},
booktitle={Proceedings of the 36\textsuperscript{th} International Conference on Machine Learning},
You can’t perform that action at this time.