Star this repository for Toybox release information and updates!
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.
.githooks first pass at public release stuff Nov 23, 2018
ctoybox first pass at public release stuff Nov 23, 2018
pubs first pass at public release stuff Nov 23, 2018
toybox first pass at public release stuff Nov 23, 2018
wasm README Nov 26, 2018
Cargo.toml first pass at public release stuff Nov 23, 2018
LICENSE.txt first pass at public release stuff Nov 23, 2018 Merge pull request #3 from jjfiv/jjfiv-patch-1 Nov 23, 2018

The Reinforcement Learning Toybox

A set of games designed for testing deep RL agents.

This is the public release repository. It contains:

  1. The rust code for building Breakout.
  2. The Python API to the Rust game.
  3. The Python OpenAI Environment for training RL agents.

The current release does not contain the testing framework, nor the tests themselves. We will release the testing framework, tests, and a larger suite of games upon acceptance as an archival, peer-reviewed venue (i.e., conference proceedings or journal publication).

If you use this code, or otherwise are inspired by our white-box testing approach, please cite our NeurIPS workshop paper:

  title={{Toybox: Better Atari Environments for Testing Reinforcement Learning Agents}},
  author={Foley, John and Tosch, Emma and Clary, Kaleigh and Jensen, David},
  booktitle={{NeurIPS 2018 Workshop on Critiquing and Correcting Trends in Machine Learning}},

Setup and Dependencies

Target Rust Version (1.28)

For this project, we are currently target rustc 1.28, here's how to make sure you're using that version (important for contributing) any newer version should also build the code just fine.

rustup override set 1.28.0

Mac Dev Setup Instructions

  • brew install rustup
  • source $HOME/.cargo/env
  • rustup-init with the default install
  • clone this repo

Lints and Formatting in Rust

The best rust tools require the nightly compiler (because they don't want to stabilize the compiler internals yet). Follow the readme instructions to get rustfmt and clippy.

Then you can check automatically format your files with cargo +nightly fmt and peruse the best lints with cargo +nightly clippy.

A pre-commit hook will ensure that your code is always properly formatted. To do this, run

git config core.hooksPath .githooks

from the top-level directory. This will ensure that your files are formatted properly pior to committing.


Tensorflow, OpenAI Gym, OpenCV, and other libraries may or may not break with various Python versions. We have confirmed that the code in this repository will work with the following Python versions:

  • 3.5
  • 3.6


Compile code and dependencies (takes a while the first time) in release mode.

cargo build --release

Developing New Games

Please contact us if you would like to see new games. We also have a slack team.