Skip to content
Tools for effectively storing and sampling from episodes for RL experiments.
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.


spacelib is a small collection of tools for off-policy reinforcement learning, particularly with recurrent agents and particularly for learning directly from pixel observations.

The package makes pretty heavy use of pytorch tensors and the openai gym APIs.


  • efficiently store completed episodes on disk in memory-mapped numpy arrays
  • transparently handle hierarchically structured action/observation spaces
  • simple interface for batch sampling of sequences from experience history
  • store hidden states for recurrent models

For a usage example, see the example notebook.


  • stop one-hot encoding discrete spaces
  • test/add an example for hidden state storage

This project is licensed under the terms of the MIT license Please let me know if you find it useful!

You can’t perform that action at this time.