A RL package for Torch that can also be used with openai gym
Lua Python CMake
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.


This package is a Reinforcement Learning package written in LUA for Torch. It main features are (for now):

  • Different environments are provided, from classical RL environments, ATARI games, to special ones like the multiclass classification environment that casts a classification learning problem to a RL problem.
    • Classic reward-based policies: Policy gradient, recurrent policy gradient, approximated q-learning with experience replay (also known as deep Q learning)
    • Imitation-based policies: Stochastic gradient-based imitation policy
    • Predictive policies: policies which goal is to predict an output and not to maximize a reward
  • The different policies can be easily used with openAI Gym directly in python by using the lutorpy package


  • 19th of October 2016: OpenAI works again....
  • 19th of August 2016: Major update !
    • Now, environments have been splitted in three components: World, Sensor and Feedback. It allows one to easily specify different sensors, and different problems on the same world. It also greatly increase the readability of the platform
    • Tutorials have been updated
    • WARNING: the client/server classes also have to be updated



  • Torch7
  • nn, dpnn
  • logroll, json, alewrap, sys, paths, tds
luarocks install nn
luarocks install dpnn
luarocks install logroll
luarocks install json
luarocks install sys
luarocks install paths
luarocks install tds
git clone https://github.com/deepmind/xitari.git && cd xitari && luarocks make && cd .. && rm -rf xitari
git clone https://github.com/deepmind/alewrap.git && cd alewrap && luarocks make && cd .. && rm -rf alewrap

For using openAI Gym:

  • openai gym
  • lutorpy


  • cd torch && luarocks make
  • Install lutorpy and OpenAI Gym
  • lauch the python script (example.py)


The package if composed of these different elements:

OpenAI Gym

(To update) The interface with the open AI Gym package is explained Here


The tutorials are avalaible here: Tutorials


  1. When installing Lutorpy, Luajit is not being detected.

Check that pkg-config can find luajit. The following should return at least one result:

pkg-config --list-all | grep luajit

If there are no results, then your .pc file for luajit is probably not in the right place. Try something like the following:

ln -s /path/to/torch/exe/luajit-rocks/luajit-2.0/etc/luajit.pc /usr/local/lib/pkgconfig/luajit.pc
  1. Exception related to no display, such as pyglet.canvas.xlib.NoSuchDisplayException: Cannot connect to "None".

OpenAI Gym needs some sort of display to record results. On ubuntu, you may try to install xvfb/asciinema. Then try running example.py like so:

xvfb-run -s "-screen 0 1400x900x24" python example.py

Author: Ludovic DENOYER -- The code is provided as if, some bugs may exist.....