Skip to content


Repository files navigation


An gym environment for acrobot in VREP, along with a sample script for training the acrobot agent using DDPG (deep deterministic policy gradient). Using a similar approach a VREP-based reinforcement learning setup can be created for more complicated control problems, or for other RL algorithms.

alt text

Folders and scripts

  1. acrobotVREP - Python package for acrobotVREP (contains

    1.1 acrobotVREP/env/ - Gym class for acrobotVREP

    1.2 acrobotVREP/env/ - A matrix library by Christoph Gohlke

  2. vrep_scenes - VREP scene file

  3. - Script for training or testing the acrobot agent


  1. Python 2.7, numpy (not tested on Python 3)
  2. OpenAI Gym (
  3. Pypot ( - provides an easy way of interacting with VREP
  4. Keras (
  5. Keras-rl (
  6. VREP simulator (


  1. Set the $HOME location to where the repository can be stored:

    ### Change the path as required
    export HOME=/set/as/needed
  2. Clone the repository (or copy it from the disk)

    cd $HOME
    mkdir -p $HOME/computing/repositories/
    cd $HOME/computing/repositories/
    git clone
  3. Download VREP

    cd $HOME
    mkdir $HOME/computing/simulators/
    cd $HOME/computing/simulators/
    # Download
    # Extract
    tar -xvf V-REP_PRO_EDU_V3_4_0_Linux.tar.gz
  4. Create the virtual environment

    cd $HOME
    virtualenv --system-site-packages $HOME/acrobot_virtualenv
    # Activate the virtual environment
    source $HOME/acrobot_virtualenv/bin/activate
  5. Add the code location to PYTHONPATH

    export PYTHONPATH=$PYTHONPATH:$HOME/computing/repositories/acrobotVREP
  6. Install the dependencies

    # numpy should also be installed
    pip install pypot
    pip install tensorflow
    pip install tensorflow==1.5
    pip install keras
    pip install keras-rl
    pip install gym
    pip install h5py
  7. Start VREP in a separate terminal

    cd $HOME/computing/simulators/V-REP_PRO_EDU_V3_4_0_Linux
  8. Run Same script can be used for testing also (check comments in the script). VREP shows a prompt when run for the first time. Tick the checkbox in the prompt and close it. This needs to be done 3 times.

    cd $HOME/computing/repositories/acrobotVREP


A gym environment for acrobot in VREP and a deep-RL training setup using the DDPG algorithm






No releases published


No packages published