Skip to content

sayantanauddy/acrobotVREP

Repository files navigation

acrobotVREP

An gym environment for acrobot in VREP, along with a sample script for training the acrobot agent using DDPG (deep deterministic policy gradient). Using a similar approach a VREP-based reinforcement learning setup can be created for more complicated control problems, or for other RL algorithms.

alt text

Folders and scripts

  1. acrobotVREP - Python package for acrobotVREP (contains acrobotVrep.py

    1.1 acrobotVREP/env/acrobotVrep.py - Gym class for acrobotVREP

    1.2 acrobotVREP/env/transformations.py - A matrix library by Christoph Gohlke http://www.lfd.uci.edu/~gohlke/

  2. vrep_scenes - VREP scene file

  3. acrobot_train.py - Script for training or testing the acrobot agent

Dependencies

  1. Python 2.7, numpy (not tested on Python 3)
  2. OpenAI Gym (https://gym.openai.com/)
  3. Pypot (https://poppy-project.github.io/pypot/) - provides an easy way of interacting with VREP
  4. Keras (https://keras.io/)
  5. Keras-rl (https://github.com/keras-rl/keras-rl)
  6. VREP simulator (http://www.coppeliarobotics.com/)

Setup

  1. Set the $HOME location to where the repository can be stored:

    ### Change the path as required
    export HOME=/set/as/needed
  2. Clone the repository (or copy it from the disk)

    cd $HOME
    mkdir -p $HOME/computing/repositories/
    cd $HOME/computing/repositories/
    git clone https://github.com/sayantanauddy/acrobotVREP.git
  3. Download VREP

    cd $HOME
    mkdir $HOME/computing/simulators/
    cd $HOME/computing/simulators/
    # Download
    wget http://coppeliarobotics.com/files/V-REP_PRO_EDU_V3_4_0_Linux.tar.gz
    # Extract
    tar -xvf V-REP_PRO_EDU_V3_4_0_Linux.tar.gz
  4. Create the virtual environment

    cd $HOME
    virtualenv --system-site-packages $HOME/acrobot_virtualenv
    # Activate the virtual environment
    source $HOME/acrobot_virtualenv/bin/activate
  5. Add the code location to PYTHONPATH

    export PYTHONPATH=$PYTHONPATH:$HOME/computing/repositories/acrobotVREP
  6. Install the dependencies

    # numpy should also be installed
    pip install pypot
    pip install tensorflow
    pip install tensorflow==1.5
    pip install keras
    pip install keras-rl
    pip install gym
    pip install h5py
  7. Start VREP in a separate terminal

    cd $HOME/computing/simulators/V-REP_PRO_EDU_V3_4_0_Linux
    ./start_vrep.sh
  8. Run acrobot_train.py. Same script can be used for testing also (check comments in the script). VREP shows a prompt when run for the first time. Tick the checkbox in the prompt and close it. This needs to be done 3 times.

    cd $HOME/computing/repositories/acrobotVREP
    python acrobot_train.py

About

A gym environment for acrobot in VREP and a deep-RL training setup using the DDPG algorithm

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages