Skip to content
Development of a virtual quadruped robot using OpenAI & Mujoco
Python
Branch: master
Clone or download
Ben Nortier
Ben Nortier add v5 and v6
Latest commit b2e8b42 Feb 1, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
src add v5 and v6 Feb 1, 2018
.gitignore add model debug code Jan 31, 2018
CHANGELOG.md Updated README, LICENSE etc. Jan 11, 2018
LICENSE Updated README, LICENSE etc. Jan 11, 2018
NOTES.md add NOTES Jan 31, 2018
README.md Updated README, LICENSE etc. Jan 11, 2018
model_debug_requirements.txt
requirements.txt Cheetah v2 Jan 10, 2018
train_requirements.txt add model debug code Jan 31, 2018

README.md

Skilos

Skilos is a project to create a virtual quadruped similar to the homicidal robot dog in Black Mirror (S4:E5, Netflix), minus the psychopathic murdering tendencies.

Metalhead Dog

It is also a learning project for me to get to know OpenAI, TensorFlow, Reinforcement learning, and AI in general.

Approach

The idea of using simulated environments to train real-world robotics is something I've been curious about for quite a while. I've done experiments before with XPlane to create control systems using Genetic Algorithms, and I've had this idea to create biologically-inspired robots using this method. With the rise of modern AI and Deep Learning, I feel the time has come to explore this further.

I'm using OpenAI Gym as a starting point, together with Tensorflow. There is a very useful environment called HalfCheetah, which is a model of... half a cheetah 😀:

Half Cheetah

I has 6 actuators, 3 for each leg (thigh, shin, foot), modelled as Mujoco motors. The control of these motors will be trained to make the Cheetah cover the maximum distance possible, with some changes (see CHANGELOG.md)

Pat Coady submitted the highest-ranking solution to this problem, so I'm using his generously open sourced (MIT) code to begin with.

You can read more about his efforts on his blog:

https://learningai.io/projects/2017/07/28/ai-gym-workout.html

And find his code for TRPO here:

https://github.com/pat-coady/trpo

Do-it-yourself

I use pyenv and venv for virtual python environments. YMMV.

Requirements

  • Python 3.5.x
  • Mujoco 1.3.0

Create python virtual environment and install dependencies

$ python --version
Python 3.5.4
$ python -m venv env
$ source env/bin/activate
$ pip install -r requirements.txt

Run the experiments

$ python ./src/train.py HalfCheetah-v1 -n 30000 -b 20
$ python ./src/train.py HalfCheetah-v2 -n 30000 -b 20
$ python ./src/train.py FullCheetah-v1 -n 30000 -b 20

Please log an issue on Github if you're trying to replicate this but aren't successful.

You can’t perform that action at this time.