Skip to content
Switch branches/tags

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

Learning to Control Self-Assembling Morphologies

NeurIPS 2019 (Spotlight)
Winner of Virtual Creatures Competition at GECCO 2019, Prague

[Project Website] [Demo Video]

Deepak Pathak*, Chris Lu*, Trevor Darrell, Phillip Isola, Alexei A. Efros
University of California, Berkeley
(* equal contribution)

This is a pytorch based implementation for our paper on learning to control self-assembling agents using deep reinforcement learning. We investigate a modular co-evolution strategy: a collection of primitive agents learns to dynamically self-assemble into composite bodies while also learning to coordinate their behavior to control these bodies. We learn compositional policies to demonstrate better zero-shot generalization. If you find this work useful in your research, please cite:

    Author = {Pathak, Deepak and Lu, Chris and Darrell, Trevor and
              Isola, Phillip and Efros, Alexei A.},
    Title = {Learning to Control Self-Assembling Morphologies:
              A Study of Generalization via Modularity},
    Booktitle = {arXiv preprint arXiv:1902.05546},
    Year = {2019}

Installation and Usage

  1. Setting up repository
git clone
cd modular-assemblies/
git clone
cd ml-agents/
git reset --hard 6c5255e
cd ..
bash envs/

python3 -m venv assemblyEnv
source $PWD/assemblyEnv/bin/activate
pip install --upgrade pip
  1. Installation

    • Requirements:
      • CUDNN-5.1, CUDA-8.0, Python-3.5
    • Detailed setup, skip to quick setup for exact replication:
    # Install Pytorch from
    pip install
    pip install torchvision
    pip install --upgrade visdom
    # Install baselines for Atari preprocessing
    pip install gym==0.9.4 # baselines install latest gym first automatically, but latest gym has moved to mujoco5 so first install old gym and then install baselines
    git clone
    cd baselines
    git reset --hard b5be53d
    pip install -e .
    # Additional packages
    pip install numpy
    pip install matplotlib
    pip install pillow
    pip install opencv-python
    # fold
    cd modular-assemblies/src/
    git clone
    cd pytorch-tools/
    git reset --hard 09dccb2
    python install
    • Quick setup for exact replication:
    pip install -r requirements.txt
  2. Run code

cd modular-assemblies/src/


Builds upon Ilya Kostrikov's Pytorch PPO implementation.


[NeurIPS 2019] Code for the paper "Learning to Control Self-Assembling Morphologies: A Study of Generalization via Modularity"





No releases published


No packages published