Skip to content


Repository files navigation

Variational Automatic Curriculum Learning

Jiayu Chen, Yuanxin Zhang, Yuanfan Xu, Huimin Ma, Huazhong Yang, Jiaming Song, Yu Wang, Yi Wu.


This repository implements a curriculum learning algorithm, Variational Automatic Curriculum Learning (VACL), for solving challenging goal-conditioned cooperative multi-agent reinforcement learning problems. The implementation in this repositorory is used in the paper "Variational Automatic Curriculum Learning for Sparse-Reward Cooperative Multi-Agent Problems" ( This repository is heavily based on


test on CUDA == 10.0

git clone
cd ~/Variational-Automatic-Curriculum-Learning
conda create -n VACL python==3.6.2
conda activate VACL
pip install torch==1.5.1+cu101 torchvision==0.6.1+cu101 -f
pip install -r requirements.txt

2. Train MPE

   conda activate VACL
   cd scripts
   # run mpe without entity progression
   # run mpe with entity progression

Cooperative scenarios:

  • simple_spread
  • push_ball
  • hard_spread

3. Train Hide-and-Seek

3.1 Install MuJoCo

  1. Obtain a 30-day free trial on the MuJoCo website or free license if you are a student.

  2. Download the MuJoCo version 2.0 binaries for Linux.

  3. Unzip the downloaded directory into ~/.mujoco/mujoco200, and place your license key at ~/.mujoco/mjkey.txt.

  4. Add this to your .bashrc and source your .bashrc.

   export LD_LIBRARY_PATH=~/.mujoco/mujoco200/bin${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
   export MUJOCO_KEY_PATH=~/.mujoco${MUJOCO_KEY_PATH}

3.2 Intsall mujoco-py and mujoco-worldgen

  1. You can install mujoco-py by running pip install mujoco-py== If you encounter some bugs, refer this official repo for help.

    sudo apt-get install libgl1-mesa-dev libosmesa6-dev
  2. To install mujoco-worldgen, follow these steps:

    # install mujuco_worldgen
    cd envs/hns/mujoco-worldgen/
    pip install -e .
    pip install xmltodict
    # if encounter enum error, excute uninstall
    pip uninstall enum34

3.3 Train Tasks

   conda activate VACL
   cd scripts
   # box locking task
   # hide and seek task

4. Publication

If you find this repository useful, please cite our paper:

      title={Variational Automatic Curriculum Learning for Sparse-Reward Cooperative Multi-Agent Problems}, 
      author={Jiayu Chen and Yuanxin Zhang and Yuanfan Xu and Huimin Ma and Huazhong Yang and Jiaming Song and Yu Wang and Yi Wu},


No releases published