HEAT

This repository contains the implementation of the paper "Training Cross-Morphology Embodied AI Agents: From Practical Challenges to Theoretical Foundations". The code is based on modular-rl, with substantial modifications to align with the methods and experiments described in the paper.

Environment Setup

1. Install CUDA 12.2

wget https://developer.download.nvidia.com/compute/cuda/12.2.0/local_installers/cuda_12.2.0_535.54.03_linux.run
sudo sh cuda_12.2.0_535.54.03_linux.run

2. Install cuDNN

wget https://developer.download.nvidia.com/compute/cudnn/9.6.0/local_installers/cudnn-local-repo-ubuntu2204-9.6.0_1.0-1_amd64.deb
sudo dpkg -i cudnn-local-repo-ubuntu2204-9.6.0_1.0-1_amd64.deb
sudo cp /var/cudnn-local-repo-ubuntu2204-9.6.0/cudnn-*-keyring.gpg /usr/share/keyrings/
sudo apt-get update
sudo apt-get -y install cudnn

Setting Up Mujoco

1. Download Mujoco200 linux

cd $HOME
wget https://www.roboti.us/download/mujoco200_linux.zip

2. Download Activation key

cd $HOME
wget https://www.roboti.us/file/mjkey.txt

3. Unzip mujoco200_linux.zip

mkdir $HOME/.mujoco
unzip mujoco200* -d $HOME/.mujoco
mv $HOME/.mujoco/mujoco200_linux $HOME/.mujoco/mujoco200

4. Move license to the bin subdirectory of MuJoCo installation

mv $HOME/mjkey.txt $HOME/.mujoco/.

5. Add system lib path

sudo gedit $HOME/.bashrc

and append the following config

export MUJOCO_PY_MJKEY_PATH=$HOME/.mujoco/mjkey.txt
export LD_LIBRARY_PATH=$HOME/.mujoco/mujoco200/bin:$LD_LIBRARY_PATH
export CFLAGS="-I$HOME/.mujoco/mujoco200/include"
export LDFLAGS="-L$HOME/.mujoco/mujoco200/lib"

including CUDA environment path, the PATH and LD_LIBRARY_PATH should be

export PATH=/usr/local/cuda-12.2/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-12.2/lib64:$HOME/.mujoco/mujoco200/bin:$LD_LIBRARY_PATH
export MUJOCO_PY_MJKEY_PATH=$HOME/.mujoco/mjkey.txt
export CFLAGS="-I$HOME/.mujoco/mujoco200/include"
export LDFLAGS="-L$HOME/.mujoco/mujoco200/lib"

6. Test Installation

source $HOME/.bashrc
cd $HOME/.mujoco/mujoco200/bin
./simulate ../model/humanoid.xml

Clone Project

cd $HOME
git clone https://github.com/airs-admin/heat.git

Install Dependencies

1. Setting up the conda environment

• Install Anaconda

2. Create environment

source $HOME/miniconda3/bin/activate
conda create -c conda-forge python=3.9 -n modular_rl

3. Activate modular_rl

conda activate modular_rl

4. Install Torch 2.5.1 with CUDA 12.1

(modular_rl) pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

5. Install as requirements

(modular_rl) pip install -r requirements.txt

6. Install baselines 0.1.5

(modular_rl) pip install baselines

7. If you encounter the following error such as: “ModuleNotFoundError: No module named ‘lockfile’”, the lockfile module however is included in the requirements.txt list. Please install it using the following command

(modular_rl) pip install lockfile==0.12.2

8. Other issues

• mujoco py install error - fatal error: GL/osmesa.h: No such file or directory

(modular_rl) sudo apt install libosmesa6-dev libgl1-mesa-glx libglfw3

• No such file or directory: 'patchelf' on mujoco-py installation

(modular_rl) sudo apt-get install patchelf

9. To run evaluation

(modular_rl) pip install matplotlib

Increase Swap Memory Size

sudo swapoff /swapfile
sudo fallocate -l 250G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
# Enable permanently
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab

Run

1. Create morphologies

(modular_rl) bash run_create_xmls.sh 

2. The folder organization should be as follows

heat/
└── src/
    └── environments/
        ├── xmls_cheetah_10/
        │   ├── file1.xml
        │   ├── file2.xml
        │   └── ...
        ├── xmls_cheetah_100/
        │   ├── file1.xml
        │   ├── file2.xml
        │   └── ...
        └── xmls_cheetah_1000/
            ├── file1.xml
            ├── file2.xml
            └── ...

3. Run training script

bash run_eval_type_morphologies.sh

4. The custom configurations are as follows

• MORPHLOGIES: specifies the type of morphology (robot or agent) used in the experiment, e.g. cheetah, hopper
• CUSTOM_XMLS: a list of custom XML environment configurations for different experiment setups, e.g. ("xmls_hopper_10"), ("xmls_hopper_10, xmls_hopper_100")
• MAX_NUM_EXP: defines how many models (experiments) should be trained for each XML setting
• MAX_TIMESTAMPS: specifies the maximum number of timesteps for training each model
• PARALLEL_COUNT: specifies the number of experiments that should run simultaneously