open-vocab-pick: Training a student-teacher rearrange policy using VLMs and URDFs

Installation

Note: You must have conda installed. If you want faster installation, you can also use the [conda-libmamba-solver]:

conda update -n base conda

conda install -n base conda-libmamba-solver
conda config --set solver libmamba

1. Create the conda environment:

conda_env_name=clip_grip

# PLEASE VERIFY THAT THE PYTORCH AND CUDA VERSION IS CORRECT FOR YOUR SYSTEM HERE:
# https://pytorch.org/get-started/previous-versions/
conda create -n $conda_env_name python=3.9 -y
conda install -n $conda_env_name \
  habitat-sim=0.2.4 withbullet headless pytorch=1.12.1 cudatoolkit=11.3 \
  -c pytorch -c nvidia -c conda-forge -c aihabitat -y

2. Install this repo and habitat-lab:

# Install this repo
conda activate $conda_env_name
pip install -e .

# Then, install habitat-lab
git clone --branch v0.2.4 git@github.com:facebookresearch/habitat-lab.git
cd habitat-lab
pip install -e habitat-lab
pip install -e habitat-baselines

4. If you are using Spot, its URDF can be downloaded from habitat-sim:

python -m habitat_sim.utils.datasets_download \
  --uids hab_spot_arm \
  --data-path $DATA_DIR