-
Install Matterport3D simulator.
Put
connectivity
in./
(or create a soft link). Putbuild
and Matterport3D datav1
in the./data
direcotry(or create a soft link). -
Install python requirements.
timm==0.9.5 transformers==4.28.1 torch==1.13.0, do not use >=2.0 numpy pandas matplotlib python-opencv tqdm pyyaml networkx jsonlines
-
Download datasets, image features and model checkpoints from here. Download the
data.zip
andlog.zip
and unzip them.
Finally, the directory layout should looks like:
.
├── connectivity
├── data
│ ├── build
│ ├── candidate_buffer.pkl
│ ├── img_features
│ ├── pretrain_data
│ ├── pretrained_model
│ ├── R2R
│ ├── RxR
│ └── v1
├── log
│ └── commit
├── pretrain.sh
├── README.md
├── run.sh
└── src
Pretrain:
sh pretrain.sh R2R
# sh pretrain.sh RxR
Note that even though multi-process training is implemented, I never use it. Therefore, there may be some bugs.
Fine-tune:
sh run.sh R2R
# sh run.sh RxR