Quick question about training time and compute

[greeneggsandyaml]

Hello, thanks for your great work. I have a very question about training.

I'm trying to run training and getting an OutOfMemoryError using a (single) 32 GB GPU (V100). What do you use for training? Also, with your compute setup, approximately how long does training take?

Thanks so much!

[shariqfarooq123]

Hi, thanks for appreciating our work!

For metric fine-tuning, We use 4 NVIDIA A100 GPUs for training our largest model (BEiT-L). Training time on NYU (~25k samples, 5 epochs) on 4 A100s (40GB) is less than 2 hours.

Relative pre-training on 12 datasets (M12 from the paper) takes around 3-5 days on 8 RTX A6000-like GPUs. This gives us the MiDaS v3.1 models. Please refer to midas repo for more details.

[greeneggsandyaml]

Brilliant, thanks for the quick response. It's great to hear that the training time is quick.

I'll create another issue if I have any more questions, but this is resolved, so I'm closing the issue.