how much gpus did you use?

[Tord-Zhang]

Hi, since the MidasNet is a very large model, how much gpus did you use and how long did it take to train the model? Since the batchsize is not larget ( 8 for each dataset), would multi-gpu training hurt the performance? Since there are a lot of batch normalization layers in the encoder.

Thanks.

[ranftlr]

The model was trained on 2 Quadro RTX 6000s for about 6 days: 1 day for pre-training on RedWeb + about 5 days for training on MIX5. Multi-GPU with low batch size was not an issue, because we froze all batch norm layers.

[Tord-Zhang]

@ranftlr thanks for your quick responce. I am training MidasNet on 2 tesla v100 gpus, but it's much slower. So the mean and val of bn layers is not updated during the training process?

[ranftlr]

Correct, mean and variance are fixed and are not updated. I can't comment on the speed on a v100 as I don't have any available.

[mathmax12]

@Tord-Zhang
I am reproducing the training processing on one V100. I am pretty new to this. I wonder what's the amount of data and how long the training takes?
Thanks