Required training time

[adamjstewart]

In your paper, you mention that the model is pre-trained on 80 V100s. How long was the model trained for? I'm working on a review paper and would like an estimate of training time.

[tung-nd]

Hi,

Thank you for your interest in ClimaX. I do not remember the training time exactly, but it was definitely less than 1 day (24 hours).

[adamjstewart]

Thanks, I'll use 24 hrs as an estimate for now.

[rejuvyesh]

Hi @adamjstewart! Small correction. I would put the training time to at least 3 days. The initial CMIP6 pretraining phase was about 200k steps. Then finetuning on ERA5 was done for about 100k steps which is a fair amount of compute time. There is some overhead in terms of time for computing various validation metrics at some frequency as well to make sure training is progressing as expected.

There have been quite a few performance fixes since the initial training and overall it should be much faster to train now.