Problems when training

[KrystalCWT]

Hi, I trained M-SFANet with part of shanghaiTech samples. BUT the loss converge too slow, and the mse and mae remain large after training a few hundred epoches. Do you know why?

[Pongpisit-Thanasutives]

If you have employed the Bayesian preprocessing, the convergence could be a bit slow. Sometimes it can take up to 600-800 epochs to converge. I think you can first try the Gaussian filter with fixed std to see the performance. And also, try experimenting with look ahead optimizer like in this paper, https://arxiv.org/abs/1907.08610, to enhance convergence rate.

[phapnm]

Hi, I trained M-SFANet with part of shanghaiTech samples. BUT the loss converge too slow, and the mse and mae remain large after training a few hundred epoches. Do you know why?

How many epochs did you train? And how much MAE value you get (large)?

[Pongpisit-Thanasutives]

@phapnm
Based on my experience, using either the Bayesian preprocessing or Gaussian filter with fixed std, If training up to >700 (700-1000) epochs, the model should converge on SHA (MAE<60) and SHB (MAE<7). To train 1000 epochs, it may take 1-2 days on a single GPU.

I am planning to release the preprocessed dataset of SH and UCF-QNRF for better reproducibility.

Thanks!