The training loss is decreasing while the validation loss is increasing

[WhiteLie98]

Sorry to bother you again. I use the pretrain model of yours and run the train_classifier.py. I visualized the log by tensorboard and found the training loss is decreasing while validation loss is increasing. I wonder if the model is overfitting?

[WhiteLie98]

These are the results of logs.

[IcarusWizard]

Yes, the model is overfitting since cifar10 is a small dataset and we did not perform any data augmentation.

If you wonder why the accuracy is still increasing too, that is because the model is trading accuracy with the cost of perplexity. That is it gets less certain about the right classification, e.g. decrease the probability from 0.9 to 0.6, but get more things right.

[WhiteLie98]

I see. And where should I extend the dataset by using data augmentations? In the pretrain program or fine-tuning program?
p.s. I'm sorry to ask such questions as this was the first CV program I run.

[IcarusWizard]

You can add data augmentations when defining the dataset, i.e. changing the transform from here. Something like RandomResizedCrop should be very helpful. But unfortunately, I do not have the computation resource to verify.