-
Notifications
You must be signed in to change notification settings - Fork 45
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
The training loss is decreasing while the validation loss is increasing #6
Comments
Yes, the model is overfitting since cifar10 is a small dataset and we did not perform any data augmentation. If you wonder why the accuracy is still increasing too, that is because the model is trading accuracy with the cost of perplexity. That is it gets less certain about the right classification, e.g. decrease the probability from 0.9 to 0.6, but get more things right. |
I see. And where should I extend the dataset by using data augmentations? In the pretrain program or fine-tuning program? |
You can add data augmentations when defining the dataset, i.e. changing the transform from here. Something like |
Sorry to bother you again. I use the pretrain model of yours and run the train_classifier.py. I visualized the log by tensorboard and found the training loss is decreasing while validation loss is increasing. I wonder if the model is overfitting?
The text was updated successfully, but these errors were encountered: