Replies: 1 comment
-
Other than overfitting, there could be a domain shift in your data :) |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi everyone,
As the technology becomes more and more advanced nowadays, there are a variety of hyperparameter automatic search scripts.
I used these scripts to get the model hyperparameters and while the performance was very good on the validation set, it degraded severely on the test set.
In addition, I have set seed and Trainer.deterministic==True, but the performance is still difficult to replicate using the Optimal hyperparameters。
Some common tricks I also used (EarlyStop, lr_shedule)
What is everyone’s opinion on this phenomenon?
Beta Was this translation helpful? Give feedback.
All reactions