Skip to content

chaning learning rate back to default of 1e-3#156

Merged
sfluegel05 merged 3 commits intodevfrom
fix/default_learning_rate
Mar 18, 2026
Merged

chaning learning rate back to default of 1e-3#156
sfluegel05 merged 3 commits intodevfrom
fix/default_learning_rate

Conversation

@schnamo
Copy link
Collaborator

@schnamo schnamo commented Feb 18, 2026

Changes
For model config files the default is now set to 1e-3.

Solves issue 150

@sfluegel05
Copy link
Collaborator

Thanks for fixing this. I took the chance to delete a redundant pretraining config and fix some other parameters. Also, there are 5 toxicity electra configs. You have changed one learning rate to 1e-3, the others are 1e-5. Is there a reason for (a) having 5 different toxicity electra configs (but none for other OPT finetuning tasks) and (b) for having different learning rates?

@schnamo
Copy link
Collaborator Author

schnamo commented Mar 18, 2026

They are used for different fine-tuning tasks (classification vs regression) and using different parameters, however, have deleted some of them. The remaining should stay.

@sfluegel05 sfluegel05 merged commit f910a39 into dev Mar 18, 2026
4 checks passed
@schnamo schnamo deleted the fix/default_learning_rate branch March 18, 2026 13:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants