Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use learning rate finder to determine optimal learning rate #50

Closed
dufaultc opened this issue Jan 11, 2021 · 0 comments
Closed

Use learning rate finder to determine optimal learning rate #50

dufaultc opened this issue Jan 11, 2021 · 0 comments
Assignees

Comments

@dufaultc
Copy link
Contributor

-Use learning_rate_finding notebook to see which learning rate gives lowest learning rate after 2 epochs
-The notebook is under the training-improvments-47 branch
-The only changes you should have to make are in sections 3 and 4 of the notebook, and in the first cell change
"!git clone https://github.com/McMasterAI/Radiology-and-AI.git" to "!git clone --branch training-improvements-47 https://github.com/McMasterAI/Radiology-and-AI.git"
-Find which accumulate_grad_batches amount finds best learning rate (maybe try 10, 15 and 20)
-Check 50 different learning rates for each
-Each should give different optimal learning rate
-Of these three optimal learning rates, run model training with each and see which gives lowest loss value after 2 epochs of training
-You will need to create a wandb account to track the model training and also make a shortcut to the macai datasets folder I shared with you in your base drive folder
-You don't need to push any changes to the repo just report the best learning rate you found and anything else you think important

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants