Skip to content

Plots the change of the loss function of a Keras model when the learning rate is exponentially increasing.

License

Notifications You must be signed in to change notification settings

surmenok/keras_lr_finder

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

15 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

keras_lr_finder

Plots the change of the loss function of a Keras model when the learning rate is exponentially increasing.

Purpose

See details in "Estimating an Optimal Learning Rate For a Deep Neural Network".

Usage

Create and compile a Keras model, then execute this code:

# model is a Keras model
lr_finder = LRFinder(model)

# Train a model with batch size 512 for 5 epochs
# with learning rate growing exponentially from 0.0001 to 1
lr_finder.find(x_train, y_train, start_lr=0.0001, end_lr=1, batch_size=512, epochs=5)
# Plot the loss, ignore 20 batches in the beginning and 5 in the end
lr_finder.plot_loss(n_skip_beginning=20, n_skip_end=5)

Loss function

# Plot rate of change of the loss
# Ignore 20 batches in the beginning and 5 in the end
# Smooth the curve using simple moving average of 20 batches
# Limit the range for y axis to (-0.02, 0.01)
lr_finder.plot_loss_change(sma=20, n_skip_beginning=20, n_skip_end=5, y_lim=(-0.01, 0.01))

Rate of change of the loss function

Contributions

Contributions are welcome. Please, file issues and submit pull requests on GitHub, or contact me directly.

References

This code is based on:

About

Plots the change of the loss function of a Keras model when the learning rate is exponentially increasing.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages