REX_LR learning rate scheduler

Very simple REX learning rate scheduler implementation based on paper “REX: Revisiting Budgeted Training with an Improved Schedule" [1].

Usage

from lr_scheduler import REX_LR

lr_scheduler = REX_LR(optimizer, num_epochs=200, min_val = 0.00001, max_val = 0.0005)

Exampple plot for parameters num_epochs=400,min_val =0.00001,max_val=0.0005

Acknowledgments

Cameron R. Wolfe

Reference

[1] Chen, John, Cameron Wolfe, and Tasos Kyrillidis. “REX: Revisiting Budgeted Training with an Improved Schedule.” Proceedings of Machine Learning and Systems 4 (2022): 64–76.