Use PyTorch schedulers in lr_config?

[alejopaullier96]

Hello, I was wondering if it is possible to use PyTorch schedulers in the lr_config parameter. For example, mmcv supports Cosine Annealing, using mmcv it would be something like this:

• cfg.lr_config = dict(policy='CosineAnnealing', min_lr=1e-6)

However, PyTorch's scheduler has different parameters:

• torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=5, eta_min=1e-4)

How could I use PyTorch's schedulers in the lr_config with their native parameters?

I tried passing PyTorch's parameters as **kwargs but that didnt work. Example:

• cfg.lr_config = dict(policy='CosineAnnealing', min_lr=1e-6, **{"T_max":5})

[ZwwWayne]

The schedulers used in MMDet 2.x have different implementation from those in PyTorch.
But we have unified them in MMDet 3.x, which you might be interested in.
You can try them at https://github.com/open-mmlab/mmdetection/tree/3.x, which uses schedulers in https://mmengine.readthedocs.io/en/latest/api/optim.html#scheduler

[alejopaullier96]

Thanks for your answer. So to be clear, in the 3.x versions mmcv and torch schedulers have the same parameters or mmcv now uses torch schedulers under the hood?

[wangg12]

@ZwwWayne Does it mean we can directly use the schedulers of pytorch? Or only the schedulers in mmengine can be used because they are just have the same APIs as in pytorch?