To add Chained Scheduler to the list of PyTorch schedulers.

[iramazanli]

In this PR we are introducing ChainedScheduler which initially proposed in the discussion #26423 (comment) .

The idea is to provide a user friendly chaining method for schedulers, especially for the cases many of them are involved and we want to have a clean and easy to read interface for schedulers. This method will be even more crucial once CompositeSchedulers and Schedulers for different type of parameters are involved.

The immediate application of Chained Scheduler is expected to happen in TorchVision Library to combine WarmUpLR and MultiStepLR https://github.com/pytorch/vision/blob/master/references/video_classification/scheduler.py#L5 . However, it can be expected that in many other use cases also this method could be applied.

Example

The usage is as simple as below:

sched=ChainedScheduler([ExponentialLR(self.opt, gamma=0.9),
                        WarmUpLR(self.opt, warmup_factor=0.2, warmup_iters=4, warmup_method="constant"),
                        StepLR(self.opt, gamma=0.1, step_size=3)])

Then calling

sched.step()

would trigger step function for all three schedulers consecutively

Partially resolves pytorch/vision#4281

[facebook-github-bot]

🔗 Helpful links

• 🧪 See artifacts and rendered test results at hud.pytorch.org/pr/63491
• 📄 Preview docs built from this PR

💊 CI failures summary and remediations

As of commit 8eaf9f9 (more details on the Dr. CI page):

---

💚 💚 Looks good so far! There are no failures yet. 💚 💚

---

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

[datumbox]

Thanks for the fast PR. I left a couple of comments, please let me know what you think.

[datumbox]

LGTM, thanks for your PR @iramazanli.

I know that the name of the scheduler might change but other than that the code looks fine to me. Feel free to merge whenever you are happy.

[facebook-github-bot]

@iramazanli has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

[facebook-github-bot]

@iramazanli has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

[codecov]

Codecov Report

Merging #63491 (8eaf9f9) into master (a6f767e) will decrease coverage by 0.07%.
The diff coverage is 88.88%.

@@            Coverage Diff             @@
##           master   #63491      +/-   ##
==========================================
- Coverage   66.84%   66.77%   -0.08%     
==========================================
  Files         695      695              
  Lines       90736    90768      +32     
==========================================
- Hits        60656    60608      -48     
- Misses      30080    30160      +80     

[facebook-github-bot]

@iramazanli merged this pull request in 5a12cb6.