Skip to content

docs: add warmup scheduler mixed-events example#3753

Merged
vfdev-5 merged 3 commits into
pytorch:masterfrom
nightcityblade:fix/issue-2441
Jun 5, 2026
Merged

docs: add warmup scheduler mixed-events example#3753
vfdev-5 merged 3 commits into
pytorch:masterfrom
nightcityblade:fix/issue-2441

Conversation

@nightcityblade

Copy link
Copy Markdown
Contributor

Fixes #2441

Description:

Adds a docstring example showing how to combine Ignite event filters so warm-up runs on iteration starts and the wrapped scheduler runs on later epoch starts.

Check list:

  • New tests are added (if a new feature is added)
  • New doc strings: description and/or example code are in RST format
  • Documentation is updated (if required)

@github-actions github-actions Bot added the module: handlers Core Handlers module label May 12, 2026
Comment thread ignite/handlers/param_scheduler.py Outdated

epoch_length = 8
combined_events = Events.ITERATION_STARTED(
event_filter=lambda engine, event: engine.state.iteration <= 5

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
event_filter=lambda engine, event: engine.state.iteration <= 5
event_filter=lambda engine, event: event <= 5

We already have the count inside event

@TahaZahid05

Copy link
Copy Markdown
Collaborator

@vfdev-5 This PR solves the documentation issue. However, I think I agree with #2441 pushback on maybe this needs an API fix. Currently, we are teaching the user to do the math on when to switch. It would be better if this logic can be internal to improve user experience.

Something like:

scheduler = create_lr_scheduler_with_warmup(
    torch_lr_scheduler,
    warmup_duration=5,
    warmup_event=Events.ITERATION_STARTED,
    post_warmup_event=Events.EPOCH_STARTED
)
# The scheduler would internally handle the event-switching logic.
trainer.add_event_handler(Events.ITERATION_STARTED | Events.EPOCH_STARTED, scheduler)

@nightcityblade

Copy link
Copy Markdown
Contributor Author

Thanks for calling that out. I agree the more ergonomic long-term answer is probably an API improvement so users do not need to manually compute the switch point. I kept this PR narrowly scoped to the current docs issue because it only documents the behavior that exists today, but I’m happy for maintainers to treat the API direction separately if that’s preferred.

Comment thread ignite/handlers/param_scheduler.py Outdated
optimizer = SGD([torch.zeros(1, requires_grad=True)], lr=0.1)
torch_lr_scheduler = ExponentialLR(optimizer, gamma=0.5)
scheduler = create_lr_scheduler_with_warmup(
torch_lr_scheduler, warmup_start_value=0.0, warmup_duration=5

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's first define the variable warmup_duration=5 and then use it rather than the raw values themselves.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review — updated this to keep the change docs-only, define warmup_duration = 5, and reuse it in the scheduler and event filters.

@TahaZahid05

TahaZahid05 commented Jun 4, 2026

Copy link
Copy Markdown
Collaborator

@nightcityblade let us keep this PR to update the docs for now. if you feel that this needs an API change in the future you can also give your reasoning on the original issue and the maintainer can decide if we need it or not. Kindly update the PR with the changes requested.

@nightcityblade

Copy link
Copy Markdown
Contributor Author

Thanks again. I re-checked the open review threads and the current branch keeps this docs-only while addressing the requested changes: it defines warmup_duration once, reuses it in the scheduler and event filters, and uses the filtered event counter directly for the warm-up condition.\n\nValidation: ran the documented snippet locally against this branch and it completes successfully.

@vfdev-5 vfdev-5 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me
@TahaZahid05 I let you approve and merge it if good to you as well

@vfdev-5
vfdev-5 enabled auto-merge June 5, 2026 16:16
@vfdev-5
vfdev-5 added this pull request to the merge queue Jun 5, 2026
Merged via the queue into pytorch:master with commit 234bca2 Jun 5, 2026
23 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

module: handlers Core Handlers module

Projects

None yet

Development

Successfully merging this pull request may close these issues.

create_lr_scheduler_with_warmup does not change init_lr to proper value

3 participants