Skip to content

Fix sync gradient#43919

Merged
SunMarc merged 5 commits intomainfrom
sync-gradient
Feb 13, 2026
Merged

Fix sync gradient#43919
SunMarc merged 5 commits intomainfrom
sync-gradient

Conversation

@SunMarc
Copy link
Member

@SunMarc SunMarc commented Feb 11, 2026

What does this PR do?

This PR enables to use sync_each_batch argument when passing gradient_accumulation_kwargs in AcceleratorConfig. I'm also removing adjust_scheduler docstring as it is not used/enabled for now as we don't prepare the scheduler with accelerate in Trainer.

Fixes #43899

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Copy link

@ojh31 ojh31 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the quick fix!

@SunMarc SunMarc merged commit 0dfdd62 into main Feb 13, 2026
26 checks passed
@SunMarc SunMarc deleted the sync-gradient branch February 13, 2026 14:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

sync_each_batch has no effect when using FSDP

5 participants