New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pytorch-Lightning 中的 total steps的变化 #5

Open

rentainhe opened this issue Sep 16, 2021 · 0 comments

Labels

Owner

rentainhe commented Sep 16, 2021 •

edited

Loading

Lightning-AI/pytorch-lightning#9179 (reply in thread)

The global_step indicates how many times the parameters are updated. Therefore, with gradient accumulation and automatic optimization enabled, the global step is only incremented after the gradients are accumulated and not after every training_step is run

Pytorch-Lightning中global steps的变化是根据你参数的更新来决定的，当gradient accumulation和自动优化打开的时候，global step是在每次梯度累积更新后再update的，而不是根据你的training_step

rentainhe added the learning notes label

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment