You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently it's not supported. However in huggingface/accelerate, users can specify DummyOptimizer and DummyScheduler to indicate they want them to be replaced by those from deepspeed config.
Pitch
Lightning should have some way to let people use optimizers from deepspeed config, like OneBitAdam or something else that cannot be manually instantiated.
Description & Motivation
Currently it's not supported. However in huggingface/accelerate, users can specify
DummyOptimizer
andDummyScheduler
to indicate they want them to be replaced by those from deepspeed config.Pitch
Lightning should have some way to let people use optimizers from deepspeed config, like
OneBitAdam
or something else that cannot be manually instantiated.pytorch-lightning/src/lightning/fabric/strategies/deepspeed.py
Line 330 in 90d04b5
Alternatives
No response
Additional context
No response
cc @Borda
The text was updated successfully, but these errors were encountered: