DummyScheduler errors

[Young1993]

scheduler defined in Deepspeed config

scheduler = DummyScheduler(
    optimizer, warmup_num_steps=config["warmup_steps"],
)

model, optimizer, train_dataloader, val_dataloader, scheduler = accelerator.prepare(
    model, optimizer, train_dataloader, val_dataloader, scheduler
)

# setup for saving training states in case preemption
accelerator.register_for_checkpointing(scheduler)

When I run train.py in pycharm, the errors happen for accelerator.register_for_checkpointing(scheduler) as follows:

Traceback (most recent call last):
File "/home/wh/.pycharm_helpers/pydev/_pydevd_bundle/pydevd_exec2.py", line 3, in Exec
exec(exp, global_vars, local_vars)
File "", line 1, in
File "/home/zju/anaconda3/envs/picard/lib/python3.8/site-packages/accelerate/accelerator.py", line 2563, in register_for_checkpointing
raise ValueError(err)
ValueError: All objects must include a state_dict and load_state_dict function to be stored. The following inputs are invalid:
- Item at index 0, DummyScheduler

how to fix it ?

[zanussbaum]

For a quick solution, you can replace it with a real scheduler. This original code was meant to interface with Deepspeed but it doesn't currently work with non-Deepsped. There will be some updates in the coming days to fix this

[dxenos]

I have the same problem :( @zanussbaum what does it mean to replace it with a real scheduler?

[Young1993]

Thanks

[sepilqi]

I have the same problem :( @zanussbaum what does it mean to replace it with a real scheduler?

in deepspeed case,

• set your environment with accelerate config first, and set deepspeed
• pass --deepspeed in your args

will initialize a real scheduler