fix get_scheduler when name is warmup_stable_decay

[zspo]

What does this PR do?

fix #31085

I also add some cases for test.

@ArthurZucker and @amyeroberts

[amyeroberts]

Thanks for fixing!

LGTM, but would be good to have a second look from @SunMarc to confirm this is as expected

[SunMarc]

Make sense ! Thanks for fixing @zspo !

[MostHumble]

@SunMarc this means that you'd have to pass, num_warmup_steps=num_warmup_steps, AND scheduler_specific_kwargs = {num_stable_steps, num_decay_steps, num_cycles, min_lr_ratio} as they're required params, this can be confusing as all of them are specific to that scheduler, I'm not sure why scheduler_specific_kwargs exists

[zspo]

@SunMarc this means that you'd have to pass, num_warmup_steps=num_warmup_steps, AND scheduler_specific_kwargs = {num_stable_steps, num_decay_steps, num_cycles, min_lr_ratio} as they're required params, this can be confusing as all of them are specific to that scheduler, I'm not sure why scheduler_specific_kwargs exists

I considered whether to put num_warmup_steps in scheduler_specific_kwargs. But this parameter appears in the function get_scheduler explicitly.

[SunMarc]

scheduler_specific_kwargs exists so that we don't have a get_scheduler with too many args + that are specific to only a few schedulers.

[MostHumble]

I see so it's juste two different styles clashing, thanks.