Hunyuan Video adjustments

[Ednaordinary]

What does this PR do?

Two things

The STG Hunyuan Video community pipeline had a typo where the forward methods were being replaced within the denoising loop. This also caused callback_on_step_end to return with the last index in the stg index list since the variable used to iterate over the forward methods was also i. This only seemed to appear in the HunyuanVideo stg pipeline but I may have missed something.

UniPCMultistepScheduler(prediction_type="flow_prediction", use_flow_sigmas=True, flow_shift=shift) works for HunyuanVideo but errors out with the current behavior. This changes that to instead just warn the user (there may be a cleaner way of doing this)

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline?
• Did you read our philosophy doc (important for complex PRs)?
• Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

@yiyixuxu @asomoza

[hlky]

Hi @Ednaordinary. I've left some comments. Ideally we would support sigmas in UniPCMultistepScheduler's set_timesteps as retrieve_timesteps is a # Copied from function and it is preferable to keep it that way. Some models/pipelines require a specific sigma schedule that is passed from pipeline, so prediction_type/use_flow_sigmas/defaulting to passing num_inference_steps may work for some cases but potentially not all.
I will let @yiyixuxu comment further on that change, and we will need to update it everywhere if we go ahead with it (this process is easy though, we would simply update diffusers.pipelines.stable_diffusion.pipeline_stable_diffusion.retrieve_timesteps then run make fix-copies).

[hlky]

Note forward_without_stg vs forward_with_stg. Let's use something like stg_idx here so it doesn't conflict with index of enumerate(timesteps).

However, any results you have to share using this PR would be interesting, as it is using forward_with_stg for both noise_pred and noise_pred_perturb.

[Ednaordinary]

Ah, thanks for the catch, I'll update. I found that this implementation became scheduler agnostic (or stg became the 'scheduler'?). I'll test that a bit more and see what the exact side effects are

[Ednaordinary]

Hi @hlky! Thanks for your review. I agree this change could cause some weirdness with sigmas. I don't believe this would work with all pipelines since hunyuanvideo passes num_inference_steps and sigmas which retrieve_timesteps states not to do (but allows us to ignore sigmas if the scheduler does not accept them). A possible cleaner change is to leave retrieve_timesteps how it is and instead introduce a ignore_custom_sigmas value in the call function. Then, we could do:

if ignore_custom_sigmas:
    timesteps, num_inference_steps = retrieve_timesteps(self.scheduler, device=device, num_inference_steps=num_inference_steps)
else:
    sigmas = np.linspace(1.0, 0.0, num_inference_steps + 1)[:-1] if sigmas is None else sigmas
    timesteps, num_inference_steps = retrieve_timesteps(self.scheduler, device=device, sigmas=sigmas)

This would have to be found by the user if they try to use UniPC or a non-accepted scheduler as a scheduler though. Not sure what a clean way to add a hint if that happens is.

[Ednaordinary]

5 steps, bnb nf4 w/ some modules skipped, para-attn @ 0.09, guidance 6

	Euler	UniPC
Current	https://github.com/user-attachments/assets/9b4756ca-c943-45c7-a8d6-b2fa2a68545a	https://github.com/user-attachments/assets/6f7d01d2-3677-4559-bed2-d520a842ea2b
Perturb (#11140 (comment))	https://github.com/user-attachments/assets/55900315-56d0-4267-a29f-1532bda0798f	https://github.com/user-attachments/assets/80d6d551-3601-4536-af60-cdc1e64f5c1f