[Test] SDXL Turbo

[s9roll7]

00.123456.Biomechanical-Style-Girl-Android-Blend-Of-Organic.1.mp4

prompt.json

animatediff generate -c config/prompts/sample_turbo_sdxl_test.json -L 80 -W 768 -H 1280
sd_xl_turbo_lora_v1.safetensors": 0.75
9 steps
generation time 122 sec

[s9roll7]

00.123456.Ethereal-Fantasy-Concept-Art-Of-Sorceress-Casting-.1.mp4

prompt.json

animatediff generate -c config/prompts/sample_turbo_sdxl_lcm.json -L 128 -W 1024 -H 1280
sd_xl_turbo_lora_v1.safetensors": 0.75
9 steps
generation time 269 sec

[tnil25]

Nice!, just curious:

For the hires fix map,

 "0": 0.5, <- Does this always start at 0?
 "0.7": 1.0 <- Is the maximum value here 1? And what does it represent?

[s9roll7]

https://github.com/s9roll7/animatediff-cli-prompt-travel#how-to-use

  "gradual_latent_hires_fix_map":{ # gradual latent hires fix
    # This is an option to address the problem of chaos being generated when the model is generated beyond its proper size.
    # It also has the effect of increasing generation speed.
    "enable": false,    # enable/disable
    "scale": {    # "DENOISE PROGRESS" : LATENT SCALE format
      # In this example, Up to 70% of the total denoise, latent is halved to the specified size.
      # From 70% to the end, calculate the size as specified.
      "0": 0.5,
      "0.7": 1.0
    },
    "reverse_steps": 5,          # Number of reversal steps at latent size switching timing
    "noise_add_count":3          # Additive amount of noise　at latent size switching timing
  },

[tnil25]

https://github.com/s9roll7/animatediff-cli-prompt-travel#how-to-use

  "gradual_latent_hires_fix_map":{ # gradual latent hires fix
    # This is an option to address the problem of chaos being generated when the model is generated beyond its proper size.
    # It also has the effect of increasing generation speed.
    "enable": false,    # enable/disable
    "scale": {    # "DENOISE PROGRESS" : LATENT SCALE format
      # In this example, Up to 70% of the total denoise, latent is halved to the specified size.
      # From 70% to the end, calculate the size as specified.
      "0": 0.5,
      "0.7": 1.0
    },
    "reverse_steps": 5,          # Number of reversal steps at latent size switching timing
    "noise_add_count":3          # Additive amount of noise　at latent size switching timing
  },

Understood. Thanks a lot!

[deeplearn-art]

I suppose controlnet will not work with this?

[s9roll7]

I suppose controlnet will not work with this?

Currently controlnet lllite and gradual_latent_hires_fix cannot be used at the same time
And sdxl turbo must be used with gradual_latent_hires_fix to get decent output.

[aduchon]

I'm trying to use prompt.json posted above in a colab. I got all the installs setup, but get these errors with the config.
I'm not using the maps, so I can make those empty, but it doesn't seem like "lcm" is a valid scheduler.

ValidationError: 6 validation errors for ModelConfig
scheduler
  value is not a valid enumeration member; permitted: 'ddim', 'pndm', 'heun', 'unipc', 'euler', 
'euler_a', 'lms', 'k_lms', 'dpm_2', 'k_dpm_2', 'dpm_2_a', 'k_dpm_2_a', 'dpmpp_2m', 'k_dpmpp_2m', 
'dpmpp_sde', 'k_dpmpp_sde', 'dpmpp_2m_sde', 'k_dpmpp_2m_sde' (type=type_error.enum; 
enum_values=[<DiffusionScheduler.ddim: 'ddim'>, <DiffusionScheduler.pndm: 'pndm'>, 
<DiffusionScheduler.heun: 'heun'>, <DiffusionScheduler.unipc: 'unipc'>, <DiffusionScheduler.euler: 
'euler'>, <DiffusionScheduler.euler_a: 'euler_a'>, <DiffusionScheduler.lms: 'lms'>, 
<DiffusionScheduler.k_lms: 'k_lms'>, <DiffusionScheduler.dpm_2: 'dpm_2'>, 
<DiffusionScheduler.k_dpm_2: 'k_dpm_2'>, <DiffusionScheduler.dpm_2_a: 'dpm_2_a'>, 
<DiffusionScheduler.k_dpm_2_a: 'k_dpm_2_a'>, <DiffusionScheduler.dpmpp_2m: 'dpmpp_2m'>, 
<DiffusionScheduler.k_dpmpp_2m: 'k_dpmpp_2m'>, <DiffusionScheduler.dpmpp_sde: 'dpmpp_sde'>, 
<DiffusionScheduler.k_dpmpp_sde: 'k_dpmpp_sde'>, <DiffusionScheduler.dpmpp_2m_sde: 'dpmpp_2m_sde'>, 
<DiffusionScheduler.k_dpmpp_2m_sde: 'k_dpmpp_2m_sde'>])
gradual_latent_hires_fix_map
  extra fields not permitted (type=value_error.extra)
img2img_map
  extra fields not permitted (type=value_error.extra)
lcm_map
  extra fields not permitted (type=value_error.extra)
region_map
  extra fields not permitted (type=value_error.extra)
unet_batch_size
  extra fields not permitted (type=value_error.extra)