does diffuer example support batch size option?

[ericlormul]

In plain diffuser, if we make prompt a list, it will batch the input, but got following error in AITemplate. I make prompt a list of size 2.

{'trained_betas'} was not found in config. Values will be initialized to default values.
[18:28:36] ./tmp/CLIPTextModel/model-generated.h:275: Init AITemplate Runtime.
[18:28:37] ./tmp/UNet2DConditionModel/model-generated.h:3262: Init AITemplate Runtime.
[18:28:37] ./tmp/AutoencoderKL/model-generated.h:678: Init AITemplate Runtime.
[18:28:40] ./tmp/CLIPTextModel/model_interface.cu:92: Error: [SetValue] Dimension got value out of bounds; expected value to be in [1, 1], but got 2
Traceback (most recent call last):
  File "examples/05_stable_diffusion/demo.py", line 46, in <module>
    run()
  File "/home/root/miniconda3/envs/ldm/lib/python3.8/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/home/root/miniconda3/envs/ldm/lib/python3.8/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/home/root/miniconda3/envs/ldm/lib/python3.8/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/root/miniconda3/envs/ldm/lib/python3.8/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "examples/05_stable_diffusion/demo.py", line 37, in run
    image = pipe(prompt).images[0]
  File "/home/root/miniconda3/envs/ldm/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/home/root/repos/AITemplate/examples/05_stable_diffusion/pipeline_stable_diffusion_ait.py", line 247, in __call__
    text_embeddings = self.clip_inference(text_input.input_ids.to(self.device))
  File "/home/root/repos/AITemplate/examples/05_stable_diffusion/pipeline_stable_diffusion_ait.py", line 139, in clip_inference
    exe_module.run_with_tensors(inputs, ys, graph_mode=True)
  File "/home/root/miniconda3/envs/ldm/lib/python3.8/site-packages/aitemplate/compiler/model.py", line 483, in run_with_tensors
    outputs_ait = self.run(
  File "/home/root/miniconda3/envs/ldm/lib/python3.8/site-packages/aitemplate/compiler/model.py", line 438, in run
    return self._run_impl(
  File "/home/root/miniconda3/envs/ldm/lib/python3.8/site-packages/aitemplate/compiler/model.py", line 377, in _run_impl
    self.DLL.AITemplateModelContainerRun(
  File "/home/root/miniconda3/envs/ldm/lib/python3.8/site-packages/aitemplate/compiler/model.py", line 192, in _wrapped_func
    raise RuntimeError(f"Error in function: {method.__name__}")
RuntimeError: Error in function: AITemplateModelContainerRun

[antinucleon]

Yes. the batch version is not merged yet. Check here:

https://github.com/terrychenism/AIT_StableDiffusion/tree/main/examples/05_stable_diffusion

[terrychenism]

batched sd: #8

[ericlormul]

Hi, i pulled the latest and commit and re-install and compile everything. but when I change the prompt variable in demo.py to a list of strings. It still gives out the above error. What's the correct way of doing batch size inference using StableDiffusionAITPipeline? Thanks!

[lileilai]

I have meet the same problem, Do you get the solution ？