Cannot reproduce multi-GPU inference time and memory efficiency: I2V-14B, 720P, H100

[FlatterThread]

I tested Wan2.1 I2V-14B model inference on multi H100 GPUs setup, with following parameter:
--ring_size 8
--ulysses_size 1
--offload_model True
inference step: 40
sample size: 720p

In my experiments, H100 x8 multi-GPU inference speed is about 3.5 times faster than H100 x1 single GPU
while H100 x8 multi-GPU inference memory usage (per GPU) is 1.1 times more than H100 x1 single GPU

Could you please share the detailed experiment setup of "Computational Efficiency on Different GPUs" session in README?
I'm seeking for approaches to reduce inference memory usage on multi-GPU setup. Besides, it would be nice to boost inference speed from x4 to x8 on 8 GPU setup as well.

[varadrane1707]

Hi , I too am facing the Memory issue for Multi GPU setup . How did you reduce the GPU memory ? What is the max memory you are getting ?

[anasjaghoub]

how long does it take for 1 single image to video 720p with you?

for me it's taking 40 minutes on 97 frame. NVIDIA A100 8 gpus.

[varadrane1707]

Hi Its slightly less as I was generating only 81 frames.
Did you try using diffusers??

[anasjaghoub]

Yes, I am using diffusers. I'll try it using 81 frames. I use 24 FPS. for 5 seconds video.

[varadrane1707]

Also could you share me the pipeline code you use? Did you use any quantization??

[anasjaghoub]

@varadrane1707 here's the code I use:

`
MODEL_ID = "Wan-AI/Wan2.1-I2V-14B-720P-Diffusers"

vae = AutoencoderKLWan.from_pretrained(MODEL_ID, subfolder="vae", torch_dtype=torch.float32)
image_encoder = CLIPVisionModel.from_pretrained(MODEL_ID, subfolder="image_encoder", torch_dtype=torch.float32)
pipe = WanImageToVideoPipeline.from_pretrained(
MODEL_ID,
vae=vae,
image_encoder=image_encoder,
torch_dtype=torch.bfloat16
).to(device)

pipe.enable_attention_slicing()

start_time = time.time()

    image = Image.open(image_path).convert("RGB")
    image = image.resize((1280, 720), Image.LANCZOS)

    logger.info(f"[{job_id}] Generating 1280x720 on {device}")
    output = pipe(
        image=image,
        prompt=prompt,
        negative_prompt="static, blurry, ugly, watermark, jpeg artifacts, extra fingers",
        num_frames=97,
        guidance_scale=4.0,
        num_inference_steps=30,
        height=720,
        width=1280
    ).frames[0]

    output_path = f"static/videos/{job_id}.mp4"
    export_to_video(output, output_path, fps=24)

    STATUS[job_id] = "done"
    JOB_TIMES[job_id] = round((time.time() - start_time) / 60, 2)

`

Usually it takes 40 minutes plus for a 4-5 seconds video