MiniMax H3 video VAE: long-clip decode OOMs are unrecoverable — tiled retry is a no-op and dynamic VRAM never evicts for the decode

[xsourced]

Summary

On a 16GB card, MiniMax H3 video generation above ~209 frames reliably completes sampling and then dies in VAEDecode, after paying the full multi-minute sampling cost. Two independent problems combine to make this unrecoverable:

1. The tiled-decode fallback is a no-op for this VAE. MiniMaxH3VideoVAE.decode_tiled() just calls decode() (comfy/ldm/minimax/vae.py, the model tiles internally and the VAE sets handles_tiling = True), so the "Ran out of memory when regular VAE decoding, retrying with tiled VAE decoding" path in comfy/sd.py re-runs the identical decode and fails identically.
2. Under dynamic VRAM, nothing is evicted for the decode. At the point of failure ~14GB of dynamically streamed weights (DiT + text encoder) sit outside the torch allocator, load_models_gpu for the VAE reports 0 models unloaded, and the decode's ~1.1–1.2GiB peak torch allocations fail with ~0.8GiB free. Raising the VAE's memory_used_decode estimate has no effect — the declared requirement is never converted into eviction on the dynamic path. Free VRAM at failure was also below --reserve-vram 1.5, so the reserve does not bound streamed residency either.

Environment

• ComfyUI v0.30.2 (dec5d945), torch 2.12.1 cu130, PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
• RTX Pro 2000 Blackwell 16GB (15.52 GiB usable, sm_120), driver 610.43.02 open modules, Debian 12 / Docker
• Flags: --fast --use-sage-attention --reserve-vram 1.5 --preview-method auto
• Models: minimax_h3_fl2va_pruned_nvfp4.safetensors (12.5GB DiT), qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors encoder, stock H3 video VAE fp16 + audio VAE fp32

Repro

Standard H3 t2v graph (MiniMaxH3ImageToVideo sans image / EmptyMiniMaxH3LatentAV path, BasicGuider → SamplerCustomAdvanced, res_multistep, 10 steps) at 864×480, length 243, then VAEDecode + VAEDecodeAudio → CreateVideo. Sampling completes (~3:40 at ~24 s/it); decode then:

Requested to load MiniMaxH3VideoVAE
0 models unloaded.
Model MiniMaxH3VideoVAE prepared for dynamic VRAM loading. 4965MB Staged. ...
Warning: Ran out of memory when regular VAE decoding, retrying with tiled VAE decoding.
!!! Exception during processing !!! CUDA out of memory. Tried to allocate 1.13 GiB.
GPU 0 has a total capacity of 15.52 GiB of which 821.56 MiB is free. Including
non-PyTorch memory, this process has 14.67 GiB memory in use. Of the allocated
memory 695.74 MiB is allocated by PyTorch, and 32.26 MiB is reserved by PyTorch
but unallocated.
Got an OOM, unloading all loaded models.
Prompt executed in 366.40 seconds

Reproduced 3/3 at 243f. At 362f it sometimes survives — when the first OOM's cleanup happens to release enough for the retry — which is a race, not a recovery path.

Note the telling number: only ~0.7GiB of the 14.67GiB in use is PyTorch-allocated. The rest is dynamically streamed weight memory the failing torch allocation cannot reclaim.

Diagnosis confirmation

Inserting a graph-level unload between the sampler and both decode nodes (a passthrough node calling comfy.model_management.unload_all_models()) makes the identical config complete with regular (non-tiled) decode: 243f finishes in 302s, no warning, repeatedly. So the memory exists; it is just never released for the decode.

I also tried raising the H3 branch's estimate_decode_memory coefficient in comfy/sd.py (9.5 → 20.0, i.e. declaring ~5.2GB at 243f instead of ~2.5GB): no behavioral change under dynamic VRAM — 0 models unloaded either way. (The 9.5 coefficient does look low against the observed working set regardless, but fixing it alone is not sufficient.)

Suggested directions

• In sd.py's decode OOM handler, actually free memory before the retry (model_management.free_memory(...) / release streamed mappings), instead of — for handles_tiling VAEs — re-running the same call.
• More generally: when a torch allocation OOMs while dynamic VRAM holds streamed weights, release streamed memory and retry. The DiT is dead weight during decode; evicting it costs a few seconds of re-staging on the next prompt.
• Honor memory_used_decode (and --reserve-vram) on the dynamic-VRAM path, so the decode's declared requirement translates into headroom before the first attempt.

Happy to provide full logs, the exact API-format graph, or to test patches — this box hits the failure deterministically.

[coderabbitai]

🔗 Related PRs

#15446 - Optimize MiniMax-H3 VAE [open]

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[DDavisWA]

Confirming this on a different setup, and I think I can point at the line responsible for
the "0 models unloaded" half.

Independent repro: RTX 5070 Ti 16 GB, Windows 11 native (not Docker), NVFP4 weights,
MiniMax_H3_Ref2VA_pruned_nvfp4 + the same qwen3vl_32b_minimax_h3_nvfp4_awq encoder.
Same shape of failure — sampling completes, VAEDecode dies, and the tiled retry re-runs
the identical decode. Both of your claims still hold on master at a779de4:
MiniMaxH3VideoVAE.decode_tiled() is still return self.decode(z), and the do_tile
branch in sd.py still only does soft_empty_cache() before retrying.

Why nothing is evicted

load_models_gpu decides up front whether the whole load set is dynamic
(comfy/model_management.py:932-935):

free_for_dynamic=True
for x in models:
    if not x.is_dynamic():
        free_for_dynamic = False

For a VAE decode the set is just [vae.patcher], and with DynamicVRAM on that patcher is
a ModelPatcherDynamic, so free_for_dynamic stays True. It is then passed straight
through to both free_memory calls (:973 and :980 — the second is the one that emits
your 0 models unloaded).

Inside free_memory, that flag short-circuits eviction for every dynamic model already
resident (:882-889):

if not DISABLE_SMART_MEMORY or device is None:
    memory_to_free = 0 if device is None else memory_required - get_free_memory(device)
    if current_loaded_models[i].model.is_dynamic() and for_dynamic:
        #don't actually unload dynamic models for the sake of other dynamic models
        #as that works on-demand.
        memory_required -= current_loaded_models[i].model.loaded_size()
        memory_to_free = 0
if memory_to_free > 0 and current_loaded_models[i].model_unload(memory_to_free):

So when a dynamic model asks for room and the residents are also dynamic, memory_to_free
is forced to 0 and model_unload is never called. The DiT and text encoder are skipped
by design, on the assumption that streaming makes their residency elastic — which holds
for weights faulting in on demand, but not for the decode's contiguous activation
allocations, which need the space to actually exist.

That also explains your memory_used_decode experiment. Raising the coefficient raises
memory_required, but memory_required is only read on the line that computes
memory_to_free — and memory_to_free is then overwritten with 0 regardless. The
declared requirement cannot become eviction on that path no matter how large it is.

A narrower lever than unload_all_models()

The capability is there; it is only the flag that suppresses it. Calling free_memory
with for_dynamic left at its default False does evict dynamic models — it reaches
model_unload, which for a ModelPatcherDynamic goes on to partially_unload →
vbar.free_memory.

Doing that immediately before the decode, measured on my box:

before:  3,162 MiB free   (MiniMaxH3 11,137 of 11,946 MiB resident)
after:  11,326 MiB free

and the decode then completes non-tiled, matching your unload_all_models() result. Peak
whole-device usage for the window came out at 15,057 MiB of 16,306.

One caveat on that measurement: I hit this from a headless harness that drives the nodes
directly rather than from a graph, so treat the numbers as corroboration of the mechanism
rather than as a like-for-like with your run. The free_memory behaviour is the same
either way. Worth noting too that passing a very large memory_required makes
model_unload skip partially_unload and fully detach instead, so the cost is a full
re-stage next prompt rather than a partial one — probably still the right trade when the
alternative is losing a multi-minute sample, but a bounded request would be gentler.

Not proposing a specific patch, since the intent behind that branch is presumably that
dynamic residency is supposed to be elastic. But it looks like the case it does not cover
is exactly this one: a dynamic model needing a large contiguous allocation while other
dynamic models hold the address space.