Regression: EasyCache degrades the audio stream on MiniMax H3 since #12231

[siraben]

Custom Node Testing

• I have tried disabling custom nodes and the issue persists

Expected Behavior

EasyCache degrades audio about as much as it degrades video.

Actual Behavior

On MiniMax H3 with EasyCache at its default reuse_threshold=0.2, the video comes out close to an uncached run but the audio does not. On a solo piano prompt the cached version has noticeably less bass and sounds slightly muffled next to the uncached one. It measures at half the amplitude, and its spectral centroid moves up from 9.6 to 10.6, which fits the missing low end. Video SSIM stays at 0.95-0.96 in the same runs, so this is easy to miss if you only watch the output.

Same seed, same graph, EasyCache the only difference:

prompt	skipped	video SSIM	audio log-mel L1	audio RMS ratio
ambience	5/20	0.963	0.428	0.695
dialogue	6/20	0.945	0.395	0.971
music	6/20	0.956	0.690	0.503

I used log-mel L1 rather than a waveform metric because two diffusion samples diverge in phase anyway, and I wanted a number that would not move for that reason alone.

EasyCache picks which steps to skip from the video stream, then applies the cached residual to audio too. See nodes_easycache.py#L61, with the [video, audio] split at L15.
#12231 says so plainly: "Caching decisions are still made based on video tensor only, which
seems to work just fine for the use case."

That works when both streams share a schedule. H3's audio does not. It returns audio velocity scaled by d(sigma_a)/d(sigma_v) at minimax/model.py#L645. I evaluated that factor across the schedule: it runs 4.00 at sigma_v=1.0 down to 0.26 near 0, and roughly halves between adjacent early steps. So a residual cached at one step is scaled wrong when it is replayed at the next.

Separately, the speedup the node prints does not count audio skips at all. L268-L270 increments total_steps_skipped only if not is_audio.

Steps to Reproduce

1. Load the workflow below (core node types only, H3 fl2va).
2. Bypass the EasyCache node, run, keep the output as reference.
3. Enable EasyCache at defaults (0.2 / 0.15 / 0.95), same seed, run again.
4. Listen to both, or compare the audio tracks numerically.

workflow JSON

{
 "prompt": {
  "6": {
   "class_type": "UNETLoader",
   "inputs": {
    "unet_name": "minimax_h3_fl2va_pruned_int8_convrot.safetensors",
    "weight_dtype": "default"
   }
  },
  "13": {
   "class_type": "CLIPLoader",
   "inputs": {
    "clip_name": "MiniMax-H3/qwen3vl_32b_minimax_h3_ultra_uncensored_heretic_int8_convrot.safetensors",
    "type": "minimax",
    "device": "default"
   }
  },
  "11": {
   "class_type": "VAELoader",
   "inputs": {
    "vae_name": "minimax_h3_video_vae_fp16.safetensors"
   }
  },
  "24": {
   "class_type": "VAELoader",
   "inputs": {
    "vae_name": "minimax_h3_audio_vae_fp32.safetensors"
   }
  },
  "104": {
   "class_type": "MiniMaxH3ImageToVideo",
   "inputs": {
    "clip": [
     "13",
     0
    ],
    "vae": [
     "11",
     0
    ],
    "prompt": "Live-action, cinematic, warm interior: 35mm photography, soft window light, shallow depth of field, gentle film grain.\n\n[Shot 1] A medium close-up holds on a calm woman in her thirties seated at a wooden kitchen table in a quiet room, morning light on her face. She looks directly at the camera and speaks clearly and unhurriedly. The camera is locked off, no movement.\n\nThe woman (S1) says: <d>[English] The train leaves at half past six, so we should go now before the rain starts again.</d>\n\nAudio: a single clear female speaking voice, close-miked and dry, in a quiet room with almost no background noise. No music. No ambience.",
    "width": 1056,
    "height": 608,
    "length": 124
   }
  },
  "15": {
   "class_type": "RandomNoise",
   "inputs": {
    "noise_seed": 12345
   }
  },
  "17": {
   "class_type": "KSamplerSelect",
   "inputs": {
    "sampler_name": "res_multistep"
   }
  },
  "16": {
   "class_type": "BasicGuider",
   "inputs": {
    "model": [
     "200",
     0
    ],
    "conditioning": [
     "104",
     0
    ]
   }
  },
  "9": {
   "class_type": "BasicScheduler",
   "inputs": {
    "model": [
     "200",
     0
    ],
    "scheduler": "beta",
    "steps": 20,
    "denoise": 1.0
   }
  },
  "14": {
   "class_type": "SamplerCustomAdvanced",
   "inputs": {
    "noise": [
     "15",
     0
    ],
    "guider": [
     "16",
     0
    ],
    "sampler": [
     "17",
     0
    ],
    "sigmas": [
     "9",
     0
    ],
    "latent_image": [
     "104",
     1
    ]
   }
  },
  "10": {
   "class_type": "VAEDecode",
   "inputs": {
    "samples": [
     "14",
     0
    ],
    "vae": [
     "11",
     0
    ]
   }
  },
  "23": {
   "class_type": "VAEDecodeAudio",
   "inputs": {
    "samples": [
     "14",
     0
    ],
    "vae": [
     "24",
     0
    ]
   }
  },
  "91": {
   "class_type": "CreateVideo",
   "inputs": {
    "images": [
     "10",
     0
    ],
    "audio": [
     "23",
     0
    ],
    "fps": 24.0
   }
  },
  "92": {
   "class_type": "SaveVideo",
   "inputs": {
    "video": [
     "91",
     0
    ],
    "filename_prefix": "video/bench/h3",
    "format": "auto",
    "codec": "auto"
   }
  },
  "200": {
   "class_type": "EasyCache",
   "inputs": {
    "model": [
     "6",
     0
    ],
    "reuse_threshold": 0.2,
    "start_percent": 0.15,
    "end_percent": 0.95,
    "verbose": true
   }
  }
 }
}

1056x608, 124 frames, 20 steps, res_multistep + beta, seed 12345, minimax_h3_fl2va_pruned_int8_convrot, RTX 4090, torch 2.11.0+cu128, ComfyUI at 16e3f30.

I checked the harness before trusting any of this: at reuse_threshold=0.04 EasyCache skips nothing and the output is byte-identical to the reference. So the differences above come from the skipped steps and not from run-to-run noise.

Debug Logs

expand

[INFO] Total VRAM 24080 MB, total RAM 128715 MB
[INFO] pytorch version: 2.11.0+cu128
[INFO] Device: cuda:0 NVIDIA GeForce RTX 4090 : cudaMallocAsync
[INFO] Using pytorch attention
[INFO] ComfyUI version: 0.30.0

[INFO] EasyCache enabled - threshold: 0.2, start_percent: 0.15, end_percent: 0.95
[INFO] Requested to load MiniMaxH3
[INFO] 0 models unloaded.
[INFO] Model MiniMaxH3 prepared for dynamic VRAM loading. 19995MB Staged. 0 patches attached. Force pre-loaded 210 weights: 1175 KB.
[INFO] EasyCache [verbose] - x_prev_subsampled: torch.Size([1, 24, 37, 5, 9])
[INFO] EasyCache [verbose] - approx_output_change_rate: 0.263671875
[INFO] EasyCache [verbose] - output_change_rate: 0.142578125
[INFO] EasyCache [verbose] - x_prev_subsampled: torch.Size([1, 24, 37, 5, 9])
[INFO] EasyCache [verbose] - NOT skipping step; cumulative_change_rate: 0.56640625, reuse_threshold: 0.2
[INFO] EasyCache [verbose] - approx_output_change_rate: 0.388671875
[INFO] EasyCache [verbose] - output_change_rate: 0.12890625
[INFO] EasyCache [verbose] - x_prev_subsampled: torch.Size([1, 24, 37, 5, 9])
[INFO] EasyCache [verbose] - NOT skipping step; cumulative_change_rate: 0.29296875, reuse_threshold: 0.2
[INFO] got prompt
[INFO] EasyCache [verbose] - approx_output_change_rate: 0.2021484375
[INFO] EasyCache [verbose] - output_change_rate: 0.07421875
[INFO] EasyCache [verbose] - x_prev_subsampled: torch.Size([1, 24, 37, 5, 9])
[INFO] EasyCache [verbose] - skipping step; cumulative_change_rate: 0.17578125, reuse_threshold: 0.2
[INFO] EasyCache [verbose] - NOT skipping step; cumulative_change_rate: 0.37890625, reuse_threshold: 0.2
[INFO] EasyCache [verbose] - approx_output_change_rate: 0.2041015625
[INFO] EasyCache [verbose] - output_change_rate: 0.0703125
[INFO] EasyCache [verbose] - x_prev_subsampled: torch.Size([1, 24, 37, 5, 9])
[INFO] EasyCache [verbose] - skipping step; cumulative_change_rate: 0.1416015625, reuse_threshold: 0.2
[INFO] EasyCache [verbose] - NOT skipping step; cumulative_change_rate: 0.30078125, reuse_threshold: 0.2
[INFO] EasyCache [verbose] - approx_output_change_rate: 0.1591796875
[INFO] EasyCache [verbose] - output_change_rate: 0.08349609375
[INFO] EasyCache [verbose] - x_prev_subsampled: torch.Size([1, 24, 37, 5, 9])
[INFO] EasyCache [verbose] - output_change_rates 6: [0.2138671875, 0.142578125, 0.12890625, 0.07421875, 0.0703125, 0.08349609375]
[INFO] EasyCache [verbose] - approx_output_change_rates 5: [0.263671875, 0.388671875, 0.2021484375, 0.2041015625, 0.1591796875]
[INFO] EasyCache - skipped 6/20 steps (1.43x speedup).
[INFO] Model MiniMaxH3AudioVAE prepared for dynamic VRAM loading. 576MB Staged. 0 patches attached. Force pre-loaded 401 weights: 539 KB.
[INFO] Model MiniMaxH3VideoVAE prepared for dynamic VRAM loading. 4965MB Staged. 0 patches attached. Force pre-loaded 128 weights: 348 KB.
[INFO] Prompt executed in 286.97 seconds

Other

Lowering the threshold is not a fix. I ran stock at reuse_threshold=0.1, which skips 3/20 on the ambience prompt, and mel L1 only came down to 0.283. To check whether the choice of steps mattered, I patched EasyCache to give audio its own change-rate accumulator and skip only when both streams agree. At the same 3/20 budget that scored 0.209. So some of the damage is which steps get skipped, not only how many.

That patch is not a proposal. It costs about half the speedup, and on the dialogue prompt it skips nothing at all, because the audio accumulator runs 2-7x higher than the video one. I suspect that is mostly a scale artifact, since the audio latent is ~20x smaller, so its per-step relative change is larger, and one shared threshold ends up much stricter on audio. Normalising each stream, or giving audio its own threshold, would probably avoid that, but I have not tried it and I do not know what shape you would want.

The cheapest useful change is probably documenting that the node caches a second stream on the first stream's schedule. Right now nothing tells the user, including the embedded docs.

experimental patch (+56/-2) and per-prompt numbers

prompt		skipped	speedup	video SSIM	audio mel L1	audio RMS
ambience	stock	5/20	1.38x	0.963	0.428	0.695
ambience	gated	3/20	1.16x	0.970	0.209	0.898
dialogue	stock	6/20	1.38x	0.945	0.395	0.971
dialogue	gated	0/20	1.00x	1.000	0.000	1.000
music	stock	6/20	1.40x	0.956	0.690	0.503
music	gated	3/20	1.17x	0.966	0.240	0.885

diff --git a/comfy_extras/nodes_easycache.py b/comfy_extras/nodes_easycache.py
index 9e907d3..9269864 100644
--- a/comfy_extras/nodes_easycache.py
+++ b/comfy_extras/nodes_easycache.py
@@ -34,6 +34,7 @@ def easycache_forward_wrapper(executor, *args, **kwargs):
     has_first_cond_uuid = easycache.has_first_cond_uuid(uuids)
     next_x_prev = x
     input_change = None
+    audio_input_change = None
     do_easycache = easycache.should_do_easycache(sigmas)
     if do_easycache:
         easycache.check_metadata(x)
@@ -55,10 +56,20 @@ def easycache_forward_wrapper(executor, *args, **kwargs):
         if has_first_cond_uuid:
             if easycache.has_x_prev_subsampled():
                 input_change = (easycache.subsample(x, uuids, clone=False) - easycache.x_prev_subsampled).flatten().abs().mean()
+            # audio gets its own signal; it may be on a different sigma schedule
+            if ax is not None and easycache.has_ax_prev_subsampled():
+                audio_input_change = (easycache.subsample_audio(ax, uuids, clone=False) - easycache.ax_prev_subsampled).flatten().abs().mean()
             if easycache.has_output_prev_norm() and easycache.has_relative_transformation_rate():
                 approx_output_change_rate = (easycache.relative_transformation_rate * input_change) / easycache.output_prev_norm
                 easycache.cumulative_change_rate += approx_output_change_rate
-                if easycache.cumulative_change_rate < easycache.reuse_threshold and can_apply_cache_diff:
+                # only skippable if both streams predict a small change
+                effective_change_rate = easycache.cumulative_change_rate
+                if (audio_input_change is not None and easycache.has_audio_output_prev_norm()
+                        and easycache.has_audio_relative_transformation_rate()):
+                    easycache.audio_cumulative_change_rate += (
+                        easycache.audio_relative_transformation_rate * audio_input_change) / easycache.audio_output_prev_norm
+                    effective_change_rate = max(effective_change_rate, easycache.audio_cumulative_change_rate)
+                if effective_change_rate < easycache.reuse_threshold and can_apply_cache_diff:
                     if easycache.verbose:
                         logging.info(f"EasyCache [verbose] - skipping step; cumulative_change_rate: {easycache.cumulative_change_rate}, reuse_threshold: {easycache.reuse_threshold}")
                     # other conds should also skip this step, and instead use their cached values
@@ -70,8 +81,9 @@ def easycache_forward_wrapper(executor, *args, **kwargs):
                     return result
                 else:
                     if easycache.verbose:
-                        logging.info(f"EasyCache [verbose] - NOT skipping step; cumulative_change_rate: {easycache.cumulative_change_rate}, reuse_threshold: {easycache.reuse_threshold}")
+                        logging.info(f"EasyCache [verbose] - NOT skipping step; cumulative_change_rate: {easycache.cumulative_change_rate}, audio: {easycache.audio_cumulative_change_rate}, reuse_threshold: {easycache.reuse_threshold}")
                     easycache.cumulative_change_rate = 0.0
+                    easycache.audio_cumulative_change_rate = 0.0
 
     full_output: torch.Tensor = executor(*args, **kwargs)
     output, audio_output = _extract_tensor(full_output, easycache.output_channels)
@@ -97,6 +109,14 @@ def easycache_forward_wrapper(executor, *args, **kwargs):
         easycache.x_prev_subsampled = easycache.subsample(next_x_prev, uuids)
         easycache.output_prev_subsampled = easycache.subsample(output, uuids)
         easycache.output_prev_norm = output.flatten().abs().mean()
+        if audio_output is not None and ax is not None:
+            if easycache.has_audio_output_prev_norm() and audio_input_change is not None:
+                audio_output_change = (easycache.subsample_audio(audio_output, uuids, clone=False)
+                                       - easycache.audio_output_prev_subsampled).flatten().abs().mean()
+                easycache.audio_relative_transformation_rate = audio_output_change / audio_input_change
+            easycache.ax_prev_subsampled = easycache.subsample_audio(ax, uuids)
+            easycache.audio_output_prev_subsampled = easycache.subsample_audio(audio_output, uuids)
+            easycache.audio_output_prev_norm = audio_output.flatten().abs().mean()
         if easycache.verbose:
             logging.info(f"EasyCache [verbose] - x_prev_subsampled: {easycache.x_prev_subsampled.shape}")
     return full_output
@@ -210,11 +230,17 @@ class EasyCacheHolder:
         self.cumulative_change_rate = 0.0
         self.initial_step = True
         self.skip_current_step = False
+        # audio equivalents of the above
+        self.audio_relative_transformation_rate: float = None
+        self.audio_cumulative_change_rate = 0.0
         # cache values
         self.first_cond_uuid = None
         self.x_prev_subsampled: torch.Tensor = None
         self.output_prev_subsampled: torch.Tensor = None
         self.output_prev_norm: torch.Tensor = None
+        self.ax_prev_subsampled: torch.Tensor = None
+        self.audio_output_prev_subsampled: torch.Tensor = None
+        self.audio_output_prev_norm: torch.Tensor = None
         self.uuid_cache_diffs: dict[UUID, torch.Tensor] = {}
         self.uuid_cache_diffs_audio: dict[UUID, torch.Tensor] = {}
         self.output_change_rates = []
@@ -244,11 +270,31 @@ class EasyCacheHolder:
     def has_relative_transformation_rate(self) -> bool:
         return self.relative_transformation_rate is not None
 
+    def has_ax_prev_subsampled(self) -> bool:
+        return self.ax_prev_subsampled is not None
+
+    def has_audio_output_prev_norm(self) -> bool:
+        return self.audio_output_prev_norm is not None
+
+    def has_audio_relative_transformation_rate(self) -> bool:
+        return self.audio_relative_transformation_rate is not None
+
     def prepare_timesteps(self, model_sampling):
         self.start_t = model_sampling.percent_to_sigma(self.start_percent)
         self.end_t = model_sampling.percent_to_sigma(self.end_percent)
         return self
 
+    def subsample_audio(self, ax: torch.Tensor, uuids: list[UUID], clone: bool = True) -> torch.Tensor:
+        """Audio is [B, C, n, T]; only the time axis is worth striding."""
+        batch_offset = ax.shape[0] // len(uuids)
+        uuid_idx = uuids.index(self.first_cond_uuid)
+        to_return = ax[uuid_idx * batch_offset:(uuid_idx + 1) * batch_offset, ...]
+        if self.subsample_factor > 1:
+            to_return = to_return[..., ::self.subsample_factor]
+        if clone:
+            return to_return.clone()
+        return to_return
+
     def subsample(self, x: torch.Tensor, uuids: list[UUID], clone: bool = True) -> torch.Tensor:
         batch_offset = x.shape[0] // len(uuids)
         uuid_idx = uuids.index(self.first_cond_uuid)
@@ -334,6 +380,8 @@ class EasyCacheHolder:
     def reset(self):
         self.relative_transformation_rate = 0.0
         self.cumulative_change_rate = 0.0
+        self.audio_relative_transformation_rate = None
+        self.audio_cumulative_change_rate = 0.0
         self.initial_step = True
         self.skip_current_step = False
         self.output_change_rates = []
@@ -344,6 +392,12 @@ class EasyCacheHolder:
         self.output_prev_subsampled = None
         del self.output_prev_norm
         self.output_prev_norm = None
+        del self.ax_prev_subsampled
+        self.ax_prev_subsampled = None
+        del self.audio_output_prev_subsampled
+        self.audio_output_prev_subsampled = None
+        del self.audio_output_prev_norm
+        self.audio_output_prev_norm = None
         del self.uuid_cache_diffs
         self.uuid_cache_diffs = {}
         del self.uuid_cache_diffs_audio

Caveats: one seed per cell, H3 only. LTX-2 was the target of #12231 and may well have a schedule where the original assumption holds, so this may be specific to H3 rather than a problem with that PR.

[coderabbitai]

🔗 Related PRs

#15210 - Support MiniMax H3 [open]
#15224 - feat: Support MiniMax-H3 (CORE-375) [merged]
#15296 - restore SDPA non-cudnn small attention bypass [merged]

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!