Stop MiniMax-H3 aborting on the default cfg-scale and on --vae-on-cpu - #1862
Open
danielhanchen wants to merge 1 commit into
Open
Stop MiniMax-H3 aborting on the default cfg-scale and on --vae-on-cpu#1862danielhanchen wants to merge 1 commit into
danielhanchen wants to merge 1 commit into
Conversation
Two independent SIGABRTs, both reachable from ordinary invocations.
1. cfg-scale. H3 is distilled with guidance baked in and has no negative prompt
semantics; its empty uncond prompt encodes to zero tokens, so building the uncond
branch trips GGML_ASSERT(!ggml_is_transposed(a)) in ggml.c. sd.cpp defaults
--cfg-scale to 7.0, so a bare `sd-cli --mode vid_gen` on H3 crashes rather than
rendering. Measured: cfg 1.0 renders, cfg 1.5 and cfg 4.0 both abort, exit 134.
There is no correct cfg > 1 behaviour to implement for a CFG-free model, so warn
and clamp to 1.0 instead of asserting deep inside ggml.
2. --vae-on-cpu. ggml_conv_1d and ggml_conv_1d_dw build an F16 im2col, and the
CPU backend additionally requires the kernel itself to be F16;
ggml_compute_forward_im2col_f16 asserts it. audio_conv_weight_type maps only
BF16 to F16 and lets F32 through, so H3's F32 audio conv kernels abort with
GGML_ASSERT(src0->type == GGML_TYPE_F16) as soon as the audio VAE decodes on the
CPU. Converting the checkpoint to fp16 does not help: the type is imposed here,
not by the file.
Cast the kernel in-graph, and only when the runner is actually on a CPU backend,
so GPU precision is untouched and no weight is degraded at load. All four
conv_1d sites in this file need it, not just the module ones: the STFT
forward_basis and the transposed-conv reversed_filter are computed F32 tensors
that reach the same assert. depthwise_conv_transpose1d therefore takes the
runner context rather than a bare ggml_context.
Verified on minimax_h3_fl2va_pruned q4_K, 640x384, 25 frames, 4 steps, seed 1234:
case before after
--cfg-scale 4.0 exit 134 exit 0
cfg omitted (default 7.0) exit 134 exit 0
--vae-on-cpu --audio-vae exit 134 exit 0
full low_vram flag set exit 134 exit 0
baseline exit 0 exit 0, byte-identical
--offload-to-cpu exit 0 exit 0, byte-identical
The two CPU-VAE cases differ from the baseline by 114 bytes, which is the
expected consequence of their conv kernels now running F16 on the CPU.
danielhanchen
added a commit
to unslothai/stable-diffusion.cpp
that referenced
this pull request
Aug 8, 2026
The prebuilt pipeline builds leejet's source at an aged release tag, not this fork's master, so the three MiniMax-H3 fixes on master reach nobody: every Studio user installs a binary that still aborts on the default cfg-scale, still aborts on --vae-on-cpu, and still quantizes H3's 1-D norms into an output uncorrelated with its own bf16 reference. Building from master instead would throw away the reason the pipeline is shaped this way, which is that what we publish should be traceable to a specific upstream release. So keep the upstream tag as the base and carry the delta explicitly: - patches/ holds one file per fix, each with its upstream pull request in the header. All three are open on leejet: leejet#1861, leejet#1862, leejet#1863. - resolve applies them to the checked-out tag, after running git apply --check over the whole set so a stale patch stops the run before the tree is half modified. That failure is the signal to delete the patch (upstream merged it) or refresh it (upstream moved the code). - a non-empty set moves the published tag to <upstream tag>-u<id>, where id is the sha256 prefix of the concatenated patches. The tag then says whether a box is stock, and a changed patch set republishes rather than matching an existing release and skipping. - the manifest and the release notes both record the applied list. An empty patches/ leaves the tag and every asset name exactly as they are today. Verified by running the resolve step against master-813-bfbef5b with gh stubbed: all three patches apply, the tag becomes master-813-bfbef5b-u<id>, and the stamped source tarball contains the fixes.
Contributor
That's quite an assumption. We could get un-distilled models in the future (like we got for Flux in the past), it would be best to find a way to make CFG work without crashing rather than overriding the users settings. (Having a default guidance settings for each model based on architecture could be a nicer solution). Also those are two unrelated changes that would probably make more sense separated in two PRs. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two deterministic aborts on MiniMax-H3, both of which a user hits on a first run.
1. The default cfg-scale aborts
`--cfg-scale` defaults to 7.0, and H3 aborts at any value above 1.0:
SIGABRT, exit 134. So a `vid_gen` command that simply omits the flag crashes instead of rendering. H3 is distilled with guidance baked in and has no negative-prompt semantics; its empty uncond prompt encodes to zero tokens, which is where the assert comes from. Measured: cfg 1.0 renders, cfg 1.5 and cfg 4.0 both abort.
Since no cfg above 1.0 has a legitimate use on this model, this clamps to 1.0 with a warning rather than failing.
2. --vae-on-cpu aborts
The trigger is the audio VAE, not the video VAE. `ggml_conv_1d` and `ggml_conv_1d_dw` build an F16 im2col, and the CPU backend then asserts the kernel is F16, while the audio VAE conv kernels reach it as F32. Bisected on the flags: `--vae-on-cpu` with `--audio-vae` aborts, the same command without `--audio-vae` renders. Converting the audio VAE checkpoint to fp16 does not help, because the requirement is imposed inside the graph rather than by the file.
This casts F32 conv1d kernels to F16 in-graph, and only on CPU backends, so GPU backends keep their F32 kernels and no checkpoint is degraded on disk.
Verified end to end on a CUDA build with a q4_K H3 denoiser, the Qwen3-VL encoder and both VAEs: cfg 7 now warns and renders; `--vae-on-cpu` renders, alone and combined with `--offload-to-cpu --clip-on-cpu`.