Skip to content

0.2.12

Choose a tag to compare

@github-actions github-actions released this 06 Aug 06:18
· 44 commits to main since this release

Changes since the last Release

Fixed

  • Restore the SD path on shared-ggml builds: GGML_MAX_NAME had diverged across the link (heap corruption). Any chimera sd run against Z-Image Turbo aborted mid-sampling with glibc's corrupted double-linked list (or a SIGSEGV — the manifestation was nondeterministic). The cause was not sd.cpp, the model, or VRAM: GGML_MAX_NAME sizes char name[GGML_MAX_NAME] inside struct ggml_tensor, so it sets sizeof(ggml_tensor) (128 -> 400 bytes, 160 -> 432) and the offset of every field after name. chimera links only llama.cpp's ggml (CMakeLists.txt, all backends) while stable-diffusion.cpp and whisper.cpp contribute their own object code, so all of them must be compiled with the same value. None were: llama.cpp's ggml and chimera's own TUs were pinned at 128, whisper.cpp got nothing and defaulted to 64, and sd.cpp compiles itself at 160 (its CMakeLists.txt sets add_definitions(-DGGML_MAX_NAME=160); upstream moved it up from 128 at some point and chimera's hardcoded 128 was never updated). sd.cpp therefore wrote tensor->extra at offset 416 into 400-byte tensors ggml had allocated — 16 bytes past the end, into the adjacent chunk's heap metadata. The stale pins even cited sd.cpp's "CMakeLists.txt:233"; that line is now 315, which is the drift itself showing in the comment.

    Nothing caught it at build time because sd.cpp's own guard is static_assert(GGML_MAX_NAME >= 128) (src/core/ggml_extend.hpp) — a minimum, so 128 compiled clean. And nothing caught it at runtime until a malloc unrelated to the overflow happened to walk the damaged free list, which is why the backtrace pointed at innocent allocation sites (copy_data_to_backend_tensor, ggml_gallocr_reserve_n_impl) that moved as soon as allocation timing changed. The attribution shortcut for this class of bug: build/stable-diffusion.cpp/build/bin/sd-cli is built self-consistently at 160 and succeeds on configurations chimera crashes on — when upstream's own CLI is fine, suspect the shared-ggml link rather than upstream.

  • Apply GGML_MAX_NAME to the CUDA and HIP languages, not just C/C++. The above fix is incomplete without this and the crash survives it. CMAKE_C_FLAGS / CMAKE_CXX_FLAGS do not reach .cu files — CUDA and HIP are separate CMake languages with their own flag variables — so ggml-cuda in both llama.cpp and whisper.cpp stayed on the 64 default while ggml-base and ggml-cpu alongside it used the pinned value. The linked ggml disagreed with its own backend about sizeof(ggml_tensor). This predates the SD divergence: llama.cpp's ggml-cuda was at 64 against a ggml-base at 128 in prior releases too, independent of whether SD was in the link. sd.cpp's tree was never exposed because it uses add_definitions(), which applies to every language. GgmlBuilder.ggml_max_name_flags() now emits CMAKE_C_FLAGS, CMAKE_CXX_FLAGS, CMAKE_CUDA_FLAGS and CMAKE_HIP_FLAGS together, and both the llama.cpp and whisper.cpp builders consume it. The HIP leg is reasoned-correct but was not exercised on ROCm hardware.

  • Pin whisper.cpp's GGML_MAX_NAME at all. The whisper builder never passed the define, so every whisper TU compiled at the 64 default against a shared ggml built at 128 — the same class of mismatch as SD's, in the opposite direction (whisper read fields at lower offsets than reality rather than overrunning, which is why it degraded quietly instead of aborting). No whisper failure was ever observed and the whisper tests passed both before and after, so this is a latent defect closed rather than an observed one fixed.

  • Fetch stable-diffusion.cpp before reading GGML_MAX_NAME off it, instead of falling back. Only reproducible from a clean tree, which is why it survived the incremental builds the fix above was developed against. Builders clone lazily and run in list order — llama.cpp, whisper.cpp, then stable-diffusion.cpp — so on a fresh make reset build ggml_max_name() read SD's CMakeLists.txt before that tree existed on disk, silently took GGML_MAX_NAME_FALLBACK, and configured llama.cpp and whisper.cpp from a guess. That resolves to the right answer only while the fallback happens to equal SD's current value: the first time upstream moves it and SDCPP_VERSION is bumped, a reset build would give llama.cpp/whisper.cpp the stale fallback and SD the new value — reintroducing the exact ABI split this machinery exists to prevent, through the machinery meant to prevent it. ggml_max_name() now clones SD on demand when its CMakeLists.txt is absent (its own builder clones it moments later regardless, so the only cost is doing it earlier) and both fallback paths now log a warning naming the ABI-split consequence rather than returning quietly.

Changed

  • Derive GGML_MAX_NAME from stable-diffusion.cpp instead of pinning it in two places. The value belongs to upstream SD, which has moved it before and may again, and a stale copy fails silently at runtime rather than loudly at build time — so re-pinning 160 would only reset the same trap. StableDiffusionCppBuilder.ggml_max_name() parses GGML_MAX_NAME=<n> out of SD's own CMakeLists.txt, and chimera's top-level CMakeLists.txt does the same with file(STRINGS ...) for its own TUs. GGML_MAX_NAME_FALLBACK = 160 is a last resort only: the SD tree is fetched on demand so the value can be read from upstream (see above), leaving the fallback to cover an unreadable or unparseable checkout, and it warns when it fires. Configure now reports the resolved value (-- GGML_MAX_NAME: 160 (from stable-diffusion.cpp)), which makes a future divergence visible in the build log. Both parsers are inert when SD_USE_VENDORED_GGML=1, where each project keeps its own ggml and no agreement is required.

    To audit agreement across a build tree, every flags.make must report the same number:

    find build/{llama.cpp,whisper.cpp,stable-diffusion.cpp}/build build/src \
         -name flags.make | xargs grep -ho 'GGML_MAX_NAME=[0-9]*' | sort -u
    

Testing

  • Verified against the Z-Image Turbo case that exposed it, on CUDA and Vulkan. scripts/case/z_turbo.sh (Z-Image Turbo Q6_K + Qwen3-4B + --offload-to-cpu --diffusion-fa, 1024x512) went from a reproducible Aborted (core dumped) to exit 0 on both backends, and was run four times on each to rule out the nondeterminism that makes single passes meaningless for heap corruption. The three variants that also aborted before — --clip-on-cpu --vae-on-cpu, without --diffusion-fa, and at 512x512 — pass on both. make test is 68 pass / 0 fail / 7 skip on each, unchanged from before the fix, with the whisper lanes passing on the 64 -> 160 move. Because chimera silences sd.cpp's logging there is no backend banner to read, so GPU residency was confirmed by sampling nvidia-smi during a run rather than inferred: CUDA peaked at 6087 MiB / 56% util, Vulkan at 5889 MiB / 77%. The Vulkan leg was built from a clean make reset, which is what surfaced the lazy-clone ordering bug above; the CUDA leg was incremental.

  • GGML_MAX_NAME agreement is audited per target, and has exactly one legitimate exception. Every flags.make across llama.cpp, whisper.cpp, stable-diffusion.cpp and chimera reports 160 — 87 targets on the Vulkan build. The exception is vulkan-shaders-gen, present once per dep, which does not carry the define and correctly should not: it is a host tool that compiles GLSL to SPIR-V, and its source includes only standard library headers, no ggml.h and no ggml_tensor. Any other target missing the define is a real mismatch.

  • Not covered. ROCm/HIP and SYCL were not built. CMAKE_HIP_FLAGS is the structurally correct counterpart to CMAKE_CUDA_FLAGS — HIP is likewise a separate CMake language — but unlike Vulkan (whose backend is a single .cpp reached by CMAKE_CXX_FLAGS, so its coverage follows from the C++ path) that cannot be discharged by inspection and needs ROCm hardware to confirm. Separately, note that the 8 GB test card cannot run this model without --offload-to-cpu on either backend; that path fails cleanly with a 5638 MB allocation error, which is correct behaviour and unrelated to this bug.