fix(capi): hoist OrEmpty out of extern "C" — the C ABI did not compile on Apple Clang - #195
Merged
Merged
Conversation
…ile on Apple Clang
`OrEmpty` returns std::string and was defined INSIDE the `extern "C" {` block, so
it inherited C linkage while returning a C++ type. GCC only warns; Apple Clang's
-Wreturn-type-c-linkage is an ERROR under the -Werror this project builds with,
so the entire C ABI translation unit failed to compile there.
Found DOWNSTREAM, not by our own gates: the LocalAI `vllm-cpp` backend
(mudler/LocalAI#11424) hit it pinning a recent SHA and had to vendor a patch to
build its metal-darwin-arm64 lane. This is that patch, upstream, so the vendored
copy can be deleted at the next pin.
The helper is hoisted above the boundary and keeps C++ linkage; the comment
states the rule it enforces — helpers that take or return C++ types stay on the
C++ side of `extern "C"`. No ABI change: `OrEmpty` is in an anonymous namespace
and was never an exported symbol.
Our CI does not build with Apple Clang, which is why a portability break in the
one file whose whole job is portability went unnoticed.
Verified: clean rebuild on Thor (sm_110), BUILD_RC=0.
FOLLOWING_AGENTS_PROTOCOL
Assisted-by: Claude Code:claude-opus-5 [ClaudeCode]
mudler
added a commit
to mudler/LocalAI
that referenced
this pull request
Aug 9, 2026
… OrEmpty patch The OrEmpty linkage fix this backend carried as patches/0001-* landed upstream (mudler/vllm.cpp#195, 7534da65), so the patch has done its job. It is deleted rather than left in place: the Makefile applies patches/*.patch unguarded and documents that "a patch that no longer applies must FAIL the clone", so keeping it against fixed source would break the build the moment the pin moved. Bumping the pin and deleting the patch therefore have to be the SAME change. Pin f921062b -> 776c56f1 (current vllm.cpp main). That range also carries the engine's ABI v17 (vllm_server_main: the OpenAI server published on the public surface). registerLib compares the library's vllm_abi_version against `abiVersion` for EXACT equality, so the constant moves 16 -> 17 in the same commit or every load fails with an ABI mismatch. The bump is safe for the layout assertions in video_test.go: diffing include/vllm.h across the two pins shows zero struct-field changes -- v17 adds one function declaration, the version macro and a doc comment, nothing else -- so every unsafe.Offsetof in the video params test still holds. Assisted-by: Claude Code:claude-opus-5 [ClaudeCode]
mudler
added a commit
to mudler/LocalAI
that referenced
this pull request
Aug 9, 2026
* feat(vllm-cpp): serve MiniMax-H3 video+audio generation
vllm.cpp's C ABI grew a video slice (ABI v12): a second engine handle
loaded from the MiniMax-H3 checkpoint SET, one blocking generate, and a
composed ffmpeg argv the caller execs. This wires that into LocalAI's
existing /video endpoint, so `vllm-cpp` now serves both text and video
and a clip comes back as an MP4 with a real audio track rather than a
silent render.
The video engine is a separate handle rather than a mode of the text
one because H3 is not a model directory: the DiT, the text encoder and
two VAEs are separate artifacts, and vllm.cpp has the two loaders refuse
each other's checkpoints. `Load` takes the video branch when the config
declares any of the video options; `parameters.model` is the DiT and the
rest of the set is named in `options:`.
Three details are worth calling out because getting them wrong is
expensive:
- The partition is DECLARED, not detected. The community quantisations
strip the release metadata and the FL2VA and Ref2VA DiTs are
byte-structurally identical, so the engine refuses to generate until
it is told which it has. Worse, a mismatch does not fail cleanly: a
reference passed to an FL2VA DiT renders for hours and returns a
coloured lattice over the frame. The backend refuses that combination
up front instead.
- ffmpeg comes from the host. libvllm writes frames plus a WAV and
composes the mux argv, then spawns nothing - that process boundary is
upstream's decision. The backend execs it, the same arrangement
vibevoice-cpp uses for transcoding, and ffmpeg also converts a
start_image upload into the binary PPM at the exact output canvas the
engine requires.
- It is slow. Roughly 176 s per denoise step at the default 1344x768
canvas on a 20-SM device, so the 50-step default is a multi-hour job.
Nothing on this path imposes a deadline.
The /video endpoint no longer forces 512x512 when the request omits the
geometry. Every video backend already supplies its own default for a
zero (512x512 for stablediffusion-ggml, 1280x720 for diffusers, 832x480
for longcat-video, 1344x768 for H3), so the hardcoded value only ever
overrode the model's trained canvas with one three of the four were
never trained at.
Moving the engine pin from ABI v10 to v16 also grows the text
vllm_model_params mirror by the v14 device field and the v16 KV-sizing
knobs. LocalAI sets none of them - 0 is the pre-v14 engine byte for byte
- but the struct SIZE is part of the layout contract, so leaving them
out would have vllm_engine_load read past the allocation.
Gallery: `minimax-h3-fl2va-q4` installs the Q4_K_M FL2VA set (~40 GB
across five weight files plus the two VAE configs that carry the latent
statistics).
Assisted-by: Claude:claude-opus-5 golangci-lint yamllint go-vet
* fix(vllm-cpp): unbreak the Darwin build at the new engine pin
src/capi/vllm_c.cpp opens one `extern "C" {` for the whole ABI surface,
so file-local helpers declared inside it inherit C linkage. The video
slice added one that returns std::string, which Apple Clang reports as
-Wreturn-type-c-linkage and vllm.cpp's target-local -Werror turns into a
build failure. GCC and upstream Clang do not diagnose it, so only the
metal-darwin-arm64 job saw it.
Suppress it the same way this Makefile already suppresses Apple Clang's
-Wgnu-folding-constant on the Metal build. The helper is never called
across the boundary so the warning describes no hazard here, but it is a
real upstream wart: the fix belongs in vllm.cpp, hoisting the helper
above the extern "C" block, and this flag should go when a pin carrying
that fix lands.
Assisted-by: Claude:claude-opus-5
* fix(vllm-cpp): patch the engine clone instead of the warning flag
The -Wno-return-type-c-linkage added in the previous commit does nothing.
vllm_cpp_set_warnings adds `-Wall -Wextra -Werror` as PRIVATE target
options, so they land after anything CMAKE_CXX_FLAGS contributes, and
-Wall re-enables the -Wreturn-type group that -Wreturn-type-c-linkage
belongs to. The darwin job failed again on the same line, which is the
evidence: a consumer cannot wave this off from outside the engine.
Position is the only fix, so carry it as a patch against the pinned SHA,
the way longcat-video patches its own upstream. It hoists the helper
above the `extern "C" {` that gives it C linkage; it is file-local and
never called across the boundary, so nothing else moves.
`git apply` is unguarded on purpose: a patch that stops applying must
fail the clone loudly, because the alternative is a pin that silently
ships without a fix it is documented to carry. The patch header names
what retires it - a pin carrying the fix upstream, where it belongs.
Verified by applying the patch with `git apply` to the exact blob at the
pinned SHA and diffing the result against the intended file.
Assisted-by: Claude:claude-opus-5
* chore(vllm-cpp): bump the engine pin to ABI v17 and drop the vendored OrEmpty patch
The OrEmpty linkage fix this backend carried as patches/0001-* landed upstream
(mudler/vllm.cpp#195, 7534da65), so the patch has done its job. It is deleted
rather than left in place: the Makefile applies patches/*.patch unguarded and
documents that "a patch that no longer applies must FAIL the clone", so keeping
it against fixed source would break the build the moment the pin moved. Bumping
the pin and deleting the patch therefore have to be the SAME change.
Pin f921062b -> 776c56f1 (current vllm.cpp main).
That range also carries the engine's ABI v17 (vllm_server_main: the OpenAI server
published on the public surface). registerLib compares the library's
vllm_abi_version against `abiVersion` for EXACT equality, so the constant moves
16 -> 17 in the same commit or every load fails with an ABI mismatch.
The bump is safe for the layout assertions in video_test.go: diffing include/vllm.h
across the two pins shows zero struct-field changes -- v17 adds one function
declaration, the version macro and a doc comment, nothing else -- so every
unsafe.Offsetof in the video params test still holds.
Assisted-by: Claude Code:claude-opus-5 [ClaudeCode]
* chore(vllm-cpp): re-pin to pick up the VLLM_CPP_SERVER=OFF link fix
The previous pin carried vllm.cpp's ABI v17 (vllm_server_main) but not the guard
that makes it link when the server is compiled out. This backend builds libvllm
with VLLM_CPP_SERVER off, so the darwin lane failed at the dylib link with
vllm::entrypoints::openai::VllmServerMain undefined.
Fixed upstream in mudler/vllm.cpp#202: the C entry point is now guarded, so the
symbol is still exported (ABI v17 stays resolvable for dlopen) while the
no-server arm reports the missing capability instead of dragging in a translation
unit that was never compiled.
Verified upstream in BOTH arms before re-pinning: SERVER=ON builds and runs, and
SERVER=OFF configures, links, produces libvllm.so, and `nm -D` shows
vllm_server_main exported next to vllm_video_generate and vllm_transcribe.
Assisted-by: Claude Code:claude-opus-5 [ClaudeCode]
---------
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
OrEmptyreturnsstd::stringand was defined inside theextern "C" {block, so it inherited C linkage while returning a C++ type.GCC only warns. Apple Clang's
-Wreturn-type-c-linkageis an error under the-Werrorthis project builds with — so the entire C ABI translation unit failed to compile there.Found downstream, not by our gates
The LocalAI
vllm-cppbackend (mudler/LocalAI#11424) hit this pinning a recent SHA and had to vendor a patch against us to build itsmetal-darwin-arm64lane. This is that patch, upstream, so the vendored copy can be deleted at the next pin.Our CI doesn't build with Apple Clang — which is how a portability break in the one file whose entire job is portability went unnoticed.
The fix
Hoist the helper above the boundary so it keeps C++ linkage, with a comment stating the rule: helpers that take or return C++ types stay on the C++ side of
extern "C".No ABI change —
OrEmptyis in an anonymous namespace and was never an exported symbol.Verified
Clean rebuild on Thor (sm_110):
BUILD_RC=0.I could not verify the Apple Clang lane directly — no Darwin host here. The reasoning is that
-Wreturn-type-c-linkagefires exactly on this construct and the downstream lane went green with the equivalent patch, but the real confirmation is LocalAI's CI at the next pin.