Upgrade llama.cpp to b10423 and adapt to server_slot_stats API - #389
Merged
Conversation
No API surface touched: an EXAONE 4.5 SWA model fix and a CI test tweak. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xc744pEHNMvXSeH2uz2zgH
Upstream added pocket-tts support to mtmd (#26871), which changed the audio-generation stop protocol: mtmd_helper_gen_audio_step_gen() gained an out_stop output parameter and now reports end-of-speech itself (also signalled by a null next hidden state), while mtmd_helper_gen_audio_inp gained a seed field. Upstream's own tools/tts/tts.cpp dropped the llama_vocab_is_eog() check on the sampled backbone token accordingly, because pocket-tts has no discrete backbone token to test. tts_engine.cpp mirrors that: the generation loop now terminates on out_stop or a null h_next instead of on an end-of-generation token (for Qwen3-TTS the new path is equivalent -- upstream sets out_stop for exactly the codec-EOS and EOG tokens the old check tested), and the caller-supplied seed is forwarded to the mmproj-side sampling as well as to the backbone sampler. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xc744pEHNMvXSeH2uz2zgH
No project-facing API change: a Qwen bare-function chat-parsing fix inside common/chat.cpp, nemotron/dflash model work, imatrix and conversion-script changes -- all upstream-internal translation units. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xc744pEHNMvXSeH2uz2zgH
common/speculative.h drops common_speculative_need_embd() and common_speculative_need_embd_nextn() (#26904); no project source calls the common_speculative_* API at all, so the removal is confined to upstream translation units. The rest is a muse-glimmer tool-call parsing fix, CUDA kernel work and a server-context call-site follow-up. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xc744pEHNMvXSeH2uz2zgH
Build-system break, fixed here: upstream #26839 ("cmake: introduce semantic
versioning") replaced LLAMA_INSTALL_VERSION with LLAMA_VERSION_BASE /
LLAMA_VERSION_MAJOR. tools/mtmd/CMakeLists.txt -- which this project adds via
add_subdirectory() from its own directory scope, where llama.cpp's variables
are not visible -- passes both to set_target_properties(), so the existing
LLAMA_INSTALL_VERSION fallback no longer prevented the odd-length argument list
that aborts the configure. The guard now defines the two new variables instead.
Other API-visible upstream changes, none of which the project code touches:
llama.h gains llama_version(); common/build-info.h's llama_print_build_info()
takes a version string (jllama.cpp only calls llama_build_info());
server_tokens loses has_media() and gains serialize()/deserialize() for slot
save/restore with media inputs (#26640); common/arg.cpp learns system- and
user-level config.ini files (#26118).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xc744pEHNMvXSeH2uz2zgH
Nothing project-facing: WebUI refactors (auto-followed by CI's build-webui job), a ggml-hip flag removal and an arm unused-variable fix. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xc744pEHNMvXSeH2uz2zgH
Upstream #26920 refactored the server's metrics. result_timings is gone, replaced by server_slot_stats (raw counters + absolute timestamps, with every duration and rate derived in accessors); the "timings" JSON keys are unchanged, but the field is now gated on stats.is_set() instead of a non-negative prompt_n. server_task_result_metrics lost its flat counter fields to an embedded server_metrics, its to_json() now returns the /slots array verbatim, and a new to_metrics() renders the Prometheus exposition text for /metrics. No production source touches those types, so only the C++ regression suite needed rewriting: the ResultTimings suite becomes ServerSlotStats (same JSON contract, plus new coverage for is_set() and the unstarted-generation zero durations), and the metrics suite now asserts the slots array and parses to_metrics() samples -- including that the per-draft-position labeled series appears only when the data exists. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xc744pEHNMvXSeH2uz2zgH
Additive only on the API surface: common/speculative.h gains common_speculative_types_from_gguf() for draft-type auto-detection (#26814), which common/arg.cpp calls -- the project uses neither. The rest is backend work (SYCL/Metal/CPU), an LFM2 tool-call parsing fix, and server-http.cpp serving index.html with no-cache while hashed assets stay immutable. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xc744pEHNMvXSeH2uz2zgH
A single upstream commit (#26952), entirely inside the OpenVINO backend:
Qwen3.5 support, memory optimization and new ops. No shared header changes, so
the only artifact affected is the openvino-{linux,windows}-x86-64 classifier,
which builds the backend from these sources unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xc744pEHNMvXSeH2uz2zgH
common/common.h gains a common_threadpools RAII helper and moves ggml_threadpool_params_from_cpu_params() next to it (#27026), so the standalone tools share one CPU-parameter path; no project code calls either. The rest is WebUI refactoring, which CI's build-webui job re-derives from this tag. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xc744pEHNMvXSeH2uz2zgH
One row per step of the 10-step walk, calling out the three steps that needed project changes (b10369 tts_engine.cpp, b10398 llama/CMakeLists.txt, b10408 test_server.cpp) and recording that all six patches applied cleanly at every intermediate tag. CLAUDE.md's C++ test table follows the ServerSlotStats / to_metrics rename and the new 485-test total. Also pins the metrics fixture's busy-slot ratio to an exact value: Prometheus samples are rendered with the default 6-significant-digit ostream precision, so comparing a rounded 4/300 against the exact double would have failed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xc744pEHNMvXSeH2uz2zgH
bernardladenthin
had a problem deploying
to
maven-central
August 13, 2026 23:17 — with 
GitHub Actions
Failure
GitHub Actions
Failure
bernardladenthin
had a problem deploying
to
startgate
August 13, 2026 23:18 — with
GitHub Actions
Error
GitHub Actions
Error
bernardladenthin
requested a deployment
to
maven-central
August 13, 2026 23:18 — with
GitHub Actions
Waiting
GitHub Actions
Waiting
|
|
Review: llama.cpp b10360 to b10423 upgrade ✅ Version consistency verified (CMakeLists.txt, README.md, CLAUDE.md, LlamaCppVersion.java) ✅ Build system correctly handles LLAMA_VERSION_BASE/MAJOR variables from b10398 ✅ TTS engine adapted for b10369 upstream changes
✅ Test suite refactored for server_slot_stats API
✅ Documentation properly updated No critical issues. Ready to merge. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
result_timings→server_slot_stats)server_slot_statsstruct that stores raw counters and timestamps instead of pre-computed durationsserver_task_result_metrics::to_json()(returns slots array) and newto_metrics()(Prometheus exposition text)seedparameter and handle end-of-speech signaling viastep_gen()callbackLLAMA_INSTALL_VERSION→LLAMA_VERSION_BASE/LLAMA_VERSION_MAJOR)Test plan
ServerSlotStats,ServerTaskResultMetrics,ServerTaskResultCmplPartial,CmplFinalVerboseFlagtest suites updated and passingRelated issues / PRs
Upstream llama.cpp #26920: refactored server timing/metrics infrastructure to use raw counters + timestamps with derived accessors, improving accuracy and enabling Prometheus metrics export.
Checklist
CONTRIBUTING.mdandCODE_OF_CONDUCT.mdhttps://claude.ai/code/session_01Xc744pEHNMvXSeH2uz2zgH