v0.17.0

• Breaking (backend SPI): LoRA and multimodal projector members move off LlamaBackend into the optional BackendLoraSupport and BackendMultimodalSupport interfaces. On a backend or routed runtime without one, LlamaEngine LoRA calls and projector loads throw LlamaUnsupportedException, before any download, and removeLoraSource of an adapter never applied does nothing. The built-in backends keep the same support and exception type; LlamaEngine.loadMultimodalProjector refuses a LiteRT-LM model with the same LlamaUnsupportedException message as loadMultimodalProjectorSource and setModel, including on a custom backend that reports the LiteRT-LM runtime and implements BackendMultimodalSupport, and through a router it no longer says the projector is unsupported by the active runtime (#362, #1541).
• Update the default llama.cpp native runtime to leehack/llamadart-native@v0.7.0-3, a wrapper rebuild of llama.cpp v0.7.0, read through llamadart-native hook contract 2: the hook downloads a backend sidecar archive, such as Windows x64 CUDA, only when that backend is selected, so Windows x64 apps that do not select cuda download 15 MiB instead of 703 MiB, and an overridden release that needs an unsupported contract fails the build with NativeHookContractException (#812). Linux arm64 bundles ship FP16 CPU variant modules, and every backend mode loads the best variant the CPU supports, falling back to the baseline, so mixed auto and explicit loads share one CPU backend. Apple apps get the runtime with llamadart_llama_cpp_flutter 0.17.0.
• Pin stable-diffusion-native v0.3.0-1 (stable-diffusion.cpp master-1017), which loads FP8 image weights as F16 on Metal: FP8 checkpoints now generate on Metal instead of being refused. The bounded image recipes are requalified on it, and Flutter iOS/macOS apps need llamadart_stable_diffusion_flutter 0.17.0 (#1446).
• Aligned the default WebGPU bridge assets to v0.1.76: removes the retained projector copy behind the repeated Web media embedding batch abort on the wasm32 core, which stays unqualified (#1189) and can still exhaust its heap on larger contexts (llama-web-bridge#200); a speculative draft that does not fit throws LlamaUnsupportedException instead of leaking until the runtime aborts, and a model whose context does not fit fails its load instead of aborting (#198); and uncached model loads report progress as they download (#195). Upstream v0.7.0@620d2225cc436ab0b001e3200e590e3b286ef9ab matches native v0.7.0-2, which the manifest records; Web LiteRT-LM stays @litert-lm/core@0.18.0. Manifest SHA-256: 90822656cef6eaf43a47eafc7fe32f2ef9c2ed56a74c6b3cae29115ba61f818a.
• Check whether a model and ModelParams can load here before any download with await engine.checkLoad(model, params: params): the LlamaLoadCheckReport is unsupported exactly when setModel would throw before downloading (LiteRT-LM on macOS x86_64, Windows arm64 or the x86_64 iOS simulator, an explicit device the platform lacks, or an option the runtime ignores), and names the runtime and device the load would use. Custom backends join through BackendLoadCheck. setModel now also rejects a LiteRT-LM model with LlamaUnsupportedException before any download, keeping the loaded model, for ComputeDevice.npu or liteRtLmEmbeddingEngine on the Web and for liteRtLmEmbeddingEngine on native platforms without qualified embeddings (Linux and Windows); these used to fail at load after the previous model was unloaded (#861).
• Load a GGUF model on chosen GPUs with ModelParams.gpuDevices (listGpuDevices indices of one backend) and set each GPU's share with tensorSplit on native llama.cpp, validated on one Metal GPU and on a Vulkan Tesla T4, with Mesa lavapipe as the second device for two-device lists and splits (two-GPU hardware has no evidence yet, and CUDA is not hardware-qualified); decision heads and multimodal projectors of the model run on a listed GPU (#1517), and indices the host lacks, a split of another length, the Web and LiteRT-LM throw LlamaUnsupportedException (#916).
• Keep Mixture-of-Experts expert weights in host memory with ModelParams.cpuMoeLayers and cache recently used experts on the GPU with the experimental moeCacheBytes budget on native llama.cpp with Metal or CUDA, validated on Metal only (CUDA and multi-GPU hosts have no hardware evidence yet); a budget on Vulkan, other backends or tensor split, and either field on the Web or LiteRT-LM, throw LlamaUnsupportedException (#1123).
• LlamaEngine.stateIdentity() reports the native runtime build, session format, context size and a digest of the context settings that shape the saved state (KV cache types, flash attention, sequences, SWA and RoPE), so an app can check a llama.cpp state file another process saved before loading it. It needs native llama.cpp with llamadart-native v0.7.0-3; Web, LiteRT-LM, custom backends and older runtimes throw LlamaUnsupportedException (#1211).
• Return images from tools with LlamaToolResultContent.content or a handler's LlamaToolOutput on native llama.cpp with a vision projector and Hermes-style templates (smoke-tested on macOS CPU and Metal with Qwen3.5-0.8B weights, a qwen3vl_merger projector and the Qwen3-VL template), rendered as llama-server renders them; check LlamaEngineCapabilities.toolResultMedia. Other formats, runtimes, audio, video and URL-only images throw LlamaUnsupportedException (#1008).
• Parse a reply as the engine does with ChatTemplateEngine.parseRendered(rendered, output), which reads how its render or engine.chatTemplate result left the thought: after the stock Seed-OSS template, Plan</seed:think>Sunny. is content, as in llama-server, where ChatTemplateEngine.parse returns reasoning Plan (#1565).
• Native LiteRT-LM under ComputeDevice.auto retries a GPU engine that fails to start on the CPU with a warning, and reports LiteRT-LM cpu; explicit gpu and npu still fail (#1034).
• Refine text-to-image output in a second pass at up to twice the size with HiresFixImageGenerationRequest and ImageHiresFix (latent upscale, bounded to 1024x1024 and 4 images) on macOS CPU or Metal with SD 1.x or SD 2.x full-VAE checkpoints; check ImageGenerationCapabilities.supportsHiresFix, and other recipes throw LlamaUnsupportedException (#1228).
• Edit images from a reference with FLUX.1 Kontext [dev] through ReferenceImageEditingRequest and ImageReferenceEditingRecipe.fluxKontext (one RGB reference at the output size, at most 1024x1024) on macOS Metal with the Kontext transformer, ae, CLIP-L and T5-XXL; check ImageGenerationCapabilities.supportsReferenceImageEditing, and other models, devices and reference counts throw LlamaUnsupportedException (#1222).
• Load an SD 1.x ControlNet as ImageModelRole.controlNet and steer generation with ControlNetImageGenerationRequest on the bounded macOS CPU recipe; other ControlNet recipes are refused at load (#1224).
• Keep an image model's text encoders or VAE on the CPU, cap the GPU memory stable-diffusion.cpp uses, or turn off its automatic fit with ImageGenerationEngine.load(placement: ImageModelPlacement(...)) on Metal and Vulkan, validated with SD-Turbo on Metal and measured on one Vulkan Tesla T4 (multi-GPU hosts have no evidence yet); a placement on a CPU engine, Android, the CPU builds or the Web, and a GPU budget with the fit off or on Vulkan without a discrete GPU, throw LlamaUnsupportedException (#806).
• Chat templates now receive enableThinking as enable_thinking on every request, including responseFormat and generic tool-call routes, as in llama.cpp, so templates such as GLM-4.6, GLM-4.7-Flash, GLM-OCR, SmolLM3, Apertus and DeepSeek-R1 distills render thinking as llama-server does for the same flag. enableThinking still defaults to true, so Apertus now deliberates and DeepSeek-R1 distills open a thought unless you pass false (#1465).
• A boolean enable_thinking or thinking chat template kwarg now overrides enableThinking, and a template's own thinking setting passed in chatTemplateKwargs, such as reasoning_effort for Hy3 and Cohere2 MoE or thinking_mode for MiniMax M3, is no longer replaced by the value llamadart derives from enableThinking, so both render as in llama-server; DeepSeek V3.1 forces its thought open from enableThinking alone (#750).
• A responseFormat or tool-call grammar after a thought the prompt leaves open now lets the model finish the thought first, as llama-server does, instead of returning the JSON as reasoning; a thought the template opens stays open with enableThinking: false, as in llama-server (#1465). A required tool call after such a thought also lets Cohere2 MoE, Nemotron 3 Nano, MiniCPM5 and Hy3 models finish it first; MiniCPM5 previously wrote the call inside the thought and returned no tool call.
• A thought that Qwen3.5, QwQ and other Hermes-style templates open in the prompt, or that DeepSeek V3.1 forces open, is kept as llama-server does: an escaped \n and a repeated <think> stay in the reasoning as generated, thinking tags after the first </think> stay in the content instead of being parsed as more reasoning, a partial </think> that ends an unfinished thought is dropped, and a <tool_call> also ends a Qwen3.5 thought (#750).
• With thinking off, DeepSeek V3.1, Qwen3, QwQ and Qwen3.5 replies keep any <think> blocks the model writes in the content, as llama-server does, instead of dropping them as hidden reasoning (#750).
• With thinking off, StepFun3.5-Flash, DeepSeek R1 distills, MiniMax M2 and Nemotron Nano v2 replies no longer stream a thought the model writes before </think> as content: content waits for </think>, <think> or the end of the reply, matching the final message at any chunking (#1532).
• With thinking off, Cohere2 MoE replies no longer stream a thought the model writes before <|END_THINKING|> as content: content waits for a thinking tag, <|START_TEXT|> or <|START_ACTION|>, where llama-server ends the thought. Command R7B and Seed-OSS templates that open the thought with thinking off hold their reply until a thinking tag, and Ministral and Solar Open replies, whose parse keeps such text as content, no longer wait (#1553, #1557). With thinking off, a Cohere2 MoE thought that ends at <|START_TEXT|> or <|START_ACTION|> is reasoning instead of content, as in llama-server; thinking tags inside a Cohere2 MoE text or action stay text; and a response schema no longer leaves the thought open (#1566).
• Cohere2 MoE and Command R7B models on native llama.cpp keep the thinking, action and response markers their GGUFs store as control tokens, as llama-server does, so their thoughts, tool calls and text parse instead of all landing in reasoning or showing up as raw JSON (#1505).
• A Cohere2 MoE thought that the prompt opens ends at a <|START_TEXT|> or <|START_ACTION|> the model writes without <|END_THINKING|>, as in llama-server, so its text and tool calls no longer land in the reasoning; with enableThinking: false and a reasoning: true chat template kwarg, its prompt ends with the template's own empty thought, without a trailing newline (#1523).
• A Seed-OSS reply to a prompt that opens no thought, as the stock template renders, has one only when it starts with <seed:think>, kept as generated up to the first </seed:think>, as in llama-server; other thinking tags stay in the content (#1561). To a prompt that opens the thought, a repeated <seed:think>, and any text before it, stay in the reasoning up to the first </seed:think>, as in llama.cpp, instead of the tag being dropped, and a thinking tag inside a tool-call argument after that thought no longer drops the call.
• Replies with parallel tool calls return one call per block instead of none, and the tool-call grammar admits those blocks, or only one call when parallel tool calls are off, as llama.cpp does: <tool_call> blocks of Qwen3-Coder XML replies, as from Qwen3-Coder, Qwen3.5 and StepFun3.5 (#1530), and <seed:tool_call> blocks of Seed-OSS replies (#1540). Nemotron 3 Nano's parallel-call grammar admits them too, where it used to admit only replies that parsed to no calls.
• ToolChoice.required now returns the tool call on DeepSeek-R1 distills, LFM2 and Functionary v3.2 instead of running to the length limit or returning the call as text: their tool-call grammars match llama.cpp's call framing, without a literal \n or a free-text LFM2 id, and DeepSeek-R1 takes one call unless parallelToolCalls is set (#1577). On Firefunction v2 it no longer makes the model write a free-text call id: its tool-call grammar, like llama.cpp's, takes only the call's name and arguments (#1565).
• Requests with an image or audio part and tools honor ToolChoice.auto instead of forcing a tool call on templates such as Qwen3-VL and Qwen3.5, and use the same tool-call grammar, triggers, parser and stops as text requests (#1463).
• A tool result whose first text is empty no longer renders a leading newline before the next text part on string-only chat templates, as llama-server does (#1468).
• A failed native stateLoadFile no longer lets the next prompt-prefix reuse skip evaluating the prompt on the cache llama.cpp cleared, which returned wrong scores and text without an error (#1491).
• Processes and isolates sharing a model cache root, such as sharedCache(), userSelected(...) or appGroup(...) directories, now download each entry once instead of returning a file another process is still writing, and never cache a file whose length differs from the one the server reported (#1444). A cache hit on a read-only model cache root returns the cached entry instead of reporting that no cached model is available.
• DefaultModelDownloadManager.remove, prune and clear no longer delete a cache entry another isolate or process is downloading: remove and clear remove the rest and throw ModelCacheEntryInUseException, and prune skips it (#1467). Overlapping remove, prune and clear calls on one cache root take turns instead of reporting each other's entries in use (#1490), and no longer fail or over-remove when they overlap in one process; a stray .REMOVAL directory in a case-insensitive cache root no longer keeps an entry reported in use (#1522); clear removes a stray .removal directory instead of reporting it in use forever (#1507); and on Windows a model file that can never be deleted, such as a read-only one, throws its FileSystemException instead of ModelCacheEntryInUseException (#1490).
• For an overridden native release, the assets.json.absent record, which notes that the release publishes no assets.json, is re-checked daily, or at once when it is dated in the future, and a record whose re-check fails or returns a page that is not JSON is kept and renewed instead of failing the build or retrying on every offline build; an assets.json that is truncated, empty, non-UTF-8 or not a manifest fails the build instead of being treated as unpublished; and a CUDA build that cannot reach its assets.json uses the cached CUDA sidecar instead of leaving CUDA out (#1490, #1506, #1522).
• The build hook reuses a cached llama.cpp bundle only while its extracted/ directory still holds what the cached archive beside it extracted to, and otherwise extracts that archive again, so a library replaced or added in the cache, such as a test stub, is no longer bundled (#1558). A cache from an earlier release is extracted again once, without network access. A cached native archive that matches its digest but lacks runtime libraries fails the build instead of being downloaded again on every build (#1490), and an incomplete cached core archive that matches its cached SHA256SUMS no longer tells an offline build to pre-seed SHA256SUMS (#1506).
• The image runtime's "not bundled" and other load-failure reasons name the iOS simulators by the build hook's bundle names, ios-arm64-sim and ios-x64-sim, instead of ios-arm64 and ios-x64 (#1565).
• The installation guide and support matrix now name libCLiteRTLM_mac.dylib as the library a sandboxed Flutter macOS app without the LiteRT-LM companion fails to load, instead of libLiteRt.dylib (#1565).