You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This commit was created on GitHub.com and signed with GitHub’s verified signature.
Add LlamaCppModelParams and LiteRtLmModelParams, ModelParams that take only the fields their runtime applies; the liteRtLm* arguments of ModelParams are deprecated in favor of LiteRtLmModelParams. Loads reject ModelParams fields the model's runtime would ignore, such as liteRtLm* fields on a GGUF model or preferMemory64 off the Web, and either type used for the other runtime, with LlamaUnsupportedException naming each field; LlamaBackend() also rejects these and LiteRT-LM platform gaps before any download when the model format is known (#858, #861).
Update the default LiteRT-LM native runtime to leehack/litert-lm-native@v0.18.1, built from upstream LiteRT-LM v0.18.1 with the ASR and stream finish-reason fixes of v0.18.0-3. Flutter Apple apps get it with llamadart_litert_lm_flutter0.16.0. Native GPU embeddings still throw LlamaUnsupportedException because EmbeddingGemma 2 still crashes in the prebuilt WebGPU accelerator.
Update the opt-in stable_diffusion runtime to leehack/stable-diffusion-native@v0.3.0, built from stable-diffusion.cpp master-982-0518b34: Windows opens model files with non-ASCII paths, and the legacy pickle reader no longer rejects checkpoints above 2 GiB (stable-diffusion-native#23). Images keep the flash attention and direct convolution settings ImageModelParams selects instead of the runtime's new automatic per-module choice. Flutter Apple apps need llamadart_stable_diffusion_flutter0.16.0; this core rejects earlier SD companions, whose sd_ctx_params_t is smaller.
Aligned the default WebGPU bridge assets to v0.1.70: a character split across tokens streams once complete instead of as a wrong one, and detokenize returns U+FFFD for invalid or incomplete bytes. Upstream v0.7.0@620d2225cc436ab0b001e3200e590e3b286ef9ab matches native v0.7.0-2, which the manifest records; Web LiteRT-LM stays @litert-lm/core@0.18.0. Manifest SHA-256: 7ac4e6e88205396f7221cd4e9e863bdf284df3498ced58cae2ffa1840996a8bf.
Native LiteRT-LM applies GenerationParams.penalty (other than the default), presencePenalty and, in chat on bundles with a thinking channel such as Gemma 4, thinkingBudget; penalties count only the tokens of the current send, capabilities reports each control, and a value it cannot apply throws LlamaUnsupportedException instead of UnsupportedError (#1155).
Add SpeculativeDecodingConfig.draftMicroBatchSize (llama.cpp --spec-draft-n-ubatch) to give native llama.cpp speculative draft contexts a smaller micro-batch and compute buffer than the target; unset keeps the target's, and WebGPU and LiteRT-LM reject it (#1124).
Templates that open a thought in the prompt: Seed-OSS, MiniMax M2, Apriel 1.5, Xiaomi MiMo and EXAONE MoE return it as reasoning until its end tag, as llama.cpp does, and stream it once instead of as content or twice; DeepSeek V3, Seed-OSS, MiniMax M2, Apriel 1.5 and EXAONE MoE also reason before a ToolChoice.required tool call and end the thought, so the call is no longer parsed as reasoning and lost (#742).
Fix the EXAONE MoE tool-call grammar emitting the tool name unquoted, so constrained tool calls are valid JSON (#1312).
Tool schemas from ToolDefinition.fromJsonSchema may list in required a property that only additionalProperties describes, as JSON Schema allows; the tool-call grammar requires that key instead of rejecting the schema (#1090).
Fix tool-loop rollbacks on ChatSession subclasses whose addMessage stores a copy of a tool result, adds a note before it or keeps the history length fixed: the rollback could leave an orphaned tool result or note behind (#1268).
Remove the WEBGPU_BRIDGE_PATCH_SAFARI_COMPAT and WEBGPU_BRIDGE_MIN_SAFARI_VERSION options of scripts/fetch_webgpu_bridge_assets.sh and the chat app's Safari patching of the bridge: the default assets already contain both fixes, and overrides to older asset tags no longer get them.