Skip to content

v0.16.0

Choose a tag to compare

@github-actions github-actions released this 23 Oct 09:42
· 249 commits to main since this release
aee2459
  • Add LlamaCppModelParams and LiteRtLmModelParams, ModelParams that take only the fields their runtime applies; the liteRtLm* arguments of ModelParams are deprecated in favor of LiteRtLmModelParams. Loads reject ModelParams fields the model's runtime would ignore, such as liteRtLm* fields on a GGUF model or preferMemory64 off the Web, and either type used for the other runtime, with LlamaUnsupportedException naming each field; LlamaBackend() also rejects these and LiteRT-LM platform gaps before any download when the model format is known (#858, #861).
  • Update the default LiteRT-LM native runtime to leehack/litert-lm-native@v0.18.1, built from upstream LiteRT-LM v0.18.1 with the ASR and stream finish-reason fixes of v0.18.0-3. Flutter Apple apps get it with llamadart_litert_lm_flutter 0.16.0. Native GPU embeddings still throw LlamaUnsupportedException because EmbeddingGemma 2 still crashes in the prebuilt WebGPU accelerator.
  • Update the opt-in stable_diffusion runtime to leehack/stable-diffusion-native@v0.3.0, built from stable-diffusion.cpp master-982-0518b34: Windows opens model files with non-ASCII paths, and the legacy pickle reader no longer rejects checkpoints above 2 GiB (stable-diffusion-native#23). Images keep the flash attention and direct convolution settings ImageModelParams selects instead of the runtime's new automatic per-module choice. Flutter Apple apps need llamadart_stable_diffusion_flutter 0.16.0; this core rejects earlier SD companions, whose sd_ctx_params_t is smaller.
  • Aligned the default WebGPU bridge assets to v0.1.70: a character split across tokens streams once complete instead of as a wrong one, and detokenize returns U+FFFD for invalid or incomplete bytes. Upstream v0.7.0@620d2225cc436ab0b001e3200e590e3b286ef9ab matches native v0.7.0-2, which the manifest records; Web LiteRT-LM stays @litert-lm/core@0.18.0. Manifest SHA-256: 7ac4e6e88205396f7221cd4e9e863bdf284df3498ced58cae2ffa1840996a8bf.
  • Native LiteRT-LM applies GenerationParams.penalty (other than the default), presencePenalty and, in chat on bundles with a thinking channel such as Gemma 4, thinkingBudget; penalties count only the tokens of the current send, capabilities reports each control, and a value it cannot apply throws LlamaUnsupportedException instead of UnsupportedError (#1155).
  • Add SpeculativeDecodingConfig.draftMicroBatchSize (llama.cpp --spec-draft-n-ubatch) to give native llama.cpp speculative draft contexts a smaller micro-batch and compute buffer than the target; unset keeps the target's, and WebGPU and LiteRT-LM reject it (#1124).
  • Templates that open a thought in the prompt: Seed-OSS, MiniMax M2, Apriel 1.5, Xiaomi MiMo and EXAONE MoE return it as reasoning until its end tag, as llama.cpp does, and stream it once instead of as content or twice; DeepSeek V3, Seed-OSS, MiniMax M2, Apriel 1.5 and EXAONE MoE also reason before a ToolChoice.required tool call and end the thought, so the call is no longer parsed as reasoning and lost (#742).
  • Fix the EXAONE MoE tool-call grammar emitting the tool name unquoted, so constrained tool calls are valid JSON (#1312).
  • Tool schemas from ToolDefinition.fromJsonSchema may list in required a property that only additionalProperties describes, as JSON Schema allows; the tool-call grammar requires that key instead of rejecting the schema (#1090).
  • Fix tool-loop rollbacks on ChatSession subclasses whose addMessage stores a copy of a tool result, adds a note before it or keeps the history length fixed: the rollback could leave an orphaned tool result or note behind (#1268).
  • Remove the WEBGPU_BRIDGE_PATCH_SAFARI_COMPAT and WEBGPU_BRIDGE_MIN_SAFARI_VERSION options of scripts/fetch_webgpu_bridge_assets.sh and the chat app's Safari patching of the bridge: the default assets already contain both fixes, and overrides to older asset tags no longer get them.