Skip to content

v0.19.5

Choose a tag to compare

@github-actions github-actions released this 11 Nov 02:48
· 70 commits to main since this release
d04ff23
  • Keep the default WebGPU bridge assets at v0.1.83 (unchanged from 0.19.4).
  • Adopt native LiteRT-LM v0.19.1-3, whose LiteRtLm has the GPU accelerator built in: GPU generation is unchanged, the unused Metal accelerator no longer ships, the macOS hook bundle is 8.7 MB smaller and iOS apps 8.1 MB smaller, and the iOS LiteRT-LM frameworks no longer use the User Defaults required-reason API (#1798).
  • Native LiteRT-LM strict responseFormat now also runs on the macOS arm64 GPU, iOS arm64 CPU and GPU, and the Android arm64, Linux x64/arm64 and Windows x64 CPU, and with Gemma 4 (bundle template), which can think before its constrained reply. It accepts the JSON Schema subset that both llama.cpp's GBNF converter and LiteRT-LM's LLGuidance enforce (nested objects, arrays, $ref, anyOf, anchored patterns, uuid/date formats, bounds). Other schemas, including oneOf, throw LlamaUnsupportedException naming the schema path, and other hosts, Android, Linux and Windows GPUs and NPU throw it before generation. Qwen3 with thinking is refused, and a reply cut off inside the JSON reports finish reason length (#174).
  • getPerformanceContext() on native LiteRT-LM returns the counters of a generation cancelled mid-stream again, as before the 0.19 runtime, instead of null (#1903).
  • engine.getContextSize() on LiteRT-LM Web withTextHistory() loads returns the context size the runtime's executor enforces when its load probe finds one below the requested size, such as Qwen3 0.6B's 4096 on WASM CPU, instead of the requested ModelParams.contextSize (#1861).
  • Parse Qwen3-Coder XML tool calls as llama.cpp does: a call needs a line break after each tag and around each value (a CRLF or compact <tool_call><function= call is dropped), its required arguments come before its optional ones, and a string value keeps its whitespace and any </parameter> not followed by a line break. Nemotron 3 Nano now parses as Qwen3-Coder XML, so a <tool_call> also ends its thought, and the tool-call grammar of these templates requires that shape and starts at <tool_call> (#1898).
  • Parse Seed-OSS tool calls closer to llama.cpp: a finished reply whose last <seed:tool_call> scope is never closed keeps a call complete through </function> and drops one cut off before it, an argument declared as a string keeps its value as generated instead of being read as JSON (\n42\n stays a string), and with ToolChoice.required the grammar lets a thought and text come before the calls (#1898).
  • Parse a MiniMax M3 reply to a forced-open thought as llama.cpp does: the thought ends at the first </mm:think> and any later <mm:think> or </mm:think> is content, so a reply such as </mm:think></mm:think><mm:think> streams the same whole or split by character, with ToolChoice.none too (#1905).
  • Support the Ling 3.0 (Bailing V3) chat format as llama.cpp does: its <tool_call>name calls with <arg_key>/<arg_value> arguments no longer parse as Hermes JSON, a <tool_call> ends a forced-open thought, a Hermes JSON call stays content, and a trailing <|role_end|> is consumed. A string argument is the raw text up to </arg_value>; any other is JSON (#1857).

Runtime baseline: native llama.cpp v0.9.0 and Web use upstream v0.9.0@cd959bae99ff4bed22bef2604f45241c09bbb8c8; native LiteRT-LM uses v0.19.1-3 and stable-diffusion uses v0.3.0-3. Web LiteRT-LM uses @litert-lm/core@0.19.0. Manifest SHA-256: ae6f488408672f9ba28d8f9e664a2b85f4e12048f290e259e18dd0b750c038e5.