You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This commit was created on GitHub.com and signed with GitHub’s verified signature.
Update the default LiteRT-LM native runtime to leehack/litert-lm-native@v0.18.1-2: on SME2 CPUs (verified on Apple M4 and iPhone 16 Pro), CPU chat and audio embeddings are deterministic again (#1340) and Gemma 4 CPU decode is faster again (litert-lm-native#84; 1.56× on M4, 1.2× on iPhone 16 Pro), while its CPU prefill is 15–30% slower than v0.18.1. Output and speed on non-SME2 CPUs (Linux arm64, Windows x64) are unchanged. Image and audio embeddings shift slightly from core 0.16.0, so re-embed stored vectors of image or audio content; text-only vectors are unchanged. Flutter Apple apps get it with llamadart_litert_lm_flutter0.16.1.
Aligned the default WebGPU bridge assets to v0.1.72: Web repeat and presence penalties now count the prompt, as native does, so output for the same seed changes at the default penalty of 1.1, and tool chats (ToolChoice.auto) leave the prompt out, also as native does. In worker mode, Stop right after a generation starts now stops it, and a bare cancel() streams the text generated before the worker yields. Text containing U+0000 is no longer truncated in tokenize, embed, score and prompts; speech text containing it is rejected. Upstream v0.7.0@620d2225cc436ab0b001e3200e590e3b286ef9ab matches native v0.7.0-2, which the manifest records; Web LiteRT-LM stays @litert-lm/core@0.18.0. Manifest SHA-256: 85075d7d278bcacab706b79cda65c4a1bf8098a90f1e9fdbd504c198b8c8b03a.
llama.cpp draftDflash and draftDspark generation throws LlamaArgumentException instead of aborting the process when a LiliCorr draft block (dflash.block_size, 16 when absent) does not fit one draft micro-batch: SpeculativeDecodingConfig.draftMicroBatchSize, or the inherited ModelParams.microBatchSize when it is unset (#1330, #1341).
Fix hexadecimal string escapes in embedded and custom chat templates, including Flint's template (#1335).
Replace a stale embedded LiteRT-LM runtime in the macOS build phase after a core upgrade instead of keeping the old dylib until a clean build.