You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
With llamadart_native_tag or llamadart_native_repository set, the build
hook keeps the release's SHA256SUMS beside the cached archive and verifies
a cached archive against it without network access, so offline builds work
again. When it cannot fetch SHA256SUMS, the build fails with NativeArchiveDigestException naming where to pre-seed it
(#1040).
Apple builds in a pub workspace now find the member app that depends on the
llama.cpp, LiteRT-LM or stable_diffusion companion, so they check and link
it as outside a workspace instead of bundling the hook's Apple assets beside
it; members with different companions get a build warning
(#1098).
cancelGeneration() during a Web generation now ends the stream normally,
as on native, instead of throwing LlamaInferenceException: on the WebGPU
worker path, which rejects with an AbortError, and on LiteRT-LM Web, whose
pending stream read rejects. Observers report the run as cancelled
(#875).
Fix getPerformanceContext() decode and prompt-eval times on a native
runtime without the llamadart exit-teardown exports (v0.5.0 or older),
where an asynchronous GPU backend such as Metal counted its work as sampling
time and decode speed read about 10x too high
(#671).
A negative ModelParams.gpuLayers now means all layers on every runtime,
as in llama.cpp, exactly like ModelParams.maxGpuLayers. Native llama.cpp
now also offloads the KV cache and reports the GPU backend instead of CPU, LiteRT-LM under ComputeDevice.auto uses its GPU backend instead
of the CPU, and the Web applies its GPU safety caps and CPU fallback
(#920).
Report responseFormat JSON schemas the grammar converter cannot represent,
such as an untyped property, a non-local $ref or one nested too deeply for
the stack, as LlamaUnsupportedException, and cyclic or other non-JSON
schemas as LlamaArgumentException, instead of StateError or a stack
overflow. A combined tool or schema grammar that llama.cpp would refuse to
load, such as about 2000 Hermes schema tools, is rejected with LlamaUnsupportedException before generation instead of a sampler
initialization failure
(#584, #1093).
Send tool results back on Gemma 2/3/3n and legacy Mistral templates, which
failed with Conversation roles must alternate user/assistant/...: when a
template that rejects the tool role fails such a request, it is rendered
again with each run of results as one user turn and the earlier calls as
JSON in the model turn
(#885).
Start a LiteRT-LM generation requested right after a cancel, on native and
Web, once the cancelled run stops, instead of failing with a wrapped StateError (generation is already in progress). An overlap with a
running generation that was not cancelled now throws LlamaStateException
(#656).
Aligned the default WebGPU bridge assets to v0.1.59, unchanged from 0.13.0:
they embed llama.cpp v0.6.0, are qualified against native v0.6.0, and
keep Web/native llama.cpp v0.6.0@d81235049384534c167caea52b85a694f6103d14
parity; Web LiteRT-LM is @litert-lm/core@0.18.0. Immutable Web asset
manifest: f823b10417b212ff1f66aa1c46bf78af7d9c393497dfcf0fa3a4f77adeb260eb.