Skip to content

v0.13.0

Choose a tag to compare

@github-actions github-actions released this 13 Oct 03:07
· 34 commits to main since this release

Decima-agent 2.2, a new family (d1, by Liquid AI), pulls by manifest digest, and a fix for decoders on CUDA that failed after an idle pause.

New model: d1 (#72)

Model By What it is Measured
d1:3b (d1) Liquid AI d1-3B on LFM2.5-VL-3B: the author's Q8_0 GGUF on llama.cpp with the author's prompt; each option scores the most likely of its answer-token forms, at the author's temperature 1.0 0.650 on typed-decisions, ECE 0.033; five questions in 48 ms in the runner on an RTX 4090
d1:3b-vision Liquid AI The same GGUF plus the author's Q8_0 projector: PNG image decisions, up to 16 images runtime parity only; no visual accuracy claimed
  • Parity. Against stock llama-server b11146 on the same device, with prompts and answer tokens from Liquid AI's pinned prompt.py: 650 text and 55 image questions on the CPU, an RTX 4090 and an RTX 5090, every decision the same, option log-probabilities within 7.7e-6 (the contributor's CPU and RTX 4070 too, on 29 and 24).
  • Cost. LFM2 keeps recurrent state, so every question evaluates the whole state again: the time grows with the state's length times the number of questions. With one 384 px image, one question takes 34 ms on the RTX 4090 and three take 104 ms.
  • License. The LFM Open License v1.0, the first license in the library with a revenue threshold: commercial use by an organization with US$10 million or more in annual revenue is not covered and needs an agreement with Liquid AI. ollaya show d1 --license prints it.
  • Weights come unmodified from LiquidAI/d1-3B-GGUF (2.9 GB, plus 583 MB for the projector).
  • Thanks @Txoka, who ported the author's prompt line for line and pinned it by hash.

Decima-agent 2.2 (#70)

decima:agent moves to A. M. Madani's Decima-agent 2.2 (Hub tag v2.2): the decisions inside a coding agent's loop, now with knowledge-graph judgments, and the author's option fitting from render.py. When a question and an option don't fit the 64-token option window, the option is kept whole and only the question's copy beside it is shortened. Before, a long question cut the options short, and two options could end up identical, with an answer of exactly 0.5.

  • Parity. Against the author's v2.2.0 code in fp32, on 591 questions, ten of them too long for the window: every token row and every decision the same, probabilities within 5.4e-6, on the CPU, an RTX 4090 and an RTX 5090.
  • The other tags. decima:base (still decima:latest) and decima:small are unchanged and give the 0.11.0 numbers. When cutting a question on the right would make two of their options identical, they now fit that question, as the author's current code does.
  • typed-decisions. 0.461 (2.1: 0.486), outside what 2.2 is trained for; the author measures 0.85 on 40 agent cases written after training (2.1: 0.83).
  • Needs 0.13.0. 2.2 has its own layout, so 0.12 and older refuse it at load instead of cutting its options the old way. To keep 2.1: ollaya pull decima:agent@sha256:c2391bf24ff36c4b2db3947f237a97377aaf7e3233d80a6eef1bb63de4fab0e3 (below).
  • Thanks @amyrmahdy, who spelled out the rendering change.

Pull a model by its digest (#64)

ollaya pull laya:en@sha256:<hex>, or {"model": "laya:en@sha256:<hex>"} on /api/pull, fetches that exact manifest, checks that it hashes to the digest before reading it or downloading anything, and stores it under laya:en, even after the tag has moved on. Without a tag, laya@sha256:<hex> is stored under that name.

  • What ollaya.dev keeps. Every manifest published under a tag from this release on, at /v2/<namespace>/<model>/manifests/sha256:<hex>, and the older ones whose files it still had, which covers every laya:en since the move to ollaya.dev.
  • Errors come before the stream. A digest the registry doesn't have is 404 MODEL_NOT_FOUND, a manifest that doesn't match it 502 DIGEST_MISMATCH, and nothing is stored either way.
  • In the CLI. ollaya show prints a model's digest (digest in /api/show), and ollaya list shortens a name pinned by digest alone to the digest's first 12 hex digits.
  • A digest pins a router's own manifest, not its targets: pin laya:en or laya:multilingual directly.
  • Thanks @maks3201 for asking.

Fixes

  • Decoders on CUDA after a pause (#69). kev and the other ONNX decoders with Scan layers, such as decider, decision, nimble, jeeves and clef, failed on the GPU with UpdateWithParentStream … Subgraph has nodes running on device … when a request came after about 10 s of idle, and could keep failing, or the runner crashed. ONNX Runtime keeps CUDA streams per calling thread and frees them when the thread exits, and the runner's request thread retired after 10 s idle, leaving the Scan bodies pointing at freed memory. The runner now loads the model and answers every request on one thread that lives as long as the runner. On Windows with an RTX 4090, up to 14 of 16 requests after 12 s pauses failed with 0.12.0; all 16 succeed now. Thanks @Endogen, whose report with the CPU and laya controls made it quick to pin down.
  • create checks the precision (#73). PARAMETER precision fp16 on a base without an fp16 graph (every ONNX model in the library except laya) created a model that could never load. ollaya create now refuses it with a 422. Thanks @kartsan03.

Upgrade

ollaya update

or curl -fsSL https://ollaya.dev/install.sh | sh (irm https://ollaya.dev/install.ps1 | iex on Windows). d1, decima:agent 2.2 and pulls by digest need 0.13.0 or newer.

What's Changed

  • create: reject a precision the base model lacks by @kartsan03 in #73
  • Add LiquidAI d1-3B Q8 text and image decisions by @Txoka in #72

Full Changelog: v0.12.1...v0.13.0