Repository navigation
v0.13.0
Decima-agent 2.2, a new family (d1, by Liquid AI), pulls by manifest digest, and a fix for decoders on CUDA that failed after an idle pause.
New model: d1 (#72)
| Model | By | What it is | Measured |
|---|---|---|---|
d1:3b (d1) |
Liquid AI | d1-3B on LFM2.5-VL-3B: the author's Q8_0 GGUF on llama.cpp with the author's prompt; each option scores the most likely of its answer-token forms, at the author's temperature 1.0 | 0.650 on typed-decisions, ECE 0.033; five questions in 48 ms in the runner on an RTX 4090 |
d1:3b-vision |
Liquid AI | The same GGUF plus the author's Q8_0 projector: PNG image decisions, up to 16 images | runtime parity only; no visual accuracy claimed |
- Parity. Against stock
llama-serverb11146 on the same device, with prompts and answer tokens from Liquid AI's pinnedprompt.py: 650 text and 55 image questions on the CPU, an RTX 4090 and an RTX 5090, every decision the same, option log-probabilities within 7.7e-6 (the contributor's CPU and RTX 4070 too, on 29 and 24). - Cost. LFM2 keeps recurrent state, so every question evaluates the whole state again: the time grows with the state's length times the number of questions. With one 384 px image, one question takes 34 ms on the RTX 4090 and three take 104 ms.
- License. The LFM Open License v1.0, the first license in the library with a revenue threshold: commercial use by an organization with US$10 million or more in annual revenue is not covered and needs an agreement with Liquid AI.
ollaya show d1 --licenseprints it. - Weights come unmodified from LiquidAI/d1-3B-GGUF (2.9 GB, plus 583 MB for the projector).
- Thanks @Txoka, who ported the author's prompt line for line and pinned it by hash.
Decima-agent 2.2 (#70)
decima:agent moves to A. M. Madani's Decima-agent 2.2 (Hub tag v2.2): the decisions inside a coding agent's loop, now with knowledge-graph judgments, and the author's option fitting from render.py. When a question and an option don't fit the 64-token option window, the option is kept whole and only the question's copy beside it is shortened. Before, a long question cut the options short, and two options could end up identical, with an answer of exactly 0.5.
- Parity. Against the author's
v2.2.0code in fp32, on 591 questions, ten of them too long for the window: every token row and every decision the same, probabilities within 5.4e-6, on the CPU, an RTX 4090 and an RTX 5090. - The other tags.
decima:base(stilldecima:latest) anddecima:smallare unchanged and give the 0.11.0 numbers. When cutting a question on the right would make two of their options identical, they now fit that question, as the author's current code does. - typed-decisions. 0.461 (2.1: 0.486), outside what 2.2 is trained for; the author measures 0.85 on 40 agent cases written after training (2.1: 0.83).
- Needs 0.13.0. 2.2 has its own layout, so 0.12 and older refuse it at load instead of cutting its options the old way. To keep 2.1:
ollaya pull decima:agent@sha256:c2391bf24ff36c4b2db3947f237a97377aaf7e3233d80a6eef1bb63de4fab0e3(below). - Thanks @amyrmahdy, who spelled out the rendering change.
Pull a model by its digest (#64)
ollaya pull laya:en@sha256:<hex>, or {"model": "laya:en@sha256:<hex>"} on /api/pull, fetches that exact manifest, checks that it hashes to the digest before reading it or downloading anything, and stores it under laya:en, even after the tag has moved on. Without a tag, laya@sha256:<hex> is stored under that name.
- What ollaya.dev keeps. Every manifest published under a tag from this release on, at
/v2/<namespace>/<model>/manifests/sha256:<hex>, and the older ones whose files it still had, which covers everylaya:ensince the move to ollaya.dev. - Errors come before the stream. A digest the registry doesn't have is
404 MODEL_NOT_FOUND, a manifest that doesn't match it502 DIGEST_MISMATCH, and nothing is stored either way. - In the CLI.
ollaya showprints a model's digest (digestin/api/show), andollaya listshortens a name pinned by digest alone to the digest's first 12 hex digits. - A digest pins a router's own manifest, not its targets: pin
laya:enorlaya:multilingualdirectly. - Thanks @maks3201 for asking.
Fixes
- Decoders on CUDA after a pause (#69).
kevand the other ONNX decoders withScanlayers, such asdecider,decision,nimble,jeevesandclef, failed on the GPU withUpdateWithParentStream … Subgraph has nodes running on device …when a request came after about 10 s of idle, and could keep failing, or the runner crashed. ONNX Runtime keeps CUDA streams per calling thread and frees them when the thread exits, and the runner's request thread retired after 10 s idle, leaving theScanbodies pointing at freed memory. The runner now loads the model and answers every request on one thread that lives as long as the runner. On Windows with an RTX 4090, up to 14 of 16 requests after 12 s pauses failed with 0.12.0; all 16 succeed now. Thanks @Endogen, whose report with the CPU andlayacontrols made it quick to pin down. createchecks the precision (#73).PARAMETER precision fp16on a base without an fp16 graph (every ONNX model in the library exceptlaya) created a model that could never load.ollaya createnow refuses it with a422. Thanks @kartsan03.
Upgrade
ollaya update
or curl -fsSL https://ollaya.dev/install.sh | sh (irm https://ollaya.dev/install.ps1 | iex on Windows). d1, decima:agent 2.2 and pulls by digest need 0.13.0 or newer.
What's Changed
- create: reject a precision the base model lacks by @kartsan03 in #73
- Add LiquidAI d1-3B Q8 text and image decisions by @Txoka in #72
Full Changelog: v0.12.1...v0.13.0