v0.13.0

lattice v0.13.0

Minor release with breaking changes, listed under "Breaking changes" with what a caller has to change for each. Most of them are in the learned LoRA router's serving API. DriftReport in lattice-transport and LRSchedule in lattice-tune become #[non_exhaustive].

The headline is time-series forecasting on the CPU: Reverso, Chronos-2 and Toto 2.0 run natively in lattice-inference behind a new forecast module and lattice forecast subcommand. The release also moves the EmbeddingGemma 2 vision tower onto Metal as an opt-in path, detects the model format before loading, and carries the learned LoRA router from recorded episodes through bounded candidate construction to held-out admission evidence, with nothing published or activated by default.

Time-series forecasting

• Reverso forecasting on the CPU (nano, small and full), the forecast module and the lattice forecast subcommand (#1962).
• Chronos-2 quantile forecasting on the CPU; lattice forecast picks the model family from config.json (#1983).
• Toto 2.0 forecasting on the CPU (#1984).
• ADR-101 records the design: one always-compiled forecast module with no new crate, dependency or feature, context preparation that follows numpy.interp arithmetic, and checkpoints converted to safetensors outside lattice with loading that refuses missing and leftover tensors (#1960).

EmbeddingGemma 2 vision on Metal

• An opt-in Metal path for the EmbeddingGemma 2 vision tower (macOS, metal-gpu); the default path is unchanged (#1940).
• Its MLP matmuls and attention run on two new simdgroup-matrix kernels, gemm_f32_sg and eg2_attention_sg_hd64 (Apple7 family or later), with a fallback to the existing kernels off those shapes (#1943). Each layer's attention block runs on the GPU in one command buffer (#1944).
• Shared-buffer copies refuse a count past the allocation in every build profile, not only in debug builds (#1942).
• A GEMM shape microbenchmark for the Metal matmul kernels (#1941).

Loading models

• Model-format detection reads safetensors headers and config.json before choosing a loader: ModelFormat gains MlxQuantized, Gguf and Refused, and inspect_format returns the reason for a refusal (#1957). An unreadable config.json does not hide hf_quant_config.json (#1959).
• Model downloads resolve at a pinned commit per repository instead of main, so an upstream file change no longer breaks first downloads against the crate's sha256 table (#1961).
• A model downloaded under a group-writable umask is loadable: the download creates directories 0755 and files 0644 (#1951).
• Q4 ingress admits a block with a negative scale; a zero or non-finite scale or bias is still refused (#1954).
• A Qwen config.json with a negative nested rope_parameters.rope_theta is refused instead of falling back to another theta (#1993).
• ADR-100 (Proposed) sets how lattice will load published quantized checkpoints (MLX quantized safetensors, then GGUF, then ONNX-format weights) without a foreign runtime (#1953).

Learned LoRA router

• Routed-completion episodes: the lattice server records what produced each routed completion (new router_episode module), and POST /v1/router/feedback judges a completion once by its id (#1949).
• lattice serve --router-learn, off by default: a router artifact's recorded learning policy is served only when learning is enabled, and every exploration draw is seeded from the completion id (mixture::seed_from_completion_id). lattice router init gains --tau and --exploration-scale (#1958).
• The router-learning feature, not in the default set, isolates the bounded router trainer; a CI probe checks that default and mixture builds contain no online-router training symbols (#1964).
• Bounded router candidate construction from recorded feedback, behind router-learning (router_refit): fitting diagnostics only, with no admission, artifact publication or activation (#1974).
• Router admission evaluates a registered held-out population before a candidate can qualify (ADR-096 decision 5, written out in #1963): replayed parent provenance, self-normalized reward ratios with a paired bootstrap lower bound, Metal teacher-forced reference scoring, population floors and an entropy guard. It produces admission evidence only; no state is published and no gate is activated. Oversized reference input is refused before tokenization (#1975).
• Router state publication is serialized and create-only, with staging synced outside the selectable namespace. Bootstrap commits its durable head before returning, format-5 pins require the canonical counter plus a hash, and headless or mixed directories refuse startup selection. See the lattice-inference changelog for the full rules and the deprecations (#1970).

Serving and the CLI

• lattice chat -p/--prompt <text>: generate once for the given text, write the reply to stdout and exit (#1905).
• --help for lattice chat, serve, doctor, prune-score, forecast and router ends with example invocations and the platform and checkpoint requirements each command enforces (#2027).
• A worker panic during a CPU streaming completion no longer ends the HTTP stream without its error event and [DONE]; the panic surfaces as "inference failed" (#1990).
• Batch admission at full capacity no longer bypasses FIFO order through a queued request treated as admitted, so later requests cannot stall on a stale slot (#1991).
• Decoder sessions always reach finish after a cancelled or failed prefill, and finish(Reusable) invalidates the live prediction on the Qwen and Gemma CPU sessions (#1902).
• LoRA load, unload and list orchestration is shared between lattice serve and lattice_serve. A load or unload body over the 1 MiB limit now answers 413 request_body_too_large on both servers instead of 400, and under lattice serve an unreadable load body answers 400 invalid_request instead of invalid_request_body (#1985).

CPU performance

• WordPieceTokenizer::tokenize_batch runs batches of more than 16 texts on the current rayon pool instead of one OS thread per CPU per call (#1929).
• Gemma 4 CPU prefill projects to vocabulary logits only for the last prompt token (#1965).
• Qwen CPU requests reserve the KV cache and attention scratch for their token budget before prefill, and decode writes each text delta into one reused buffer. On Qwen3.5-0.8B the warm decode region of a streaming request goes from 13 allocations and 17 reallocations to 0 and 3, with no measurable change in peak resident memory (#1966).
• The standalone Q8 generation path keeps its two macOS tile buffers for the whole request instead of allocating them on every projection (#1982).

Security

• rustls moves from 0.23.45 to 0.23.46 in Cargo.lock for RUSTSEC-2026-0346, -0347, -0348 and -0349. It reaches the workspace through ureq from lattice-inference; no other package moves (#1950).

Design records

• ADR-102 (Proposed) reorganises Metal execution around shared mechanisms: split metal_qwen35.rs first, then share helpers two models use, then migrate the remaining generation routes onto the backend-independent decoder, under a 3000-line file-size gate (#1994). It narrows ADR-087 D2 for those moves (#1997).
• docs/capability-matrix.md is re-derived at current code, with citations repointed to the split lattice binary (#1947).
• docs/models.md gains a Qwen3.6 support matrix (27B dense and 35B-A3B MoE) and corrects its Q8 MoE and serving rows (#2029).

Testing, CI and benchmarks

• Measurement lock files have their times refreshed on acquire, so macOS no longer deletes a held lock file from /tmp (#1903), with native regressions run on macOS (#1969).
• examples/bench_serve_prepare.rs holds the GPU measurement lock on the Metal worker thread for the worker's whole life (#1995).
• CI gates the combined Metal and mixture feature set (#1968) and BERT pooling parity across the native and HTTP embedding paths (#1987).
• Benchmark allocation reports warm before strict samples (#1980). The qwen_cpu_driver_allocations group compiles again under bench-internals,test-utils, with a CI step for that pair (#2011), and runs on a one-thread pool so its repeated counts agree on multi-core machines (#2012).
• The benchmark supervisor refuses in-phase arms on load the process walk cannot see (#1998), and bench-compare refuses a bench source that names no Criterion group literally, with its reason, instead of exiting silently (#2010).
• Grammar membership-scan diagnostics, opt-in under bench-internals (#1989). Linux HTTP benchmark spawns retry transient failures (#1978).
• Large inline unit test modules moved into their own files across all five crates (lattice-inference, lattice-embed, lattice-fann, lattice-transport, lattice-tune) and the two serving binaries (#1992, #2001, #2002, #2004, #2007, #2009, #2016, #2017, #2018, #2019, #2020, #2021, #2022, #2026, #2028, #2030, #2031). Test names are unchanged; a few modules stay inline where other tests read their file's source.
• CI caps maintained files at 3000 lines. Files already over the cap are listed in a ratchet manifest and may only shrink, with their recorded ceiling lowered in the same change (#2015).
• An integration test derives the Metal Qwen3.5 source population from its module graph and fails when a file under that tree is reached by no declaration; the existing source guards are not yet moved onto it (#2032).
• The measurement-lock source contract checks that the measurement worker returns the shared GPU-lock guard, hands it to the spawned worker unchanged and keeps it alive until the guarded work returns (#2033).
• Clippy covers lattice-inference with test-utils, with and without mixture, on Linux and with the Metal feature set on macOS (#2024).
• The WASM parity script fetches model config and tokenizer files at pinned revisions and checks them against the WASM model registry's SHA-256 values (#2023).
• The manual post-merge benchmark workflow gains a same-commit A/A calibration mode, and its detection resolution is stated as uncalibrated (#2025).
• The decode slope fit rejects incomplete or duplicated run, context and repeat records before fitting (#2034).
• The context-scaling benchmark's Ollama leg times eval_duration instead of total_duration, so model loading and prompt evaluation stay out of its slope even in a single cold run; older Ollama context-scaling timings need a fresh baseline (#2037).
• npm prebuild packages build only the Linux targets on pull requests (#2003).

Breaking changes

cargo semver-checks check-release --release-type patch against 0.12.1 (cargo-semver-checks 0.50.0) reports the changes below. It also reports two findings in lattice-inference that need no caller change:

• RouterArtifactError gains variants in the middle of the enum (#1970), so the implicit discriminants of twelve later variants change. Every variant carries fields, so no numeric cast of the enum compiles in either version, and the enum was already #[non_exhaustive].
• router_state::write_artifact, commit_head and load_latest are deprecated (#1970). They still compile, with a warning; the lattice-inference changelog names their replacements.

lattice-inference:

• router_init::bootstrap_policy takes the tau and the exploration scale, and BootstrapSpec gains tau and exploration_scale (#1958).
• ResolvedRouter::report takes the applied policy; RouterReport gains applied_policy and no longer derives Copy (#1958).
• ServingRouter::new and ServedRouter::new take learning: bool (#1958). ServingRouter::route and ServedRouter::route take the context vector by value and return RoutedRequest (#1949), and take the completion id (#1958).

lattice-transport and lattice-tune:

• DriftReport and LRSchedule are #[non_exhaustive] (#1945). Code outside the crate can no longer build a DriftReport with a struct literal, or match LRSchedule without a wildcard arm. Reading fields and building a schedule from its variants are unchanged.

Upgrading

Code that does not use the router serving types, build a DriftReport or match on LRSchedule needs no source changes. Behaviour changes that reach existing callers:

• LoRA load or unload bodies over 1 MiB get 413 instead of 400 on both servers (#1985).
• Router learning is served only with lattice serve --router-learn (#1958).
• Model downloads fetch from a pinned commit per repository, not main (#1961).