v0.12.0

lattice v0.12.0

Minor release with breaking changes, listed under "Breaking changes" with what a caller has to change for each. All of them are in lattice_inference::router_state, the on-disk LoRA router artifact API. Code that does not use that module compiles unchanged.

The headline is faster and smaller CPU embedding on Linux: a SentencePiece tokenizer that takes a third of the memory, an opt-in x86-gemm feature that cuts encoder CPU time per text by a third to two thirds on x86_64, and decode that splits single-row products across cores. The release also adds the EmbeddingGemma 2 vision tower on the CPU, the request and read-out layers for multi-slot decision models, and the next steps of the learned LoRA router.

CPU embedding on Linux

• The SentencePiece tokenizer stores its piece trie in flat arrays instead of a hash map per byte node, and with_max_seq_len shares the pieces and the trie instead of copying them. On paraphrase-multilingual-MiniLM-L12-v2 the tokenizer's live heap drops from 127.7 MiB to 38.8 MiB and the whole loaded model's from 148.2 MiB to 59.2 MiB, on an x86_64 Linux host measured after malloc_trim(0). Token ids and embeddings are bit-identical, and CPU time per text did not change beyond run-to-run spread (#1933).
• New x86-gemm feature on lattice-inference, forwarded by lattice-embed, off by default. On Linux x86_64 it sends every multi-row matmul_bt product to the gemm crate (pinned at 0.18.2) on the calling thread, with one packing allocation per call, and keeps single-row products on the existing kernels. Measured end to end through NativeEmbeddingService::embed on one core of an AMD EPYC (Zen 4) host, CPU time per text was 0.64x for all-MiniLM-L6-v2 and 0.65x for the multilingual MiniLM-L12 on a short text, and 0.34x to 0.40x on longer texts and batches. Embeddings stay within 2.4e-7 of the default build. The feature also moves decoder prefill onto gemm, so enable it where only encoders run. ADR-099 Amendment 1 records the scope (#1934).
• On x86_64, a single-row matmul_bt product with n * k >= 524,288 now splits its output columns across the current rayon pool, so CPU decode steps use more than one core. The split stays off below the threshold, on a one-thread pool, and inside a rayon worker. Each chunk runs the same kernel as before, and outputs are bit-identical (#1931).
• ADR-099 (Proposed) sets how lattice makes f32 CPU embedding fast on Linux x86_64 and aarch64: stock gemm first under a one-allocation-per-call exception, a per-phase threading policy, and CPU and memory budgets for the hosts it runs on (#1926).

EmbeddingGemma 2 images on the CPU

• EmbeddingGemma 2 checkpoints can now embed inputs that mix text and image parts. Images are validated, resized to the 16-pixel patch grid within the patch budget, and run through the vision tower in f32 on the CPU, then bridged into the text model. Vision weights load only when an image is embedded, so text-only use never reads them (#1906).
• Against the Hugging Face reference on a Linux x86_64 CPU, the normalized embedding differs by at most 1.2e-6 per element, with cosine at least 0.99999999998, on both test inputs (#1906).
• examples/dump_embeddinggemma2_vision.rs writes every stage to safetensors for comparison with a reference (#1906).

Decision models (System One)

• Qwen35Model::prefill_readout(input, slots, rows), documented as unstable: one prefill that returns logits at chosen positions over chosen lm_head rows, with each slot equal to what the single-position scorer computes for that prefix. PrefillInput is #[non_exhaustive] (#1915).
• serve::systemone, under the serve feature: request parsing, a byte-exact port of the decider-0.8b prompt layout, the checkpoint's own decider_config.json reader, a 255-entry decision label table (option_scoring::resolve_decision_labels), and answer assembly. It adds no HTTP route yet (#1914).
• ADR-098 is amended to match the published decider checkpoints (#1914, #1915).
• bench/decision-index/ holds a reference table for public decision models on Decision Index edition 0.3, with the scripts that produced it. No row runs lattice yet (#1907, #1909, #1919).

Learned LoRA router

• Router artifacts move to format 5 and record their lineage and admission policy. A state directory can commit a head that selects the served version, and --router-pin takes <version> or <version>:<hash-prefix>. Format 4 artifacts still load and stay reachable by pin (#1916).
• lattice router init provisions an empty router state directory with a deterministic, untrained bootstrap gate, and GET /v1/lora reports the gate's training_state (#1921).
• RouterPolicy, new in this release, records the admission policy including exploration_scale, and AdapterRouter::route_explored (behind the mixture feature) draws the served mixture from a Gaussian around the gate's output and records what is needed to score that draw. route_detailed returns the same record without a draw; route returns the same weights as before (#1928).
• lattice-fann (online-router feature) can update the gate from an explored mixture and refit one round with a leave-one-out baseline. Every update entry now refuses inputs that would write a non-finite value into the gate, and the gate write is all-or-nothing (#1917).

Serving fixes

• lattice serve --max-cpu-generations <N> (default 2) caps concurrent CPU chat generations. A request over the cap gets HTTP 503 server_busy immediately, and a client disconnect does not release the slot before the generation ends (#1927).
• A Qwen3.5 Q4 checkpoint on Metal now refuses logprobs and top_logprobs with 400 unsupported_feature during request preparation, instead of failing the request with 500 internal_error during generation (#1901).
• lattice_serve's POST /v1/lora/load and /v1/lora/unload validate the request before checking adapter support, so a malformed request gets its own error on a model without adapter support (#1899).
• A Qwen config.json with no text_config object is read from its top level when it carries num_hidden_layers and hidden_size, the shape a text-only Qwen3_5ForCausalLM checkpoint writes. Before, lattice substituted the Qwen3.6-35B-A3B MoE preset for such a file. A file with neither is now refused with a message (#1913).

lattice-tune

• train_grad_full --gradcheck refuses --load and --save, which it used to accept and ignore (#1894).

npm packages

• The npm release job publishes through npm trusted publishing instead of a stored token, and keeps provenance attestations (#1908).
• The native package's prepublish test loads the platform binding from the validated binaries (#1911).

Testing, CI and benchmarks

• The x86_64 embedding-drift job is now required by the parity gate, as the aarch64 job already was (#1922).
• CI runs train_grad_full's own argument-parser tests (#1896).
• The Metal measurement lock contract requires three generation targets to hold the GPU lock across their generate calls (#1900).
• The test module walkers resolve #[path] attributes relative to the declaring file, as the Rust Reference specifies (#1895).
• The benchmark supervisor protects the Cargo target directory a run actually uses, and refreshes the times of inherited lock files so macOS does not delete them from /tmp (#1897, #1904).

Breaking changes

All of these are in lattice_inference::router_state, which compiles in every feature set.

• resolve_startup(dir, pin: Option<u64>) is now resolve_startup(dir, pin: Option<&RouterPin>). Build the pin with RouterPin::parse, which takes <version> or <version>:<hash-prefix> (#1916).
• RouterArtifact has a new public field, provenance: RouterProvenance, so a struct literal needs it. Use RouterProvenance::Legacy for an artifact without recorded lineage (#1916).
• RouterArtifact and ResolvedRouter no longer implement Eq; they keep PartialEq (#1916).
• RouterArtifactError::UnknownFormat.known is a String instead of a u32, so the error can name both known formats (#1916).
• ROUTER_ARTIFACT_FORMAT is 5. ROUTER_ARTIFACT_FORMAT_LEGACY is 4, and format 4 artifacts still load (#1916).

cargo semver-checks check-release --release-type patch against 0.11.1 reports the provenance field, the two removed Eq derives, and shifted discriminants on RouterArtifactError, whose new variants sit before existing ones. Every variant of that enum carries fields and the enum is #[non_exhaustive], so no caller can depend on those values. The signature and field-type changes above are not covered by its lints and come from reading the diff. lattice-fann, lattice-transport, lattice-embed and lattice-tune report no breaking change.

Upgrading

Code that does not use router_state needs no source changes. Behaviour changes that reach existing callers:

• lattice serve on the CPU backend now refuses a third concurrent chat generation with 503 server_busy by default. Raise --max-cpu-generations on a host with spare cores and memory (#1927).
• A Qwen config.json without text_config loads its own geometry instead of the MoE preset, or is refused when it carries neither shape (#1913).
• Metal Qwen Q4 chat requests with logprobs get 400 instead of 500 (#1901).