v0.14.0

lattice v0.14.0

Minor release. The workspace gains two crates, lattice-inference-core and lattice-tokenizer, split out of lattice-inference with every old import path kept as a re-export. Both HTTP servers now run requests through one serving facade chosen by the model's provider, lattice_serve serves a 32768-token window by default instead of the checkpoint's full maximum, and Qwen chat requests turn thinking off unless the request asks for it.

Breaking changes

No public item is removed at default features. cargo semver-checks against 0.13.0 passes for lattice-fann, lattice-transport, lattice-embed and lattice-tune. For lattice-inference it reports the items that moved into lattice-inference-core and lattice-tokenizer (InferenceError, the tokenizer types and functions, and the Gemma audio and image marker constants) as missing, because it does not follow re-exports of items defined in another crate; a crate that imports every one of those 68 paths from lattice-inference compiles. The changes that need action from callers are behavioural and listed below.

Behaviour changes

• Qwen thinking is off unless requested. A Qwen chat request that omits chat_template_kwargs.enable_thinking now renders with thinking off. A positive request reasoning_budget turns thinking on; a server-configured generation.reasoning_budget alone does not. Send enable_thinking: true to keep the old default (#2049).
• lattice_serve serves a 32768-token window by default. It sized its Metal KV cache from the checkpoint's max_position_embeddings (262144 for Qwen3.5) and reserved several GiB before the first request. The window is now min(model maximum, 32768); --max-context N sets it, and the KV cache, the max_tokens clamp, the tokenizer cap and the prompt-fits-window refusal all read that one value. A client that sent prompts longer than 32768 tokens to a default lattice_serve is now refused; --max-context 262144 restores the old window. Gemma 4 checkpoints refuse --max-context and keep their window. The startup line names the window and where it came from (#2082).

New crates

• lattice-inference-core holds InferenceError and has no dependencies. lattice-tokenizer holds the WordPiece, BPE, Gemma BPE and SentencePiece tokenizers, the shared tokenizer types and the incremental detokenizer. lattice-inference depends on both and re-exports every moved item at its old path, so downstream code compiles unchanged; a parity test imports each item through both paths (#2083, ADR-103 step 1).
• ADR-103 records how lattice-inference is split into focused crates behind an unchanged facade, and its first amendment states how each extraction clears semver checks (#2063, #2067). The roadmap carries the decomposition plan (#2062).

Serving

• Both HTTP binaries hold one library type, serving_host::ServingHandle, and neither matches on CPU or Metal to run a request (#2078). lattice serve asks the selected provider to open its route instead of choosing a backend and staging the load itself (#2080), and the selected provider owns the standalone server's model load (#2075). Model-specific serving construction lives in the provider modules (#2068), and the serving worker host moves into its own serving_worker module (#2071).
• The pooled embedding model moves under the Qwen provider and is served through a PooledEmbeddingService handle, used by lattice serve startup, /v1/embeddings, lattice router init and the routed-context embedding. Router admission also checks an artifact against the service's own loader and width; the loader identifier and existing messages are unchanged, so recorded router state still admits (#2089).
• The BERT text embedding model behind lattice_serve's /v1/embeddings moves under a BERT serving provider and is served through a TextEmbeddingHandle. Startup messages are unchanged, and the handler runs its token count, per-item window check, batch budget and admission in the same order (#2092).
• A chat request that sets enable_thinking: true now opens the generation turn with <think>\n, as the Qwen3.5 chat template does; omitted and false render as before (#2044).
• A reasoning-budgeted request closes the thinking block at the first line boundary after the budget, or at most 32 tokens later, instead of at the budget itself. The decode cap and answer budget are unchanged (#2047).

Memory and correctness

• A Q4 Qwen3.5 checkpoint on Metal no longer copies its whole embedding table into an f16 buffer at load. The .q4 file is mapped read-only and one row is decoded per token, bit-identical to the old copy. For a 248320 x 2560 vocabulary that removes 1.27 GB resident and close to 3 GB at load peak, which brings a 4B checkpoint under the budget of a 16 GB Mac. lattice doctor charges embed_tokens at its on-disk size (#2061).
• /v1/embeddings budgets image inputs before preprocessing them, as the chat vision route already did: the 2048 px side cap, 4096 patches (LATTICE_EMBED_VISION_MAX_PATCHES) and a 32 MiB preprocessed-input cap. An image over the budget is refused with HTTP 400 before anything is embedded (#2093).
• The tokenizer.json parser refuses nesting deeper than 64 levels with an error, where a deeply nested file used to exhaust the thread stack and abort the process (#2097).
• GdnStatePool::alloc called twice for a sequence that already holds a slot returns that slot instead of leaking one slot of capacity per call (#2040).
• gpu_test_lock() refreshes the shared lock file's timestamps with futimes, so a second local user who can open the file read-write no longer panics taking it (#2053).

Benchmarks and tooling

• bench-compare builds every bench executable before its arm's in-phase load sampler starts, so a build is not read as foreign load (#2065).
• Context-scaling reports keep their recorded timestamps, hardware and revisions in the TSV and print them on chart regeneration (#2054); a failed chart render no longer publishes over the previous outputs (#2059).
• The GPU and decode report consumer refuses invalid numeric inputs (#2055). An Ollama model is matched by exact name before a pull is skipped (#2057).
• The parity reference is published only after its checks pass, so a failed regeneration keeps the previous one (#2056).
• bench_serve_http also times POST /v1/embeddings (one text and a batch of eight) when BENCH_EMBEDDING_MODEL is set, and refuses a reply that measures nothing (#2086).
• scripts/public-api-listing.py lists every public item reachable from a crate root with re-exports resolved across crates (#2064), including glob re-exports of the crate's own modules (#2077).

Tests

• Characterization of the shared Metal serving worker (#2052), of the Qwen serving worker over HTTP on real checkpoints in the new serve_worker_live target (#2073), and of the standalone server's request and startup contracts (#2074). The standalone /v1/embeddings route is characterized on a synthetic checkpoint (#2084), and embedding-model startup through both serving binaries in the new serve_startup_embedding_golden target, with a test-utils synthetic Qwen3.5 checkpoint writer (#2085). The lattice binary's pooled /v1/embeddings route and the text embedding request checks are characterized ahead of the embedding service move (#2087).
• The Metal Qwen3.5 source guards run on every host from a portable integration test (#2042). The compact min-p test runs through the sampling wrapper (#2060). Disposable Git fixtures disable auto-gc (#2058).
• Test modules of metal_qwen35.rs and the JSON Schema compiler helpers move into their own files (#2046, #2048, #2050, #2051).

Dependencies

• taiki-e/install-action 2.87.26 to 2.87.28 in CI (#2045).
• The download feature's HTTP client moves from ureq 2 to ureq 3, clearing RUSTSEC-2026-0381, -0382 and -0383 (rustls-webpki 0.103.16, webpki-roots and webpki-root-certs 1.0.10). Downloads still ignore proxy settings in the environment, read bodies of any size, give up a connect after 30 s and follow at most 5 redirects. Download error messages now carry ureq 3's error text. (#2088)