sidekick v0.12.1

sidekick v0.12.1 lets a client refuse over-length input to /v1/embeddings instead of having it silently truncated, lists each model's input limit in /v1/models, and raises Qwen3-Embedding-0.6B's ONNX build from 512 to 4,096 tokens, graded A on a new long-input parity corpus. It prepares for 0.13.0, where over-length input to /v1/embeddings becomes a 400 by default.

Upgrading

• HTTP API: additions, one behaviour change.
  • /v1/embeddings takes an optional truncate: NONE, START or END in any case, or TEI's boolean (true means END, false means NONE). Without it, behaviour is unchanged: over-length input is truncated, keeping the start.
  • Behaviour change: truncate used to be ignored on /v1/embeddings, so a client that sent truncate: false or "none" got a truncated vector and a 200. It now gets a 400 for over-length input, which is what those values ask for. true and "end" behave as before.
  • /v1/models lists each embedder's and classifier's max_seq_len, and each Core ML embedder's buckets, after the CPU cap (D33).
• Plan for 0.13.0: /v1/embeddings will refuse over-length input with a 400 by default, as OpenAI's embeddings API and Hugging Face Text Embeddings Inference do, and as sidekick's /v1/classify and /v1/rerank already do. To keep truncation, send truncate: "END" (or true) now.
• Error messages: an over-length error names the input's index in the batch, its token count, the model and its limit, or max_tokens when the request set it. The whole request is refused, and no inference runs.
• Qwen3-Embedding-0.6B on onnx-cpu now takes up to 4,096 tokens. An input over 512 tokens embeds differently from 0.12.0, which truncated it at 512. The model's Core ML build still stops at 512, so the two builds now give different vectors for long text: re-embed long documents, and don't mix the builds in one index.
• C ABI and config keys: unchanged. sk_embed still truncates.
• Rust API: additions only. sidekick_onnx::MemoryOptions and SHRINK_FROM_TOKENS, OnnxModel::load_with, the ONNX embedder's load_with, and EmbedInput::check_limits. Nothing was removed or changed.
• The release archives are the same four as 0.12.0, with the same bundled ONNX Runtime versions.

Over-length input to /v1/embeddings

/v1/embeddings has always truncated an input longer than the model's limit, with nothing in the response to say so, and the README didn't mention it. A consumer embedding a 2,355-document corpus (median 375 tokens, longest 2,408) through Qwen3-Embedding-0.6B's 512-token ONNX build found 854 documents (36%) truncated. Against full-length vectors, the truncated ones fell as low as cosine 0.855, while the in-limit ones agreed to 0.99999. Every published gate graded the model A, because no gate tested past about 520 tokens.

• truncate: "NONE" refuses any input over the limit with a 400, including on a model whose own tokenizer.json truncates (LFM2.5-Embedding-350M, at 512).
• The README now documents the default, the limit, and where to read it.

Qwen3-Embedding-0.6B at 4,096 tokens on onnx-cpu (D40)

• Grades: A on the main corpus (worst 1 − cos 1.7e-11, unchanged) and on a new long-input corpus at 600, 1,500, 2,400 and 4,096 tokens (1 − cos ≤ 1.2e-11), with no rank flips.
• The long-input corpus, fixtures/parity/corpus_long.toml, has its own hash and references. Existing models' references are unchanged, and the parity suite grades it for models whose limit reaches 600 tokens.
• Cost at length (M1 Max, 8 threads): about 0.9 s per input at 512 tokens, 7 s at 2,400 and 13–14 s at 4,096. The process's footprint is about 1.8 GB above its pre-load size once loaded, and peaks at about 2.6 GB above it at 2,400 tokens and 4.2 GB at 4,096. Long inputs suit background indexing, not interactive requests.
• Memory is given back: a run of 1,024 tokens or more now ends by releasing ONNX Runtime's arena growth, so after later inputs the footprint returns near its loaded size instead of staying at the peak. This applies to every ONNX model; shorter runs are unchanged.
• Timeouts: four 4,096-token inputs in one request can exceed the default 60 s request_timeout_secs, and the work continues after the 504. Send one or two long inputs per request, or raise the timeout.

Fixes

• START truncation on a model whose tokenizer.json truncates (LFM2.5) kept the start of the text instead of its end (#114). On a text of two paragraphs, the vector's cosine to the second went from 0.139 to 0.991.
• The static embedder tokenizes every input before embedding any, so one bad input refuses the whole request, as on Core ML and ONNX.

What's Changed

• feat(embeddings): truncate on /v1/embeddings, max_seq_len and buckets on /v1/models by @ironblock in #113
• feat(onnx,parity): Qwen3-Embedding-0.6B's ONNX build at 4,096 tokens, graded on a long-input corpus by @ironblock in #115
• chore(release): 0.12.1 by @ironblock in #119

Full Changelog: v0.12.0...v0.12.1