Skip to content

sidekick v0.13.0

Choose a tag to compare

@github-actions github-actions released this 20 Oct 02:51
· 232 commits to main since this release
72da724

sidekick v0.13.0 refuses over-length input to /v1/embeddings by default (D43). A request without truncate now gets a 400 naming the input, its token count and the model's limit, where 0.12 truncated the text silently and returned a normal vector. Send truncate: "END" to keep the old behaviour. This is the only change from 0.12.1, which added the field so clients could opt in first.

Upgrading

  • HTTP API: one breaking change. On /v1/embeddings, a request without truncate (or with null) is now NONE: an input longer than the model's limit is a 400 invalid_request_error, and the whole request is refused before any inference runs. To keep 0.12's behaviour, send truncate: "END" (any case) or true; the vectors are byte-identical to 0.12's. OpenAI's Python SDK sends the field through extra_body.
  • Each model's limit is max_seq_len in /v1/models (since 0.12.1), in tokens, counting special tokens and the model's prompt prefix.
  • Unchanged, still truncating by default:
    • /v2/embed, whose truncate defaults to END as in Cohere's v2 contract;
    • the C ABI's sk_embed, which has no truncate parameter;
    • the Rust API's Embedder::embed (EmbedLimits::default() stays END);
    • /v2/rerank, and the laya, gliner2, fev and nli classifier formats, which truncate by design.
  • The daemon and the dylib now differ for an over-length input. A host that uses the daemon and falls back to libsidekick.dylib should send truncate: "END" to the daemon to get the same vectors on both, or handle both behaviours.
  • Error text: the over-length message now tells the client to send truncate: END or START, on /v2/embed's NONE too.
  • Config keys, the C ABI and the Rust API: unchanged.
  • The release archives are the same four as 0.12.1, with the same bundled ONNX Runtime versions.

Why (D43)

No decision ever chose silent truncation: /v1/embeddings passed through the embedders' library default. A consumer embedding a 2,355-document corpus through Qwen3-Embedding-0.6B at 512 tokens found 36% of its documents over the limit, with vectors that agreed with full-length ones only down to cosine 0.855, while every gate graded A. D10 already refuses silent truncation of an embedding's dimensions, and the APIs sidekick's are compared with refuse over-length input too: OpenAI's /v1/embeddings answers a 400, and Hugging Face Text Embeddings Inference returns an error unless the request sends truncate: true. Within sidekick, /v1/rerank and text-classification models on /v1/classify already refuse unless the request asks to truncate.

The error says how to proceed:

input 1: 600 tokens, over model `bge-small-en-v1.5`'s limit of 512 tokens, and truncate is NONE (send truncate: END or START to truncate it)

What's Changed

Full Changelog: v0.12.1...v0.13.0