sidekick v0.16.0

sidekick v0.16.0 publishes converted builds on Hugging Face and installs them with one command: sidekickd fetch bge-small-en-v1.5 downloads a graded build, checks every file against sidekick's checksum ledger, and installs it ready to serve. The first published builds are bge-small-en-v1.5 and Qwen3-Embedding-0.6B, each in Core ML and ONNX; Qwen3-Embedding's Core ML build now takes inputs up to 2,048 tokens. The daemon also compiles Core ML models ahead of requests, so a model's first request after an install, upgrade or macOS update no longer waits for that compile, and the macOS archives' sidekickd keeps one code-signing identity, so upgrading no longer recompiles every Neural Engine model. The docs are reorganised around one page per task, with a shorter README. Existing requests behave as in 0.15.0.

Upgrading

• The first start after this upgrade compiles every Neural Engine model once more. From this release, the macOS archives' sidekickd is signed with a fixed identifier, io.github.ironblock.sidekickd. Core ML keys each compiled ANE program to the identifier of the process that loads it; until now that was cargo's hashed name, which changed with every version, so every upgrade recompiled every ANE bucket. It changes once more now, and not on later upgrades. If you build sidekickd yourself, run tools/sign_sidekickd.sh target/release/sidekickd after cargo build (docs/INSTALL.md).
• The daemon now compiles Core ML models at startup, in the background (D52). A child process loads each bucket the daemon will serve, one at a time, so Core ML compiles it before a request needs it; nothing stays loaded, so memory and idle unloading are unchanged. Requests are served meanwhile: one that needs the bucket being compiled waits for that compile, then loads it from the cache. To keep the old behaviour (compile on first request), set warm_on_start = false or pass --no-warm; to leave out one model, [models."<id>"] warm = false. warm_budget_secs (default 1,800, 0 for no limit) stops starting new compiles after that long.
• HTTP API: additions only. /v1/models lists each Core ML bucket's compile_cache state (warm, warming, cold or failed), and /health has a warmup object with the warm-up's progress and each bucket's load times. A model capped by its manifest's new cpu_max_seq_len says so in seq_cap.reason.
• Command line: new subcommands. sidekickd fetch and sidekickd warm, below. A bare sidekickd still serves.
• Config keys: warm_on_start, warm_budget_secs and [models."<id>"] warm, all optional. docs/CONFIG.md now lists every key, flag, environment variable and per-OS default, and a test keeps it complete.
• Manifests: one new optional key. cpu_max_seq_len lowers a Core ML model's CPU length cap (D33's 1,024) for a model whose buckets already differ on the CPU below it. Qwen3-Embedding-0.6B's manifest sets 512. A daemon before 0.16.0 ignores the key and serves that model's CPU path to 1,024 tokens, where its buckets disagree, so serve the published Qwen3-Embedding Core ML build with 0.16.0 or later.
• Qwen3-Embedding-0.6B's Core ML build goes to 2,048 tokens. The committed manifest lists buckets 128, 256, 512, 1,024 and 2,048, and the converter now converts a manifest's own buckets. On the ANE and the GPU, inputs past 512 tokens are now embedded whole; the 512-token build refused them (D43's 400) or cut them (truncate), so their vectors differ. On the CPU it serves up to 512 tokens.
• C ABI: unchanged. SK_ABI_VERSION is the same. A host embedding libsidekick.dylib is cached by Core ML under its own process's identity; docs/INTEGRATING.md explains what that means for a host's upgrades.
• Rust API: additions, with one source-level change. sidekick_embed::warm is new. ModelManifest and ClassifierManifest gain a public cpu_max_seq_len field, so code that builds either with a struct literal must name it. sidekick_core::cpu_seq_cap takes the manifest's cpu_max_seq_len.
• The release archives are the same four as 0.15.0, with the same bundled ONNX Runtime versions. Their README.md and INTEGRATING.md now link to this release's files on GitHub instead of relative paths that didn't resolve once unpacked.
• Docs have moved. Links into the old README's sections now land on its top. Each topic has its own page: INSTALL, RUNNING, CONFIG, API, BACKENDS and DEVELOPMENT, beside the existing ones (below).
• If you run sidekickd as a LaunchAgent, the documented plist now sends standard output, where the daemon logs, to ~/Library/Logs as well as standard error. The old one logged standard error only, under /tmp (docs/INSTALL.md).

Published builds on Hugging Face (D50, D51)

Converting a model for the Neural Engine takes a pinned toolchain, gigabytes of downloads and, for a large model, hours. sidekick now publishes the builds it has graded, under the sidekickd organisation, one repository per model and backend.

model	Core ML	ONNX
bge-small-en-v1.5	sidekickd/bge-small-en-v1.5-coreml: fp16, 128–512 tokens; ANE, GPU and CPU all grade A	sidekickd/bge-small-en-v1.5-onnx: fp32; grade A
Qwen3-Embedding-0.6B	sidekickd/qwen3-embedding-0.6b-coreml: fp16, 128–2,048 tokens; ANE A, GPU A, CPU B (to 512 tokens)	sidekickd/qwen3-embedding-0.6b-onnx: fp32 (default) and fp16-weights, 4,096 tokens; both grade A

• A repository holds one directory per build, exactly as sidekick installs it, plus a checksum file per build, the model's licence text (and its NOTICE, where upstream has one), and a model card. Core ML builds are compiled programs (.mlmodelc), and their buckets share one weights file per chunk (D49), so Qwen3-Embedding's five buckets download as 1.2 GB, not 6.1 GB.
• The model card holds only what is particular to the model: the builds and their grades on each compute path, the licence and training data, the original model's scores, and every file's sha256. Everything else (how fetch works, what a grade means, how to reproduce a build) links to this release's docs. docs/PUBLISHING.md describes the card.
• Which models can be published depends on their licences and their training data's, as recorded in docs/data/licences.toml for all 28 models in the registry. A model whose training data carries non-commercial terms, such as MS MARCO for Qwen3-Embedding, is published with a notice at the top of its card, not behind a gate: whether such terms bind a model trained on the data is unsettled, the user decides whether their use is permitted, and the model's own licence is unchanged. Only terms that must be passed on to every recipient (the Gemma Terms of Use) put a repository behind Hugging Face's access gate; a model whose licence doesn't allow redistribution (LFM2.5-Embedding) isn't published. docs/PUBLISHING.md gives the policy.
• The checksum ledger, docs/data/ledger.json (tools/ledger.py), records every build sidekick has converted, published or not: every file's sha256 and size, what produced it (converter commit, coremltools, PyTorch, OS, Xcode, chip), the grades of the reports that graded those exact files, and where it is published. MODELS.md's "Published builds" table is generated from it.
• The comparison page (https://ironblock.github.io/sidekick/) gains Licence and Training data columns, a licence filter, and a "Leave out" filter for, among others, models with non-commercial training data.

sidekickd fetch (D51)

sidekickd fetch bge-small-en-v1.5                       # Core ML on macOS, its published build
sidekickd fetch qwen3-embedding-0.6b --backend onnx     # fp32; --variant fp16-weights for half the download
sidekickd fetch qwen3-embedding-0.6b --buckets 128,512  # only the buckets you'll use

• Download: with the Hugging Face CLI (hf, 1.0 or later) installed, fetch runs it, with your login, Xet transfers and HF_* settings. Without it, fetch downloads anonymously with the system's curl into the same cache, in the layout hf uses, so either can reuse the other's files. --no-hf uses curl even when hf is installed.
• Verification: every file is checked against the Hub's hash, then against the checksum ledger built into sidekickd. The installed files must match one recorded build of that model, backend and variant, or nothing is installed. The repository, revision and variant default to the build's latest published record; --repo and --revision fetch from elsewhere, verified the same way.
• Install: the files are hard-linked from the download cache into the models directory, never symlinked (Core ML can lose the Neural Engine on some builds through a symlink, docs/design/coreml-cache.md), so the files on disk are stored once. A gated repository's terms are shown, and fetch stops until you pass --accept-terms <model>.
• Then, fetch compiles the Core ML buckets it installed (D52; --no-warm skips this). The daemon reads its models directory at startup, so restart it to serve the new model.

Compiling ahead of requests (D52)

A Core ML bucket's first load in a process compiles it for the device: a few seconds on the ANE for a small model, a minute or more for a long bucket of a large one. The cache that holds the result is keyed to the executable, its code-signing identity, the home directory, the model's path and its files, and the macOS build, so a new install, a moved model or a macOS update starts cold.

• When: at startup in the background, after sidekickd fetch, and on demand: sidekickd warm [<model>…] [--buckets …] [--budget-secs N] [--list].
• What: exactly the buckets the daemon serves, after the length caps (D33, D48) and the operator's compute units (D38).
• How: in a child process running the daemon's own executable, so it fills the cache requests read; one compile at a time, since Core ML's Neural Engine compiles run one at a time across the whole machine; buckets not yet compiled first, smallest first.
• Measured on an M1 Max with macOS 27.0: bge-small-en-v1.5's first request per bucket on the ANE drops from 4.0–5.4 s to 0.9–1.3 s, and an NLI model's ANE bucket load from 2.1–3.8 s to 25 ms.

Core ML's compile cache, documented (docs/design/coreml-cache.md)

An audit of Core ML's compile cache on macOS 27.0 corrected several things the docs said. The measured findings now have one page, which the decision records cite:

• The cache key includes the code-signing identifier (for the ANE), the compute units, and each weights file's inode and birth time; a model copied to a new path compiles again.
• The ANE's compiled programs live in a system cache that can't be deleted, about one copy of the weights per compile. Deleting a home's cache frees the GPU and CPU programs only.
• ANE compiles run one at a time across the whole machine, so a long compile delays every app's first ANE load.
• A symlinked model loses the ANE only on Core ML's older loader, and only through a link that leaves the program's directory; fetch and the registry use hard links regardless.
• macOS 27's Core AI framework also compiles ahead of time, but on this OS release its device step crashes and falls back to a full compile, so it saves no time yet (#163).

Docs

• The README is now a front page: what sidekick is, a quickstart from the release archive, and where to go next, with the comparison page prominent.
• New pages: INSTALL (archives, platforms, ONNX Runtime, building, signing, the LaunchAgent), RUNNING (the models directory, installing models, first loads, memory and disk, logs), CONFIG (every setting), API (routes, authentication, errors, length and truncation, prompts), BACKENDS (compute units and the length caps, ONNX, the static tier, Foundation Models) and DEVELOPMENT.
• Corrected: the API key guards every route except /health, not only /v1 (D13 amended; the behaviour is unchanged); the models directory holds one program per bucket; MODELS.md lists Qwen3-Embedding-0.6B among the validated models; CONVERTING.md lists every converter.
• CI now checks every relative link and anchor in the docs.

Measurements and tooling

• A release gate (tools/release_gate.py) runs before every tag. It serves each model with a build on the release Mac through the release candidate's sidekickd, runs its task sample over HTTP (MultiNLI pairs, an MTEB slice, the parity corpora, the reranker set), checks each build against the ledger, and compares every number with a committed baseline (docs/data/release-gate.json), which covers 24 models. This release passed it.
• The parity suite grades Core ML's CPU path under D33's 1,024-token cap, as the runtime serves it (D33 amended). Some models' CPU grades may change at their next regrade.
• Converters take --keep-packages DIR to keep each bucket's .mlpackage, and ONNX parity reports record their files' hashes, so the ledger can attach their grades.

What's Changed

• feat: publishing prep: checksum ledger, licence tiers, model cards (#136, #137) by @ironblock in #157
• feat: sidekickd fetch (#137, D51) by @ironblock in #158
• feat(tools): pre-tag release gate (#156) by @ironblock in #159
• feat: prepare bge-small-en-v1.5 for its first Hugging Face publication by @ironblock in #160
• feat(tools): model cards hold only what is particular to the model by @ironblock in #161
• feat(publishing): ship each model's licence text as LICENSE (and NOTICE) by @ironblock in #165
• fix: correct Core ML cache facts, and sign sidekickd with a stable identifier by @ironblock in #166
• feat: warm Core ML's compile cache at startup, on demand and after fetch (D52) by @ironblock in #162
• docs: restructure the docs around one home per fact (0.16.0) by @ironblock in #168
• feat(models): publish Qwen3-Embedding-0.6B (Core ML to 2,048 tokens, ONNX) by @ironblock in #167
• fix(publishing): gate only terms that must travel; non-commercial training data is a notice by @ironblock in #169
• feat(pages): licence and training-data columns and filters on the comparison page by @ironblock in #170
• feat(tools): release gate keeps a persistent home; baseline covers 24 models by ledger build (#156) by @ironblock in #171
• chore(release): 0.16.0 by @ironblock in #172

Full Changelog: v0.15.0...v0.16.0