sidekick v0.11.0

sidekick v0.11.0 makes ONNX Runtime use its AVX-512 kernels on Intel Macs (D42), making embeddings on a Mac Pro's CPU 1.2–1.7× faster. It also converts ModernBERT NLI models, adds ModernBERT-large-zeroshot-v2.0 with a licence flag, and publishes NLI grades for an Intel Mac's GPU.

Upgrading

• HTTP API, C ABI and config keys: unchanged.
• A new environment variable, SIDEKICK_ONNX_AVX512=0, turns the AVX-512 change off. It only has an effect on an Intel Mac with AVX-512.
• Rust API: additions only. The new module sidekick_onnx::avx512 has enabled and enable_on_this_thread. Nothing was removed or changed.
• Embeddings on AVX-512 Intel Macs change in the last digits. The AVX-512 kernels round differently from the AVX2 ones, so onnx-cpu vectors from 0.11.0 and 0.10.0 on the same Mac agree to within 1 − cos ≈ 2e-11, not bit for bit. Stored vectors don't need re-embedding. Other machines are unaffected.
• The release archives are the same four as 0.10.0, with the same bundled ONNX Runtime versions: 1.28.3, or 1.23.2 on Intel macOS.

AVX-512 on Intel Macs (D42)

ONNX Runtime uses its AVX-512 kernels only when the CPU register XCR0 shows the operating system saves AVX-512 state. macOS turns that state on lazily, per thread, the first time a thread runs an AVX-512 instruction. Until then XCR0 says AVX-512 is off, so on a Mac Pro or iMac Pro, ONNX Runtime has always run its AVX2 kernels. Newer ONNX Runtime releases check the same way, so a newer or source-built runtime wouldn't have fixed this.

sidekick now runs one AVX-512 instruction on the threads that load ONNX Runtime and create its sessions, so ONNX Runtime selects its AVX-512 kernels. Linux turns AVX-512 on for every thread from boot and needs none of this. Apple silicon is unaffected.

Measured on a Xeon W-3235 Mac Pro, through /v1/embeddings at about 290 tokens:

threads	bge-small	gte-modernbert	embeddinggemma-300m	Qwen3-0.6B
1	1.51×	1.57×	1.60×	1.39×
8 (the default)	1.17×	1.28×	1.29×	1.19×

• Each cell is how many times as long the AVX2 kernels took; at 16 tokens the gain is 2–11% at 8 threads.
• Eight threads is still the fastest setting, so the thread default stands.
• Every exported embedder, EmbeddingGemma 2 included, grades A on onnx-cpu with the AVX-512 kernels.
• docs/MODELS.md's Xeon thread table and grades are re-measured on an idle machine.

ModernBERT NLI models

• tools/convert_nli.py converts ModernBERT-family NLI models. It converts them through sidekick's ModernBERT backbone, with the checkpoint's own classification head.
• MoritzLaurer/ModernBERT-large-zeroshot-v2.0 grades B on the GPU, with buckets to 2,048 tokens, at 19 ms per request and 249 ms per pair at 2,048 tokens on an M1 Max. Its MNLI accuracy, entailment against not, is 0.961 matched and 0.955 mismatched, the best of the NLI models here, on fp32 and every Core ML path. It has no ANE build: the best ANE variant grades C.
• Licence: that model is trained partly on ANLI, which is licensed CC BY-NC 4.0. Whether that restriction applies to the weights is unsettled, so treat it as non-commercial unless you've cleared it. Its manifest and docs/MODELS.md say so, and the comparison page marks it "non-commercial?". The zeroshot-v2.0 -c models, trained only on commercially licensed data, stay the recommended set.

NLI on an Intel Mac's GPU

Graded on the same Mac Pro (AMD GPU, macOS 26.6.2):

• bge-m3-zeroshot-v2.0-c grades B and passes every gate. It costs 234 ms per pair at 1,024 tokens and 425 ms at 2,048.
• deberta-v3-base-zeroshot-v2.0-c, nli-deberta-v3-base and ModernBERT-large-zeroshot-v2.0 grade F on bucket invariance only. This is the Intel GPU behaviour described in 0.9.0. Within a bucket they're at A or B level, and their MNLI predictions match fp32 on every pair.
• roberta-large-mnli also grades F, and should be avoided on an Intel Mac. Its probabilities move by up to 0.08 between buckets, so its scores depend on input length there.

Also

• EmbeddingGemma 2 now has its Intel Mac onnx-cpu grade: A. Its Linux grades are still to come.

What's Changed

• feat(tools): ModernBERT-large-zeroshot-v2.0 (licence-flagged) and the page's licence badge by @ironblock in #86
• docs(data): NLI models graded on an Intel Mac's GPU by @ironblock in #87
• feat(onnx): AVX-512 kernels for ONNX Runtime on Intel Macs (D42) by @ironblock in #88
• chore: release v0.11.0 by @ironblock in #89

Full Changelog: v0.10.0...v0.11.0