Repository navigation
SuperEmbedder 1.0.0
SuperEmbedder turns text into embedding vectors whose bytes are identical on every certified build and machine.
1.0.0 is the first release. It runs Qwen3-Embedding-0.6B on the SuperSLM 1.18.1 integer engine, and its banks
are searched by SuperFAISS.
Downloads
| File | SHA-256 |
|---|---|
qwen3-embedding-0.6b-1p5.sslm (model, 634,489,808 bytes) |
0be28bf42637264df85481eb925c46f674da7db97fd7601c6732881ad8f99fbf |
qwen3-embedding-0.6b-tokenizer.sslm (tokenizer, 3,465,492 bytes) |
2ba15e621e28f4f58046a89a4e4b1e3e6ea8c4e1a4f5397238634b589082b9d3 |
Check the digests before use. A model you convert yourself is certified only if its bytes equal these.
What 1.0.0 guarantees
The same text, encoded by two installations whose encoder_fingerprint is equal and whose builds are members
of the certified set that fingerprint's certification_epoch names, produces byte-identical vectors. The
certified set is what was tested: two cells — a cell is a build at one CPU tier — with their one pair proven,
on an AMD Ryzen 9 5950X:
| Cell | OS | Toolchain | Dispatch | CPU tier | Certified on |
|---|---|---|---|---|---|
msvc19.33-auto@avx2 |
Windows x64 | MSVC 19.33.31631 | auto | AVX2 | AMD Ryzen 9 5950X |
clangcl18.1.8-auto@avx2 |
Windows x64 | clang-cl 18.1.8 (MSVC frontend 19.33.31631) | auto | AVX2 | AMD Ryzen 9 5950X |
Not yet certified (planned for a later update): AVX-512 cells; forced-SSE2 builds; MSVC 19.44 and GCC 13.3
(Linux) builds; NVIDIA GPUs at 1.18.1. Adding cells keeps the certification epoch, which digests only the
engine pin and the proof corpus, so banks baked with 1.0 stay valid when they arrive.
An auto build on a host whose best tier is AVX-512 or SSE2 runs uncertified in 1.0: it works, but a bank baked
by a certified cell is refused there by name (SEMB_FINGERPRINT_MISMATCH). Check this before you ship a
baked bank to players.
Run semb --version to see whether your build is a certified cell on your host. The full statement, and what
it does not cover, is in docs/DETERMINISM.md.
Highlights
- Integer-built output. Each output
float32is constructed from its exact integer value in integer code,
so the bytes do not depend on the compiler, the optimisation level, the CPU tier or the thread's
floating-point settings. - A library and a tool. A C interface with no escaping exceptions, and
semb bake,query,verifyand
inspect. - Encode from memory.
semb_encoder_open_memoryopens from bytes you already hold;semb_encode_query
encodes a query without a bank. - Memory sized to the work. Opening reserves no per-slot workspace; each encode's workspace is sized to its
document. - GPU on Windows. Direct3D 12, same bytes as the CPU, re-proved at every open. Verified on one GPU: an
AMD Radeon RX 7900 XT, on the engine's multi-row path at SuperSLM 1.18.1. - Checked linking. Every open refuses an engine other than the one the build was configured against.
Measured
Readiness, from process start through the first query result (process startup, encoder open, bank open and
the first encoded query), on the msvc19.33-auto@avx2 build on an AMD Ryzen 9 5950X, CPU device: 9,204 ms
with a warm file cache (median of 3 runs); 635.6 MiB of host memory with the encoder open, 1,233.7 MiB at peak
during the open; a median query of 376.5 ms in one --stdin session of 239 queries. Encoding one document
on the same build takes 2,681 ms at 256 tokens (95.48 tokens/s), 12,925 ms at 1,024 tokens (79.23 tokens/s)
and 95,688 ms at 4,096 tokens (42.81 tokens/s).
Retrieval quality, as the fraction of a floating-point run's nDCG@10 kept, on a fixed pool per set, with paired
95% intervals: SciFact 0.9798 [0.9630, 0.9965], FiQA-2018 0.9654 [0.9405, 0.9882], SCIDOCS 0.9383
[0.9110, 0.9646]. Measured on an MSVC 19.33.31631 Release build (commit 2001662) on an AMD Ryzen 9 3950X. The 1.0.0 release produces the same document and query vectors as the build measured, so these figures apply to it unchanged.
GPU, at 1.18.1 on a GPU-enabled MSVC 19.33 build, on an AMD Ryzen 9 5950X with an AMD Radeon RX 7900 XT
(driver 32.0.21030.2001): a query on an open handle takes 0.2149 s against 0.4096 s on the CPU. A single
semb query process is faster on the CPU (10.00 s against 14.79 s), because the GPU open (15.41 s against
9.43 s) costs more than one query saves. For bakes the CPU is faster: at the default 8 workers it bakes about
three times the GPU's rate (for example 329.87 against 110.11 positions/s on long FiQA-2018 documents); the GPU
is 7% to 53% faster than one CPU worker. One 4,096-token document bakes at 62.68 positions/s on the GPU against
35.54 on one CPU worker. An open GPU encoder holds 472.06 MiB of GPU memory, and a document at the
32,768-position cap peaks at 2,300.31 MiB.
Verification at 1.18.1: the certified CPU builds (MSVC 19.33 and clang-cl 18.1.8, AVX2) were certified on an
AMD Ryzen 9 5950X, with the same row digest as the earlier certification on an Intel i5-14400F at engine
1.15.0. On an AMD Radeon RX 7900 XT, a full GPU bake of the 239-document certification corpus matched the CPU on
every row. On the RX 7900 XT, the GPU test suite passed except for a known test issue that was fixed in the tests
(C64 a/a2 timing-record drift), and a 32,767-token long-context correctness capture passed 2 of 2.
Know before you upgrade
- Banks from any pre-release build are refused (
SEMB_FINGERPRINT_MISMATCH). Re-bake them. - A document longer than 32,768 tokens is refused, never truncated. A query is truncated to fit, by design.
- The engine writes one
preflight:line to stderr on every encoder open. It is expected. - Only English retrieval has been measured.
- The GPU path is not yet verified on NVIDIA at 1.18.1. At this pin the GPU encode runs the engine's
multi-row prompt kernels, which have been run on AMD only. Earlier NVIDIA checks (RTX 2080 SUPER at engine 1.7.1 or
earlier, and RTX 5060 at engine 1.17) were on the older one-row path. Every GPU open still checks itself against the CPU and
refuses on any difference.
Full list of changes: CHANGELOG.md.