Skip to content

v0.11.1 — Windows wheel could not mount any pack

Choose a tag to compare

@spranab spranab released this 29 Jul 08:01
e0df03a

The v0.11.0 Windows wheel could not mount any pack

If you are on Windows and installed 0.11.0, every pack from
packs.yantrikdb.com failed at mount with:

PackEmbedderMismatch: pack was built with embedder blake3:85766a09…,
this database's vectors were built with blake3:f1524499…
(both 64-dimensional, which is why this cannot be caught later)

Linux and macOS wheels of the same release were unaffected. Upgrade to
0.11.1 and the packs you already downloaded will mount — no rebuild, no
re-download.

What happened

The vectors were never wrong. Embedding the same sentence under the Windows
wheel and a Linux build returns the same 64 floats to every printed digit —
same model, same weights, same output. Only the identity of the embedder
disagreed.

Two of the four bundled embedder assets are .json, and the repository had no
.gitattributes. Git treats .json as text and normalises line endings on
checkout, so the windows-latest build runner compiled CRLF while the Linux and
macOS runners compiled LF. bundled_embedder_fingerprint() hashes the raw
include_bytes! blobs, so the Windows wheel ended up disagreeing with its own
sibling wheels from the same release about which embedder it contained.

Reproduced exactly in both directions:

assets fingerprint which build
LF blake3:85766a09… Linux, macOS, and every published pack
CRLF blake3:f1524499… the 0.11.0 Windows wheel

The mount guard was doing its job. It refused rather than searching a query
vector across two indexes built by different models — which returns
plausible-looking nonsense and cannot be detected downstream. It was pointed at
a false mismatch caused by a build that was not reproducible across platforms.

The fix

.gitattributes marks crates/yantrikdb-core/assets/** as -text, disabling
EOL conversion in both directions on every platform, so the fingerprint is a
property of the repository rather than of the machine that ran the build. The
same treatment covers *.ydbpack and *.safetensors: any file whose bytes are
hashed or signed rather than parsed must never be normalised.

A new test, bundled_json_assets_have_no_crlf, asserts the invariant on the
compiled-in bytes rather than trusting the attributes file — a .gitattributes
entry is easy to lose in a merge, and its absence is silent on the platform that
does not convert. It runs on the Windows CI runner, which is where the original
wheel was built.

How it was found

By installing the published wheel into a clean virtualenv with no PYTHONPATH
and mounting a pack downloaded from the live catalog. Nothing in the build or
test loop had ever done that: every script ran with PYTHONPATH=src, which
shadows site-packages with the local development build.

Full changelog: v0.11.0...v0.11.1