ferrovec 0.6.0: offline model loading, compact(), and Retrace

Released together: crates.io https://crates.io/crates/ferrovec/0.6.0 and npm https://www.npmjs.com/package/ferrovec/v/0.6.0. The Rust core's code is unchanged from 0.5.0; its version moves to 0.6.0 so crate and npm share one release number.

Offline model loading — the transformers option (npm)

• Ferrovec.open(name, { transformers: { env, dtype } }), and the same option on Engine.create and createTransformersEmbedder, configures transformers.js inside the store's worker before the model loads.
• env accepts allowRemoteModels, allowLocalModels, localModelPath, remoteHost, useBrowserCache, useWasmCache, wasmPaths and numThreads; dtype sets the pipeline dtype.
• With local model files, remote models off and a local onnxruntime wasm, ferrovec runs fully offline and inside a Manifest V3 browser extension with a strict CSP. See "Offline: local model and wasm files" in the package README.

db.compact() (npm)

• Removed and replaced documents leave the store's files, not only its results. The index is rebuilt from the live vectors, and both snapshot slots and the write-ahead log are rewritten.
• It re-inserts every live vector and writes two full snapshots, so call it after a deletion that must really be gone, or now and then in a store with many removes and upserts.

Upgrade note

• No format change, no action needed. The on-disk format is unchanged from 0.5.0; 0.6.0 reads and writes the same stores.

Retrace — a Chromium extension built on ferrovec (extensions/retrace/, v0.1.0)

Semantic search over every page you have read. Type rt, a space, and what you remember in the address bar. Text, vectors and the embedding model stay in your browser; no account, server or API key. It is what needed the two library changes above.

• Install from the zip: download retrace-0.1.0-chromium.zip from this release, unzip it, open chrome://extensions, turn on Developer mode, click Load unpacked, and choose the unzipped folder. Needs Chrome 116 or newer. Or build it from source (steps in the extension README).
• Limits: version 0.1.0; not on the Chrome Web Store, load unpacked only. Chrome is tested; Edge and Brave should work but are untested; Firefox port planned, not built. It only knows pages you open after installing it. Indexing costs about 1 s of CPU per long page in the background, and the queue holds 200 pages. Tested up to about 2.9k passages; memory use not measured. Single-page-app navigations are not re-captured. Omnibox suggestions are not covered by the automated test. The per-URL bookkeeping record can linger in Chromium's IndexedDB log until Chromium compacts it.
• Verified: 36 of 36 end-to-end checks and 50 of 50 unit tests pass; in an automated run, zero requests were initiated by the extension. Measured on one desktop: queries at 2.8k passages p50 19 ms, p95 26 ms.

ferrovec 0.5.0 — per-document metadata, LangChain.js adapter, local RAG example

Released together: crates.io https://crates.io/crates/ferrovec/0.5.0 and npm https://www.npmjs.com/package/ferrovec/v/0.5.0. The Rust core's code is unchanged from 0.4.0; its version moves to 0.5.0 so crate and npm share one release number.

Per-document metadata (npm)

• insert(text, { id, metadata }) stores a JSON object beside the text; it comes back on every query() hit and list() row.
• Covered by the same write-ahead log and checksummed snapshots as the text: a new log record kind (3) and an optional third element in the snapshot's text table. 0.5 reads every 0.4 store unchanged.
• Capped at 64 KiB per document; TypeError for anything that is not plain JSON, RangeError('metadata too large') over the cap. An upsert without metadata clears the previous metadata.
• Upgrade note: 0.4.x cannot open a store that contains metadata (it truncates the log at the first metadata record and rejects a snapshot slot that holds any). Close every tab of your app before upgrading.

LangChain.js adapter — ferrovec/langchain

• FerrovecVectorStore, a LangChain VectorStore: addDocuments, similaritySearch / similaritySearchWithScore with an exact-match metadata filter, delete, fromDocuments, fromTexts, and the inherited asRetriever().
• @langchain/core ^1 is an optional peer dependency, only loaded when you import the subpath.

Example — examples/ai-sdk-local-rag

• Vite page: notes go into ferrovec in the browser, a Vercel AI SDK tool call retrieves them, the model answers through any OpenAI-compatible endpoint (Ollama on localhost by default). Nothing leaves the machine.

Tests

• Node 90 pass / 1 skipped, browser durability 5/5 including a new kill-and-reopen metadata case, chaos harness 100 cycles / 467 acknowledged writes / 0 lost.

ferrovec 0.4.0 — crash-safe storage: checksummed snapshots, write-ahead log, published kill-the-tab test

Crash-safe storage (M7). The index on disk is now two alternating, CRC-32-checksummed snapshot slots plus a write-ahead log. Every insert/remove is a checksummed log record flushed to OPFS before its promise resolves; checkpoints are written before the log is reset, so open always takes the newest valid snapshot and replays the log on top. A torn or bit-flipped file is detected and skipped, never decoded into a wrong index. If nothing trustworthy is left, open refuses with a typed error instead of starting empty over your data.

Tested like a database. npm run test:chaos kills the tab at a random moment (including mid-insert), reopens, and checks every acknowledged id. Published results: 1,000 cycles each in Chromium 149 and Firefox 151, 5,518 + 5,445 acknowledged writes, 0 lost. A negative control (log write deferred by 30 ms) loses everything, so the harness detects loss. https://singhpratech.github.io/ferrovec/chaos.html

Also new

• Ferrovec.list({ offset, limit }) enumerates stored documents.
• durability: 'strict' | 'relaxed' and db.flush().
• Ferrovec.destroy(name), db.storagePersisted, requestPersistentStorage option.
• The log header carries the embedding model id, so a store is refused under a different model before any download, even before its first checkpoint.
• Typed errors cross the worker boundary: StoreCorruptError, EmbeddingSpaceMismatchError, WalCorruptError, SnapshotCorruptError, EmptyOverwriteRefusedError.
• Live demo with two tabs: semantic search, and a notebook you can kill mid-save. https://singhpratech.github.io/ferrovec/demo.html

Upgrading from 0.3.x. Existing single-file stores are migrated to the slot format on the first commit. Close every tab of your app before upgrading.

Not promised. Power-loss durability (OPFS flush() is not fsync) and cross-device sync.

crates.io: https://crates.io/crates/ferrovec/0.4.0 · npm: https://www.npmjs.com/package/ferrovec/v/0.4.0