Skip to content

ferrovec 0.6.0: offline model loading, compact(), and Retrace

Latest
Latest

Choose a tag to compare

View all tags
@singhpratech singhpratech released this 10 Nov 06:02
· 3 commits to main since this release
v0.6.0
143c372

Released together: crates.io https://crates.io/crates/ferrovec/0.6.0 and npm https://www.npmjs.com/package/ferrovec/v/0.6.0. The Rust core's code is unchanged from 0.5.0; its version moves to 0.6.0 so crate and npm share one release number.

Offline model loading — the transformers option (npm)

  • Ferrovec.open(name, { transformers: { env, dtype } }), and the same option on Engine.create and createTransformersEmbedder, configures transformers.js inside the store's worker before the model loads.
  • env accepts allowRemoteModels, allowLocalModels, localModelPath, remoteHost, useBrowserCache, useWasmCache, wasmPaths and numThreads; dtype sets the pipeline dtype.
  • With local model files, remote models off and a local onnxruntime wasm, ferrovec runs fully offline and inside a Manifest V3 browser extension with a strict CSP. See "Offline: local model and wasm files" in the package README.

db.compact() (npm)

  • Removed and replaced documents leave the store's files, not only its results. The index is rebuilt from the live vectors, and both snapshot slots and the write-ahead log are rewritten.
  • It re-inserts every live vector and writes two full snapshots, so call it after a deletion that must really be gone, or now and then in a store with many removes and upserts.

Upgrade note

  • No format change, no action needed. The on-disk format is unchanged from 0.5.0; 0.6.0 reads and writes the same stores.

Retrace — a Chromium extension built on ferrovec (extensions/retrace/, v0.1.0)

Semantic search over every page you have read. Type rt, a space, and what you remember in the address bar. Text, vectors and the embedding model stay in your browser; no account, server or API key. It is what needed the two library changes above.

  • Install from the zip: download retrace-0.1.0-chromium.zip from this release, unzip it, open chrome://extensions, turn on Developer mode, click Load unpacked, and choose the unzipped folder. Needs Chrome 116 or newer. Or build it from source (steps in the extension README).
  • Limits: version 0.1.0; not on the Chrome Web Store, load unpacked only. Chrome is tested; Edge and Brave should work but are untested; Firefox port planned, not built. It only knows pages you open after installing it. Indexing costs about 1 s of CPU per long page in the background, and the queue holds 200 pages. Tested up to about 2.9k passages; memory use not measured. Single-page-app navigations are not re-captured. Omnibox suggestions are not covered by the automated test. The per-URL bookkeeping record can linger in Chromium's IndexedDB log until Chromium compacts it.
  • Verified: 36 of 36 end-to-end checks and 50 of 50 unit tests pass; in an automated run, zero requests were initiated by the extension. Measured on one desktop: queries at 2.8k passages p50 19 ms, p95 26 ms.
Assets 3
Loading