Skip to content

v0.35.0

Choose a tag to compare

View all tags
@stffns stffns released this 27 Apr 15:33
· 96 commits to develop since this release
v0.35.0
4f77461
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

v0.35.0

Headline change: the v0.33.0-deprecated fts_only=bool parameter is now removed from every public surface (breaking). The migration is a straight rename to retrieval_mode="fts_only".

This release also lands the LongMemEval retrain experiments, the bge-small-rrf-lme-v1 chat-memory specialist model, and the v2 paper draft.

Removed (breaking)

  • fts_only=True bool parameter on VstashStore.search, Memory.search, Memory.ask, and the vstash_search / vstash_ask MCP tools (#281, #298). Migration: pass retrieval_mode="fts_only" instead.

Added

  • vstash retrain --eval-queries flag (#299) -- accepts a held-out labeled JSONL for the eval gate. Drives a refuse-to-save policy: candidates that regress NDCG@10 on the holdout are retained at output.candidate/ for inspection but never replace the active model. This is the deployable surface of the gated domain-adaptation loop documented in the v2 paper draft.

Changed

  • VstashStore._resolve_retrieval_mode is now single-argument (string only); the legacy bool conflict branch was removed.
  • _coerce_bool helper in vstash.mcp removed; no remaining callers.

Research artifacts

  • Paper v2 draft -- paper/v2/vstash-paper-v2.md, "vstash v2: Eval-Gated Domain Adaptation for Local-First LLM Memory". LaTeX conversion to paper/arxiv/ is the next milestone.
  • bge-small-rrf-lme-v1 -- chat-memory specialist embedding (33M params, 384-dim) trained via the new --eval-queries gate on 398 LongMemEval-s questions. Holdout NDCG@10 0.6878 vs vanilla BGE-small 0.6143 (+11.85% relative). R@5 +3.79pp on the 102-query holdout, 95% CI [+1.72, +6.19] (paired bootstrap, B=1000, n=102). Published at https://huggingface.co/Stffens/bge-small-rrf-lme-v1.
  • LongMemEval head-to-head harness -- experiments/longmemeval_retrieval.py, experiments/longmemeval_colbert.py, experiments/colbert_minimal.py (faithful HF ColBERTv2 re-implementation), experiments/h2h_combine.py (5-arm aggregation).
  • Eval-gated retrain reproducibility -- experiments/lme_prepare_retrain.py (deterministic 80/20 stratified split, seed 42), experiments/lme_holdout_bootstrap.py (paired bootstrap CIs), and the resulting lme_eval.jsonl / lme_train.jsonl / lme_retrain_meta.json / lme_full_500_*.json artifacts under experiments/results/.
  • Cross-domain transfer failure evidence -- Stffens/bge-small-rrf-v3 (BEIR-tuned, v0.34.0 release) regresses -2.45pp NDCG@10 on the LongMemEval holdout vs vanilla BGE-small. This is the v2 paper's core empirical claim and the motivation for per-domain retrain.

Install

pip install --upgrade vstash

PyPI: https://pypi.org/project/vstash/0.35.0/

Assets 2
Loading