Skip to content

v1.4.0 — retrieval improvements + temporal knowledge lifecycle

Choose a tag to compare

@PeterPirog PeterPirog released this 24 Aug 20:46
· 74 commits to main since this release

1.4.0

Section-aware chunking, read-only search, SimHash dedup, multilingual profiles, temporal knowledge lifecycle.

Task 5 — Section-aware chunking

  • chunk_memory(): splits by Markdown headings, prefix with title/type/tags/scope
  • Short memories get exactly 1 chunk; deterministic chunk IDs <memory_id>:<section-slug>:<ordinal>
  • Config: retrieval.chunking.*; SQLite schema v3

Task 6 — Read-only search / usage tracking

  • Search/context never mutate Markdown; usage lives in the SQLite usage store
  • migrate-usage --dry-run/--apply [--strip] [--json] with timestamped backups
  • index --rebuild preserves usage; content_hash excludes usage fields

Task 7 — SimHash deduplication

  • Exact: SHA-256; Near: 64-bit SimHash (pure stdlib), Hamming <= 3
  • remember/import: exact block, near warn, --force bypass; archived informational
  • remember --json returns status + duplicate + separate conflict list

Task 8 — Multilingual PL/EN profile

  • retrieval.profile: english-fast (default) | multilingual (intfloat/multilingual-e5-small, query:/passage: prefixes)
  • Cache keys include model identity — profiles never share vectors
  • Benchmark: multilingual > english-fast on PL group (R@1 12.5%→18.75%), no EN regression
  • pack.py --profile english-fast|multilingual

Task 9 — Temporal, safe knowledge lifecycle

  • Optional schema-2 frontmatter: confidence, valid_from, valid_to, supersedes, derived_from (schema-1 compatible)
  • supersede <ref> --by <new>: closes validity window, links both directions; history never deleted
  • search --at YYYY-MM-DD: validity-window filter; history stays retrievable; bad dates ignored
  • timeline: sorts by effective validity
  • context: read-only HISTORY & CONFLICTS section
  • consolidate --dry-run [--json]: deterministic read-only report + plan for the agent; no deletion, no LLM summarization

Tests

  • 115 unit tests OK (incl. tests/test_lifecycle.py)
  • self_test.py: PASS