taOSmd optimisations: semantic compression, memify, recall scoring, intent-aware retrieval

[jaylfc]

Summary

Implement advanced memory system optimisations inspired by SimpleMem, Cognee, LangMem, and on-device LLM research.

Optimisations (priority order)

1. Semantic Compression (SimpleMem)

Compress raw conversation into self-contained memory units with resolved coreferences and absolute timestamps. SimpleMem achieves 26.4% F1 improvement + 30x token reduction.

• Distill dialogue windows into compact facts
• Resolve pronouns ("he" → "Jay", "it" → "taOS")
• Add absolute timestamps instead of relative ones
• Store compressed units alongside raw archive (zero-loss preserved)

2. Memory Memification (Cognee)

Self-improving graph that evolves over time:

• Prune stale/unused nodes periodically
• Strengthen frequently-accessed connections (increase confidence)
• Reweight edges based on usage signals
• Derive new facts from existing patterns (transitive relationships)
• Runs as a background dreaming task

3. Recall Scoring (LangMem)

Multi-signal retrieval ranking: score = similarity * importance * recency * strength

• Importance: how critical is this fact (preference > trivia)
• Recency: recently accessed/created facts rank higher
• Strength: frequency of access/confirmation
• Currently we only use semantic similarity

4. Intent-Aware Retrieval (SimpleMem)

Classify query intent before searching:

• Factual query → search KG first (structured facts)
• Recent context → search archive (recent events)
• Semantic exploration → search QMD (vector similarity)
• Currently we search all layers regardless of intent

5. KV Cache Compression (HAN Lab)

For on-device inference:

• StreamingLLM attention sinks for infinite-length generation
• DuoAttention head-specific treatment
• Would let Pi handle much longer context windows

Research References

• SimpleMem: https://github.com/aiming-lab/SimpleMem (26.4% F1 gain)
• Cognee: https://github.com/topoteretes/cognee (self-improving memory)
• LangMem: https://langchain-ai.github.io/langmem/ (recall scoring)
• ReMe: https://github.com/agentscope-ai/ReMe (memory management kit)
• MIT HAN Lab: DuoAttention + StreamingLLM

Current Baseline

• KG: 100% precision on structured queries, 13ms
• Regex extraction: 80% on simple text, 15ms
• LLM extraction: 72% recall, 17s/turn (Qwen3-4B on NPU)
• Combined retrieval: 46% precision

[m13v]

the recall scoring approach (similarity * importance * recency * strength) is the right direction. we built something similar for a knowledge extraction system and the biggest win was importance weighting. not all facts are equal, a user preference like 'always use dark mode' is worth 10x more than 'opened Chrome today'. for semantic compression (point 1), one gotcha: aggressive compression loses the original phrasing context which matters when you need to reconstruct why a decision was made. we keep both the compressed version and a pointer to the raw source so you can always trace back.

[m13v]

our knowledge extraction database with structured categories and importance weighting: https://github.com/m13v/ai-browser-profile/blob/main/ai_browser_profile/db.py and the semantic search layer with compressed representations: https://github.com/m13v/ai-browser-profile/blob/main/ai_browser_profile/embeddings.py

[jaylfc]

Thanks @m13v — great insights and really useful code references.

What we're adopting from your approach:

1. Importance weighting via hit rate (accessed_count / appeared_count) — this is exactly right. We'll add access tracking to our KG triples and archive events, then weight retrieval results by importance. Currently we treat all facts equally which hurts precision.

2. Keep compressed + raw with traceability — we already do this (zero-loss archive stores raw, KG stores compressed triples) but we don't have explicit pointers from KG triples back to the archive entry they were extracted from. Adding source_archive_id to triples.

3. Canonical tag categories — our memory type classifier currently has 5 types (fact, preference, decision, event, discovery). Your 8-category normalised tag system with migration mappings is more robust. Will expand.

4. Minimum appearance threshold — great idea for filtering noise in profile generation. We'll add this to the context assembler.

Your ai-browser-profile repo is a solid reference — the superseded_by chain pattern is cleaner than our temporal validity windows for single-value facts. We might use both: supersede chain for explicit updates, validity windows for time-bound facts.

Current taOSmd scores: 97.2% Recall@5 on LongMemEval-S, 100% on our real-world agent benchmarks. These optimisations should push us even higher.

[jaylfc]

Also — taOSmd now has its own dedicated repo if you want to follow the memory system specifically: https://github.com/jaylfc/taosmd

The benchmarks and architecture are documented there. Would love your feedback on the retrieval approach (hybrid semantic + keyword scoring).

[m13v]

the dual approach (supersede chains for explicit updates, validity windows for time-bound facts) makes sense. one thing to watch: you need a clear rule for which mechanism wins when they conflict. if a supersede chain marks fact A as replaced by fact B, but fact A's validity window hasn't expired yet, which one does the retrieval scorer trust? in our experience the explicit supersede should always win, otherwise you get stale facts resurfacing during their validity window even after the user corrected them.

on the minimum appearance threshold, start conservative (3+ appearances before a fact influences profile generation). the failure mode of too-low thresholds is worse than too-high, you get one-off mentions polluting the profile and those are hard to debug because they look like real preferences. you can always lower it later once you have access tracking data to validate against.

for the source_archive_id pointer, consider making it a list rather than a single ID. the same triple often gets confirmed or refined across multiple conversations, and tracking all source entries gives you a natural confidence signal (more sources = higher confidence) without needing a separate counter.

[jaylfc]

Excellent follow-up @m13v — all three points are spot on.

Supersede wins over validity window: Agreed. We'll make explicit supersede always take priority. If superseded_by is set on a triple, it's dead regardless of valid_to. The validity window becomes purely informational ("this was true from X to Y") rather than a resurrection mechanism.

Conservative appearance threshold (3+): Makes sense. We just added appeared_count and accessed_count tracking to our KG triples. Starting at 3 appearances before surfacing in profile/context assembly. Easy to tune later with the access data.

source_archive_id as a list: This is a great insight — multiple confirmations = natural confidence boost without a separate counter. We'll store source_ids: list[str] and use len(source_ids) as a confidence multiplier. Clean and self-validating.

All three going into the next iteration. The dual tracking (appeared_count for frequency, source_ids for provenance) gives us two independent confidence signals that can cross-validate each other.

[jaylfc]

Major update — most of the optimisations in this issue are now implemented:

Implemented in taOSmd v0.2 (dev branch):

✅ Recall scoring — Ebbinghaus decay (retention.py): score = salience * exp(-lambda * days) + sigma * sum(1/daysSinceAccess). Tiers: hot/warm/cold/evictable.

✅ Intent-aware retrieval — Intent classifier routes to optimal memory layer with weights. New timeline intent for temporal queries.

✅ Parallel fan-out retrieval — queries all memory layers simultaneously (vector, KG, archive, catalog, crystals), merges via RRF.

✅ Cross-encoder reranking — ms-marco-MiniLM ONNX second-stage reranker (retrieve 15, rerank to 5).

✅ Query expansion — entity extraction + temporal resolution before search.

✅ Session catalog — LLM-derived timeline directory for "what was I doing Tuesday?" queries.

✅ Secret filtering — 17 regex patterns auto-redact API keys/tokens on all ingest paths.

✅ Graph expansion — BFS from search results through KG to find related facts.

✅ Importance scoring via hit_rate (adopted from m13v's suggestion).

✅ Supersede chains + source_ids provenance tracking.

Benchmark: 97.0% Recall@5 on LongMemEval-S (verified, 500 questions, real distractors).

Still TODO from this issue:

• Semantic compression (deduplicate similar memories)
• Memify-style structured extraction (convert free text to structured KB entries)
• Cross-memory reflection with formal cluster analysis (basic version built in reflect.py)