[RFC / Proposal] Triple-Layer Persistent Memory (Markdown Rules + Vector Embeddings + Hybrid Vitality Ranking) for DeepSeek Harness

[avaritiachaos]

RFC Proposal: 4-Tier Cognitive Long-Term Memory & Self-Correction Engine for DeepSeek Harness

Status: Production-Ready & Open-Sourced (v0.3.1)
Reference Implementation: @shion-lab/dsh-plugin-memory (MIT License, 100% Passing Live TypeScript Test Suite)
Origin: Distilled from the companion AI cognitive architecture of Shion (紫苑)

---

1. Motivation & Background / 背景与痛点

Currently, DeepSeek Harness (dsh) operates in a stateless manner:

1. The "Cold Start" Amnesia: Every time a session closes, project conventions, user architectural preferences, and hard-earned debugging lessons are lost.
2. Context Window Waste: Re-instructing the model consumes precious context budget and developer patience.
3. The "Pollution & Hallucination" Risk: Naive memory stores frequently get polluted by temporary stack traces, diff snippets, and contradictory directives (e.g. "disallow tailwind" vs "use tailwind").

To solve this systematically, we propose a 4-Tier Cognitive Long-Term Memory & Self-Correction Architecture.

---

2. 4-Tier Architecture Overview / 四层立体认知防护体系

                      ┌──────────────────────────────────────────────────────────┐
                      │    🧠 @shion-lab/dsh-plugin-memory (v0.3.1)              │
                      └────────────────────────────┬─────────────────────────────┘
                                                   │
      ┌───────────────────────────┬────────────────┴──────────────┬───────────────────────────┐
      ▼                           ▼                               ▼                           ▼
【Tier 1: MemoryGuard】     【Tier 2: ConflictDetector】    【Tier 3: Git-Tracked MD】  【Tier 4: Vector Vitality】
  - Blocks stack traces       - Resolves polarity flips       - .dsh/MEMORY.md            - Dense Cosine Embeddings
  - Blocks test outputs       - Chinese grammar negations     - [✔ Verified] protection   - 30-day true half-life decay
  - Blocks API secrets        - Verified immunity gate        - Async atomic writes       - Background dream() cycle

🛡️ Tier 1: MemoryGuard (Red-light Anti-Pollution Gate)

• Strictly blocks noisy stack traces (Traceback...), test runner summaries (pytest PASSED/FAILED), diff blocks, and API credentials (sk-...) on both topics and contents before entering memory.

⚖️ Tier 2: MemoryConflictDetector (Subject & Polarity Resolution)

• Automatically detects contradictory rules targeting the same subject across natural language expressions (e.g. 禁止使用 Tailwind vs 统一使用 Tailwind).
• Verified Rule Immunity Gate: Unverified spontaneous instructions are prevented from deleting/overriding certified [✔ Verified] rules without explicit correct() or forceOverride: true.

📝 Tier 3: Human-in-the-Loop Git-Versioned Markdown (.dsh/MEMORY.md)

• Plain text, human-auditable knowledge base with [✔ Verified] test certification badges.
• Decoupled Disk Persistence: Complete, unbudgeted disk serialization (serializeForDisk()) guarantees zero data loss, while active prompt injection (renderForPrompt()) adheres strictly to token budgets.
• Transactional Atomic Writes: Managed with async mutex and temporary file rename (fs.rename) to guarantee zero file corruption under concurrency.

🧠 Tier 4: Dense Vector Semantic Engine & True Half-Life Dynamics

• Dense Vector Search: Compatible with OpenAI embeddings and local Ollama (nomic-embed-text, bge-m3).
• True 30-day Exponential Half-life:
  $$\text{Score}_{\text{recency}} = 0.5^{\Delta t / 30}$$
• Memory Vitality (Reciprocal Rank Fusion):
  $$\text{Score} = 0.45 \cdot S_{\text{sem}} + 0.25 \cdot S_{\text{lex}} + 0.10 \cdot 0.5^{\Delta t / 30} + 0.05 \cdot \log_{10}(N_{\text{access}} + 1) + 0.05 \cdot S_{\text{importance}} + S_{\text{verified}}$$
• Dream Consolidation (dream()): Background clustering cycle that consolidates near-identical memories and merges access counts.

---

3. Reference Implementation & Try It Now

The complete plugin suite is available on GitHub and ready for DSH:

• Repository: https://github.com/avaritiachaos/dsh-plugin-memory
• Install: npm install -g @shion-lab/dsh-plugin-memory

plugins:
  "@deepseek-ai/dsh": {}
  "@shion-lab/dsh-plugin-memory":
    storagePath: ".dsh/MEMORY.md"
    topK: 6
    autoRecall: true
    embedding:
      enabled: true
      provider: "ollama"
      model: "nomic-embed-text"

We welcome community feedback and look forward to discussing how this can be merged or adopted as an official standard for DeepSeek Harness! 🚀

[zoahdev]

This RFC's Layer 1 (Git-tracked Markdown rules) has a missing half: where rules come from, and how they stay honest. That half now exists as a running, verification-driven loop ? complementary, not competing:

dsh-evolve (https://github.com/zoahdev/dsh-evolve, v0.4.0, MIT):

• extract / reflect ? rules are produced from failure logs and retrospectives automatically
• verify ? every rule is stamped by a real check (dsh-plugin-doctor, official tests, or any exit-0 command)
• evolve_apply ? verified rules install into AGENTS.md / MEMORY.md with backup; unverified rules are refused
• audit ? rule-library health: verified counts, near-duplicate detection, tag/source distribution
• plugin tools evolve_learn / evolve_apply so agents run the loop mid-session

Integration path with this RFC: evolve_apply can write its rendered block into .dsh/MEMORY.md instead of (or alongside) AGENTS.md ? your Layer 1 becomes the storage, ours the production + validation pipeline. The combination gives the ecosystem what memory alone cannot: rules that can be trusted.

@avaritiachaos happy to align storage format and dedup semantics. cc @imccyu @tianyicui ? a memory layer + a verification layer is the full answer to "lost lessons" without the "learned wrong lessons" problem.

[avaritiachaos]

@zoahdev Thanks for checking out our RFC!

That's an interesting complementary perspective. @shion-lab/dsh-plugin-memory is designed as an open, decoupled foundation: Layer 1 (.dsh/MEMORY.md) is deliberately plain Git-versioned Markdown so that any external pipeline (whether manual developer instructions, IDE hooks, or rule-verification loops like yours) can easily write standard structured blocks into it.

Once written, our Layer 2 & 3 (dense vector cosine index + 30-day half-life decay & frequency ranking) automatically handle the zero-bloat semantic recall and context budget injection for DeepSeek.

Feel free to format your outputs to match our schema! 🚀

[avaritiachaos]

🎉 Status Update: Officially Published to npm!

The complete Shion Lab DSH Essential Plugin Suite (including this memory plugin and our newly released vision-to-text bridge) is now officially published on npm with full test coverage and verified Cordis bindings!

# Install Memory Plugin
npm install -g @shion-lab/dsh-plugin-memory

# Install the full suite (Memory + Multimodal Bridge + Smart Patch + Windows Guard + MCP Bridge)
npm install -g @shion-lab/dsh-plugin-multimodal-bridge @shion-lab/dsh-plugin-memory @shion-lab/dsh-plugin-smart-patch @shion-lab/dsh-plugin-windows @shion-lab/dsh-plugin-mcp

• Show Your Plugins Announcement: See our full suite release discussion here: [Release] Shion Lab DSH Essential Plugin Suite (5 Packages: Multimodal Bridge, Memory, Smart Patch, Windows, MCP) #3026
• Source Repos:
  • Memory: https://github.com/avaritiachaos/dsh-plugin-memory
  • Multimodal Bridge: https://github.com/avaritiachaos/dsh-plugin-multimodal-bridge

Thank you all for the feedback! 🚀