Skip to content

Architecture

Jump to bottom
Siddh Mistry edited this page Sep 7, 2026 · 1 revision

Architecture

The memory pipeline

Daily/YYYY/MM/DD/*.md   →   Memory-Review/*.md   →   Hermes native MEMORY.md / USER.md
 (raw session archive)      (staged candidate         (durable, injected every
                              facts, human-reviewed)    turn — promoted by hand)
  1. Daily/ — a Hermes cron job (hourly by default) exports every chat session as redacted markdown, one file per session, organized YYYY/MM/DD/. This is raw material, not curated — expect noise.
  2. Memory-Review/ — Scripts/consolidate_memory.py (a "dream cycle," mechanical only, no LLM) parses raw fact sources, strips anything matching secret/credential patterns into Excluded-Sensitive.md, dedupes exact and near-duplicate facts, flags 60+ day undecided candidates as stale, and rewrites Promotion-Candidates.md as a checkbox list.
  3. Native memory — checking a box in Promotion-Candidates.md and re-running the script archives that decision to Consolidation-Log.md. The script never writes to Hermes's real MEMORY.md/USER.md — copying an approved fact there stays a manual human step.

Folder skeleton

Folder Purpose
Daily/ Raw session archive (see above)
Memory-Review/ Staged fact candidates + review criteria (TEMPLATE.md)
Research/ Working space for in-progress investigation; optional trend-digest output lands in Research/Trend-Digest/
Skills-Notes/ Index of installed Hermes skills, and (if multi-agent) teammate profiles
Projects/ One note per active project
Scripts/ Cross-platform automation (PowerShell + bash pairs)
.obsidian/ Plugin manifests: Dataview, Smart Connections, Local REST API, brain-atlas, Tokyo Night theme

Cross-platform scripts

Purpose Windows Linux/macOS
Memory consolidation python Scripts\consolidate_memory.py python3 Scripts/consolidate_memory.py
Trend digest — Ollama backend Scripts\pipeline.ps1 Scripts/pipeline.sh
Trend digest — sentence-transformers backend Scripts\New-TrendDigest.ps1 Scripts/trend_digest.sh

All scripts resolve the vault path relative to their own location ($PSScriptRoot/$(dirname "${BASH_SOURCE[0]}")), so there are no hardcoded personal paths anywhere in the template.

Design decisions

  • Vault scope is deliberately narrow. It should only ever contain this memory-pipeline content, never a whole personal OneDrive/Documents folder.
  • Only one archiving cron job per vault. Concurrent writers to Daily/manifest.jsonl will race and corrupt the index.
  • Secrets never live in vault notes. The Local REST API key, or any other credential, goes through Hermes's protected env (hermes config set env.KEY_NAME "..."), not a markdown file — vaults commonly sync via cloud storage or git, and plaintext secrets there are a real leak vector (this happened once during this template's own development; see the repo's Security notes in the README).
  • Consolidation is mechanical, not LLM-based, specifically so its output is deterministic and auditable — a secret-pattern regex, exact + fuzzy dedup via difflib, and a 60-day staleness flag, nothing more.

Clone this wiki locally