A living, self-improving architecture document for the Hermes AI agent. Updated daily through autonomous research and iteration.
Integrated with Hermes Second Brain — an AI-native knowledge management system that compiles raw research into a structured, queryable wiki with GraphRAG. Device monitoring via pi-telemetry-bot — a Telegram bot for Raspberry Pi system telemetry and alerts.
Hermes is a multi-modal AI agent that operates across CLI, Telegram, Discord, Slack, and other platforms. This repository tracks its architecture evolution, design decisions, and improvement roadmap.
| Diagram | Description |
|---|---|
| System Overview | High-level agent architecture with all major components |
| Orchestrator Pattern | v2: subagent roles, interrupt flow, escalation |
| Agent Loop | Core conversation and tool execution loop |
| Memory System | Persistent memory, context management, and session recall |
| Travelling Wave Memory | Biological memory model: fusion over deletion |
| Tool Pipeline | Tool discovery, dispatch, and execution |
| Multi-Platform Gateway | Telegram, Discord, Slack, WhatsApp adapters |
| Nightly Pipeline | Automated research and self-improvement cron jobs |
| Security + Sandbox + Disk Hygiene | Malware scanning, lightweight sandboxing, and low-disk cleanup flow |
| Skill System | Skill loading, discovery, and execution |
| IMAP Trust Protocol | Immune Memory Attestation Protocol for agent trust |
| Session Compaction | Context eviction and anchored summarization when sessions hit the wall |
| Second Brain Integration | |
| Obsidian Sync Flow | |
| Telegram Telemetry Architecture | |
| Integrated Nightly Loop |
Hermes integrates with Hermes Second Brain, an AI-native knowledge management system that implements Andrej Karpathy's LLM Wiki pattern. Second Brain acts as Hermes's externalized, compounding knowledge cortex.
Daily Learnings → Skills → Graph → Wiki
↓ ↓ ↓ ↓
New research Procedural Entity Queryable
arrives knowledge graph documentation
- Daily Learnings: New research (papers, articles, repos) is read and ingested
- Skill Extraction: Patterns identified → SKILL.md files auto-generated → symlinked into Hermes
- Knowledge Graph: Entities and concepts extracted; relationships (
uses,implements,inspired_by) built - Wiki Compilation: LLM creates/updates Markdown pages with citations and wikilinks
- Query & Synthesis: Agent/human queries the knowledge base; answers can be saved back as synthesis pages (feedback loop)
- Zero maintenance: LLM compiles and maintains wiki; humans curate sources only
- Provenance: Every claim traces to a raw source file
- Compounding: Answers saved from queries enrich the knowledge base
- Obsidian-native: Files are Markdown with wikilinks; open in Obsidian for graph view
- GraphRAG-powered: Hybrid TF-IDF vector + graph retrieval for accurate, contextual answers
- Real-time sync: Obsidian edits automatically pushed to GitHub via inotify watcher (see Obsidian Sync Flow)
Second Brain jobs are fully integrated into Hermes's pipeline (see Integrated Nightly Loop):
- 06:30 daily — Compile:
hermes-brain-compile --incrementalprocesses new research → updates wiki - 06:45 daily — Graph: Wikilink extraction auto-updates
memory/graph.edges.json - 07:30 Sun — Digest: Full lint + weekly insights → Telegram summary
Additional background tasks:
- Watcher daemon (
watch_and_push.sh) watches Obsidian vault → auto-commits & pushes changes - Post-push hooks rebuild TF-IDF index and update graph from wikilinks
- Telemetry Bot monitors Raspberry Pi health; alerts pushed to Telegram (Telemetry Bot architecture)
From research to usable skill in one pipeline:
Research article → Pattern detection → SKILL.md auto-gen → Symlink → Hermes skill
All skills live in synthesis/skills/ and are symlinked into ~/.hermes/skills/ instantly. Schema validated before deployment; usage feedback refines future generations.
See full architecture for complete details.
| Area | Status | Notes |
|---|---|---|
| Agent Orchestration | Active | Multi-agent delegation, parallel task execution |
| Memory Management | Active | Cross-session persistence, context compression, RAG |
| Token Efficiency | Active | Prompt caching, compression, model routing |
| Sandboxing | Active | Lightweight command isolation for low-disk ARM runtime |
| Daydreaming | Exploring | Autonomous exploration and creative reasoning |
| Agentic Payments | Planned | Self-managed budgets, payment rails |
| Skill Evolution | Active | Auto-creation, maintenance, and optimization of skills |
Every day the agent researches improvements and updates this repository. See the iterations log for daily changes.
MIT