Add SQLite memory system with FTS5 full-text search#9
Merged
Conversation
Implements a persistent memory system backed by SQLite with FTS5 for full-text search. Memories can be saved/searched via agent tools and auto-extracted from conversations via configurable middleware. - MemoryStore: SQLite + FTS5 with configurable path, max size, and TTL - DefaultMemoryExtractor: extracts [REMEMBER:] markers and user preferences - memory_search / memory_save tools exposed to the agent - Auto-extraction middleware (beforeLoopBegin prunes, afterLoopIteration extracts) - Config: memory.enabled, path, maxSizeMB, ttlDays, extractor, autoExtract - Env vars: RA_MEMORY_ENABLED, RA_MEMORY_PATH, RA_MEMORY_MAX_SIZE_MB, etc. - Custom extractors via memory.extractor config path https://claude.ai/code/session_01UES55H4Pujmi9K3ggjuosx
…LLM reflection Layered memory: - Two memory layers: session (ephemeral, per-conversation) and long-term (durable) - Session memories auto-expire via sessionTTLHours (default: 24h) - Long-term memories persist with ttlDays (default: 90 days) - promote() to upgrade session memories to long-term - Size enforcement prioritizes removing session memories first Configurable extraction patterns: - PatternExtractor with configurable regex patterns, role filters, max length, capture modes - DEFAULT_PATTERNS provides baseline (REMEMBER markers, user preferences) - Custom patterns via config (merged with defaults) - Deduplication within extraction passes LLM-driven reflective extraction: - ReflectiveExtractor sends conversation to an LLM for structured learning extraction - Runs on afterLoopComplete (end of conversation, not per-iteration) - Best-effort: never fails the main flow - Configurable model (use a cheaper model for reflection) New config options: sessionTTLHours, patterns[], reflect, reflectionModel New tool: memory_delete New env vars: RA_MEMORY_SESSION_TTL_HOURS, RA_MEMORY_REFLECT, RA_MEMORY_REFLECTION_MODEL https://claude.ai/code/session_01UES55H4Pujmi9K3ggjuosx
…pt injection The LLM discovers when to save/search memories purely from the tool descriptions (which include PROACTIVELY directives). The middleware stays focused on pruning and pattern extraction — no message injection. https://claude.ai/code/session_01UES55H4Pujmi9K3ggjuosx
- ReflectiveExtractor accepts optional custom prompt (must contain {CONVERSATION})
- Export DEFAULT_REFLECTION_PROMPT for reference when writing custom prompts
- Config: memory.reflectionPrompt, env: RA_MEMORY_REFLECTION_PROMPT
- Thread through middleware options and src/index.ts wiring
https://claude.ai/code/session_01UES55H4Pujmi9K3ggjuosx
Instead of relying on the LLM to call memory_search, inject the top N most recent long-term and session memories as a <recalled-memories> user message at the start of each loop. This gives the LLM context without extra tool calls. - Configurable via memory.injectLimit (default: 20, set 0 to disable) - Env var: RA_MEMORY_INJECT_LIMIT - Long-term memories shown with tags, session memories tagged with session ID https://claude.ai/code/session_01UES55H4Pujmi9K3ggjuosx
…flection Session is now the default layer for all memory operations: - store.save() defaults to session - memory_save tool defaults to session - Pattern extractor saves to session - Only the reflective extractor (afterLoopComplete) promotes to long-term The reflection prompt is updated to set a high bar: only extract facts with proven, lasting importance — reinforced preferences, architectural decisions, recurring patterns. https://claude.ai/code/session_01UES55H4Pujmi9K3ggjuosx
Delete remains available on MemoryStore for programmatic use but is not exposed as an agent tool. https://claude.ai/code/session_01UES55H4Pujmi9K3ggjuosx
Drop layers, extractors, and reflection in favor of a single flat store with three tools (memory_save, memory_search, memory_forget). Fix forget() returning inflated count from FTS triggers and list() ordering by second-precision timestamp instead of ID. https://claude.ai/code/session_01UES55H4Pujmi9K3ggjuosx
- Remove unused UPDATE trigger and buggy size-based enforceMaxSize loop - Replace maxSizeMB with count-based maxMemories (reliable, no WAL issues) - Combine schema init into single exec call - Add memory section to README, configuration, tools, and concepts docs https://claude.ai/code/session_01UES55H4Pujmi9K3ggjuosx
- Guard search/forget against empty queries and FTS5 syntax errors (unmatched quotes, parens, bare operators) that would throw at runtime - Add PRAGMA busy_timeout for safe multi-process access - Add 12 new edge case tests: FTS special chars, empty queries, FTS sync after forget, persistence across reopen, trim/prune no-ops https://claude.ai/code/session_01UES55H4Pujmi9K3ggjuosx
- memory_save: directive language, concrete examples, dedup guidance (forget old version before saving updated one) - memory_search: explains that recent memories are already injected, use this for targeted lookups beyond the recalled set - memory_forget: broadened triggers to include contradictions and pre-update cleanup, not just explicit user requests - Tags: suggest concrete categories (preference, project, convention, team, tooling) instead of vague "comma-separated tags" - Query hints: "single keywords work best" to avoid FTS syntax issues - Injection message: explains what recalled memories are and tells the LLM to forget outdated ones https://claude.ai/code/session_01UES55H4Pujmi9K3ggjuosx
- --memory CLI flag enables memory without editing config - /memories [n] lists stored memories in the REPL - /forget <query> lets users manually delete memories by search - Update help text, README, and REPL docs https://claude.ai/code/session_01UES55H4Pujmi9K3ggjuosx
The REPL /forget command called store.forget(query) without a limit, defaulting to 10. Users expect /forget to delete all matching memories, not just the first 10. https://claude.ai/code/session_01UES55H4Pujmi9K3ggjuosx
ra --memory --memories prints all stored memories and exits, providing a quick way to inspect memory contents without starting a REPL session. https://claude.ai/code/session_01UES55H4Pujmi9K3ggjuosx
- `ra --memories` now auto-enables memory (no need for `--memory --memories`) - `ra --forget "dark mode"` deletes matching memories and exits - `ra --forget "dark mode" --dry-run` previews matches without deleting - --forget and --memories both imply --memory automatically https://claude.ai/code/session_01UES55H4Pujmi9K3ggjuosx
…-forget Replace --dry-run workflow with a simpler pattern: ra --memories # list all memories ra --memories dark mode # search memories matching "dark mode" ra --forget dark mode # delete memories matching "dark mode" Users search first with --memories, then use the same query with --forget. Both flags use positionals as the query and auto-enable memory. https://claude.ai/code/session_01UES55H4Pujmi9K3ggjuosx
ra --memories "dark mode" # search ra --forget "dark mode" # delete matches ra --memories # list all (no query) Pre-extracts --memories/--forget from argv before parseArgs since util.parseArgs doesn't support optional string values. https://claude.ai/code/session_01UES55H4Pujmi9K3ggjuosx
- Merge conflict in shutdown(): keep try/catch from main + memoryStore.close() - Consistent error messages across CLI and REPL - README: document --memories and --forget CLI flags - Config docs: add --memory CLI flag to table https://claude.ai/code/session_01UES55H4Pujmi9K3ggjuosx
Keeps context lean by default — inject only the 5 most recent memories per loop. Users can increase via config if needed. https://claude.ai/code/session_01UES55H4Pujmi9K3ggjuosx
…ions FTS5 handles multiple keywords well (implicit AND). Updated search and forget tool descriptions to show multi-keyword examples. https://claude.ai/code/session_01UES55H4Pujmi9K3ggjuosx
Replace hacky pre-extraction loop with standard util.parseArgs options. --memories is now a regular string arg, --list-memories is a boolean for listing all, and --forget is a regular string arg. https://claude.ai/code/session_01UES55H4Pujmi9K3ggjuosx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements a persistent memory system backed by SQLite with FTS5 for
full-text search. Memories can be saved/searched via agent tools and
auto-extracted from conversations via configurable middleware.
https://claude.ai/code/session_01UES55H4Pujmi9K3ggjuosx