How high order memories such as insights, semantic facts, procedural skills are consumed?

[veyselberk]

Hello all,
I am a new user for agentmemory and I spent about two days to fully understand how it works including almost all published articles and documentation before I ask here so I would be really very happy if someone can answer these questions. Before I land here, I almost tried all SOTA memory systems such as Honcho, Hindsight, EverOS, Claude-mem, Observatory memory, builtin agent memories and many other memory systems, etc... Most of them are really good and smart but in real world usage, they started to accumulate obsolete memories and noise. I think agentmemory approach is really good (best parts of these SOTA memory systems) and want to make sure that I do utilize its full feature set in my agents.

1. When I check the codebase and documentations, I could not find where these higher order memories such as insights, semantic facts, procedural skills are consumed in Claude Code. I enabled consolidation, reflect and auto compress and graph, so all these high order memories are being created and decayed and maintained but I could not find where they are used (they are not injected or exposed to recall or smart search). These are the most valuable memories so I wonder if I miss something how to use them.
2. When I use smart search, it gives a compact list with a score and LLM should expand it if needed, which is great but LLM cannot expand it using memory ids provided since it still returns compact list instead of full content in the second round even though ids are provided in the second request.
3. What is the reason/main use case to use Claude Code memory_claude_bridge_sync if we are using agentmemory? Should we also enable Claude Code auto-memory and use this sync or just only use agentmemory? If we use them together, would not it create duplicate memory injections since Claude Code also injecting them to context, agentmemory injects some of these memories through memory slots, profile, summaries and observations etc..
4. Agentmemory does not auto-capture agent responses but mainly captures tool responses. Is this a deliberate design choice? If so, why it captures agent responses in json session imports? Is there something out that I am missing?
5. Should we add memory instructions about agentmemory recall or smart search and how it uses progressive disclosure to CLAUDE.md or agentmemory's tool descriptions are enough for LLM to make calls when needed? What is your day to day experience on this?
   Thanks in advance,

[alxjzx100]

Have about the same questions. I'm also new to agentmemory and there is a lack of information across the internet. I ran a couple of sessions but memories/crystals/lessons and so on are 0. Only function calls and token savings are updated.

Should we instruct the model in any way to use agentmemory?

Thank you.

[rohitg00]

@veyselberk @alxjzx100 — thanks for the careful read. Honest answers, one by one.

1. Where higher-order memories are consumed

You're right that there's a gap here. Today, mem::context (the function the SessionStart / PreCompact hooks call to build the injected context block) reads:

• Pinned memory slots
• Project profile (top concepts / files / conventions / common errors)
• Lessons (filtered by project, ranked by 1.5 × confidence if project-scoped else 1.0 × confidence, top 10)
• Recent session summaries
• Recent observations (BM25 + vector + graph reranker)

It does not currently read semantic facts (KV.semantic), procedural skills (KV.procedural), crystals (KV.crystals), insights (KV.insights), or facets (KV.facets). The consolidation + reflect pipelines write them, but they're "dark memory" — accumulated but never injected. memory_smart_search doesn't index them either; it searches compressed observations + lessons.

This is a real bug, not a missing config. Filing it now — should ship in v0.9.25. The right cut is: small ranked block of high-confidence semantic facts + relevant crystals + active procedural triggers at the top of the context, sized so it fits in budget.

2. memory_smart_search expand round-trip

The expand parameter is expandIds (typed as Array<string | { obsId: string; sessionId: string }>), not ids. If you pass ids or memoryIds, the handler ignores it and returns the compact list again. The MCP tool description should be more explicit about the field name — will tighten it. Example second call:

{
  "expandIds": [
    { "obsId": "obs_xxx", "sessionId": "ses_yyy" },
    "obs_zzz"
  ]
}

Bare obsId works only when there's a session-id hint in the same payload; otherwise it falls back to scanning sessions.

3. memory_claude_bridge_sync vs agentmemory's own injection

Different surfaces, partial overlap. agentmemory injects via the SessionStart / PreToolUse / PreCompact hooks — that only works when Claude Code is the active session and the plugin is loaded. The Claude bridge writes a markdown block into ~/.claude/projects/<slug>/MEMORY.md so the content persists into Claude Code's own context window even outside the plugin path (e.g. when you cd into a project that hasn't wired agentmemory yet, or for a recap when the agent decides to read MEMORY.md on its own).

If you have both on: yes there's duplication. The bridge block lives in MEMORY.md; agentmemory's hooks inject a parallel context block. Pick one for primary, the other as backup. Most users want the plugin-driven hooks on (richer) and the bridge off, unless they're explicitly using non-plugin paths.

4. Agent response capture

Deliberate. Claude Code's hook surface fires on tool events (PreToolUse, PostToolUse, Stop, SessionEnd, etc.) — there's no per-assistant-message hook. Live capture sees what tools do, not what the model says between tool calls. JSONL transcripts (the import path) contain the full assistant text because that's what Claude Code persists to disk regardless of hooks, so mem::replay::import-jsonl reads it. If/when Claude Code ships a PostAssistantMessage hook we'd capture live too.

5. CLAUDE.md instructions

Tool descriptions alone are not enough in practice. Adding a short block to CLAUDE.md reliably improves recall use. Pattern that works for us:

## Memory (agentmemory)

- Call `memory_smart_search` early when the user references prior work, asks
  "how did we do X before", or starts a task that touches files you don't
  have context on yet.
- The response is compact (titles + IDs). If a result looks relevant, pass
  its `obsId` + `sessionId` back as `expandIds` in a second
  `memory_smart_search` call to get full content.
- Use `memory_lesson_save` after fixing a non-obvious bug or making a
  decision that future-you will want to remember.
- Skip recall for trivial questions or when you already have full context.

@alxjzx100 — 0 crystals / lessons

That symptom (memories captured, function calls counted, but consolidation outputs at 0) is the bug we shipped a fix for yesterday in v0.9.23 — CONSOLIDATION_ENABLED defaulted to false, so the consolidate pipeline (which writes crystals + semantic facts + procedural skills + lessons) never ran. Now defaults true whenever an LLM provider is configured. Upgrade with:

npm install -g @agentmemory/agentmemory@latest   # 0.9.24

Then either restart the daemon or run POST /agentmemory/consolidate-pipeline once to sweep your existing observation backlog. Graph nodes, lessons, and crystals should populate within a few minutes.

[alxjzx100]

Thank you for the detailed answer! It clarified a lot of my questions.

To be honest, I’ve been using v0.9.24 since yesterday across two different instances. Although I can see the consolidate pipeline calls in the UI and have even triggered them manually, I'm still not seeing any memories, crystals, or lessons.

On the bright side, the graph has started working - I already have over 400 nodes on one of the instances!

Will keep playing around.

[rohitg00]

@alxjzx100, that's the bottom-up gate problem, not a v0.9.24 regression. Each tier has a minimum-count gate:

• semantic facts: needs summaries.length >= 5 (src/functions/consolidation-pipeline.ts:63)
• procedural skills: needs 2+ memories where type === "pattern" seen in 2+ sessions (line 158). Most observations have type: "event", so this rarely fires until something classifies them as patterns.
• reflect / insights: each cluster needs facts + lessons + crystals >= 3 (src/functions/reflect.ts:235). With facts and lessons at 0, every cluster gets skipped.
• lessons: come from memory_lesson_save calls. The agent has to be told to call it. The tool description alone is not enough in practice.
• crystals: come from mem::crystallize or mem::auto-crystallize over the action stream. NOT part of mem::consolidate-pipeline.

Graph populates first because it's per-observation. No gates there.

A fresh install with 2 sessions and zero memory_lesson_save calls produces exactly: graph nodes growing, everything else 0. Matches your symptom.

Quick triage:

1. GET /agentmemory/sessions. Need 5+ summaries before the semantic tier fires.
2. GET /agentmemory/lessons. If 0, add the CLAUDE.md block from the answer above so the agent calls memory_lesson_save proactively.
3. POST /agentmemory/crystals/auto. Fires the auto-crystallizer over your existing action stream.

Filing a tracking issue for UI: each tab should show why it's empty (Crystals: 0, needs 2 patterns, currently at 0) instead of just blank. The dark-memory problem is two issues, not one: where they're consumed AND why they're not being produced.

[caioribeiroclw-pixel]

Useful thread. One debug surface I’d add when the high-order memory path lands: a tiny “memory consumption receipt” for each injected context block.

Not enforcement, just a low-cardinality trace that separates three failure modes users are already mixing together:

1. dark memory: facts/crystals/skills exist but were not eligible/injected
2. ignored memory: injected into context but never referenced before the first meaningful plan/edit/tool call
3. duplicate memory: same fact came through agentmemory hooks and the Claude bridge / MEMORY.md path

A minimal shape could be something like:

{
  "receipt_type": "agentmemory.memory_consumption_receipt.v1",
  "source": "SessionStart|PreCompact|manual_recall|bridge_sync",
  "budget_tokens": 1800,
  "candidate_counts": { "lessons": 10, "facts": 24, "crystals": 3, "procedural": 2 },
  "injected_counts": { "lessons": 5, "facts": 6, "crystals": 1, "procedural": 1 },
  "withheld_reasons": ["below_confidence", "over_budget", "duplicate_bridge_memory"],
  "acknowledged_before_plan": ["fact:api-auth-refresh", "lesson:db-migration-order"],
  "unreferenced_before_first_edit": ["crystal:legacy-cache-policy"]
}

The key is: don’t log raw memory text in the receipt. Just ids/types/counts/reasons. That gives users a way to tell whether “memory didn’t work” means retrieval/consolidation failed, injection/budgeting failed, or the agent simply didn’t attend to what was injected.

This would also make the bridge-vs-hook duplication answer much easier to verify in practice.

[ferhimedamine]

The three failure modes @caioribeiroclw-pixel identified — dark memory (exists but not injected), ignored memory (injected but not referenced), hallucinated memory (referenced but fabricated) — are exactly the right decomposition. Most memory systems only solve for storage and retrieval, not for the feedback loop between "what was retrieved" and "what actually influenced the output."

Two patterns from production that address the gap between "memories exist" and "memories influence behavior":

1. Access-frequency reinforcement: Every time a memory is successfully recalled AND the agent's output references it (detectable via simple substring or embedding similarity check), bump the memory's importance score. Memories that are recalled but never influence outputs are functionally dark — they take up retrieval slots without adding value. Decaying their importance naturally pushes them below the retrieval threshold over time, making room for memories that actually get used.

2. Consolidation with provenance: Rather than waiting for magical crystals/lessons to emerge, explicitly trigger consolidation passes that merge related low-level observations into higher-order summaries. The key is keeping provenance: the consolidated summary links back to its source memories, so you can trace WHY a summary exists and validate it against the originals if needed.

The "memory consumption receipt" idea is the right direction for observability. We track this as importance score + access count + last_accessed_at per memory — the combination tells you whether a memory is alive (high importance, recently accessed), stale (decayed importance, old access), or dark (never accessed despite being stored at high importance).

Python example showing the store → recall → analytics loop: https://github.com/Dakera-AI/dakera-py/blob/main/examples/analytics.py

[jdandrea]

I've moved my prior comment to its own thread (#980) since it's likely tangential, and I also don't want to hijack this thread besides. 😄