feat(memory): Design and implement inject_memories retrieval strategy

[hamza-jeddad]

Background

Part of #3011 — the inject_memories turn_start builtin proactively retrieves relevant memories before every model call. This issue is scoped specifically to the retrieval strategy: how keywords are extracted from the user prompt, how SearchMemories is called, and what the end-to-end retrieval pipeline looks like.

Confirmed Design Decisions

1. Prompt availability in turn_start

executeTurnStartHooks must be updated to populate hooks.Input.LastUserMessage with sess.GetLastUserMessageContent(). This follows the exact pattern already used in executeStopHooks and makes the user's current prompt available to the builtin without breaking the stateless-builtin contract:

// pkg/runtime/hooks.go — executeTurnStartHooks
return contextMessages(r.dispatchHook(ctx, a, hooks.EventTurnStart, &hooks.Input{
    SessionID:       sess.ID,
    LastUserMessage: sess.GetLastUserMessageContent(), // new
}, events))

2. Configurable keyword extraction strategy

The builtin supports two strategies, selected via YAML config:

Strategy	Description
local (default)	Reuses the BM25 tokenizer from pkg/rag/strategy/bm25.go: lowercase → strip punctuation → split on whitespace → drop tokens ≤ 2 chars → filter stopwords (the existing 20-word list). No external calls.
llm	Asks the LLM to extract a compact keyword list from the prompt before calling SearchMemories. Costs one extra inference call per turn; useful for complex/multi-topic prompts.

YAML example:

agents:
  my_agent:
    tools:
      - type: memory
    inject_memories: true
    inject_memories_strategy: local   # or "llm"
    max_inject_memories: 20

3. Retrieval pipeline (end-to-end)

User prompt text (Input.LastUserMessage)
    │
    ▼
Keyword extraction (local BM25 tokenizer OR LLM)
    │
    ▼ keywords []string
SearchMemories(ctx, strings.Join(keywords, " "), "")
    │  ← AND-match LIKE query; all keywords must appear in memory text
    │
    ▼ []database.UserMemory (in insertion order)
Cap to max_inject_memories (default: 20)
    │
    ▼
Format as XML block and return as turn_start AdditionalContext (transient, never persisted)

4. Fallback behaviour

No fallback to GetMemories. SearchMemories is always used. When the prompt is short or empty, no keywords are extracted and SearchMemories("") returns all memories (since an empty query returns everything in the SQLite layer). The max_inject_memories cap prevents context bloat.

5. Output format

Memories are injected as an XML-wrapped block:

<memories>
  <memory id="..." category="preference">User prefers dark mode</memory>
  <memory id="..." category="project">Deploy target is AWS us-east-1</memory>
</memories>

Implementation Checklist

• pkg/runtime/hooks.go — populate Input.LastUserMessage in executeTurnStartHooks
• pkg/hooks/builtins/inject_memories.go — new turn_start builtin; implement local and llm strategies
• pkg/hooks/builtins/builtins.go — register InjectMemories constant + handler; add InjectMemoriesStrategy to AgentDefaults
• pkg/config/latest/types.go — add InjectMemoriesStrategy string to AgentConfig
• pkg/agent/agent.go — add InjectMemoriesStrategy() string accessor
• Unit tests: mock DB, assert correct keyword extraction for both strategies, assert XML output shape
• Integration test: turn_start injects relevant memories, empty-prompt returns all memories up to cap

Acceptance Criteria

• With inject_memories: true and inject_memories_strategy: local, relevant memories (matching BM25-extracted keywords) appear as a transient system message before every LLM call
• With inject_memories_strategy: llm, an LLM call extracts keywords before SearchMemories is invoked
• An empty/very short prompt with no extractable keywords causes SearchMemories("") to return all memories (capped at max_inject_memories)
• Injected memories are never persisted to the session transcript
• inject_memories: false (default) produces no change in behaviour for existing agents
• The <memories> XML block is correctly formatted with id and category attributes