feat(memory): deterministic buffer turn-summaries (no LLM)

[jkyberneees]

Problem

The tier-2 buffer (short-term working memory) is injected into the system prompt on every turn, but each entry was a naive truncation of raw text — message[:97]+"..." (a byte slice that can split a UTF-8 rune) or shorten(s, 100). For an assistant turn this usually captured only filler ("Sure, I'll help with that. Let me start by…"), so the buffer carried almost no recall signal — close to dead weight.

Change

Centralize summarization in MemoryManager.AppendBuffer via a new deterministic, no-LLM, rune-safe summarizeForBuffer (no per-turn LLM calls — consistent with the hot-path cleanup in #12/#13). Pipeline:

1. Strip fenced code blocks (all-code → [code] placeholder, never blank)
2. Unwrap inline code/backticks
3. Conservative markdown strip (headings, bullets, blockquote, bold, links/images)
4. Collapse all whitespace to single spaces
5. Drop a leading filler clause — only when substantive text remains
6. Excerpt to 200 runes at a sentence (.!?) → word → hard-cut boundary; never splits a rune

All 8 call sites now pass raw text.

Why placement matters

Summarization lives in AppendBuffer, not Buffer.Append — because RestoreBuffer calls Buffer.Append directly with already-formatted, already-summarized lines. This keeps restore from re-processing (and corrupting) persisted summaries. Documented as an invariant on both functions and guarded by a test.

• Backward compatible: old session.Buffer lines restore verbatim; no migration.
• shorten() retained: still used by non-buffer callers (session labels at main.go:1792-1793, tool-result display :1840, serve.go:639).

Tests

• summarize_test.go: table-driven heuristic cases (empty, whitespace, passthrough, all-code, code+prose, markdown, inline-code, filler-dropped, filler-only-kept, whitespace-collapse) + sentence-boundary truncation + hard-cut single long token + multibyte boundary safety (世×N, utf8.ValidString).
• memory_test.go: TestAppendBufferCleansAndDoesNotMidWordCut (no \n/code-fence, valid UTF-8, ≤ cap) and TestRestoreBufferPreservesLinesVerbatim (invariant guard).

Verification

go build ./...          # ok
go vet ./...            # clean
gofmt -l ...            # empty
go test ./internal/memory/... -race   # ok
go test ./...           # ALL TESTS PASS

🤖 Generated with Claude Code

[jkyberneees]

🔍 AI Verification Protocol v5.2.7 — Certificate

Classification: NovelBehavior (new deterministic heuristic) · LOC_filtered: 341 (7 files) — under the 1,500 standard-pipeline ceiling.

Adversarial pass (Agent D) — findings & fixes

Red-teaming summarizeForBuffer surfaced two real defects, both now fixed (commit 860248c):

ID	Severity	Finding	Fix
D-01	🔴 Critical (correctness)	A single early sentence terminator collapsed the excerpt: "e.g., we should refactor…" → "e.g.…" (5 runes), "node.js…" → "node.…", "v1.2…" → "v1.…". The code picked the last .!? in the window, which with one early terminator destroyed the summary.	Prefer a sentence cut only when it lands ≥ half the window; otherwise fall back to the word boundary near the cap. Early-abbreviation inputs now yield ~196-rune excerpts.
D-02	🟡 Minor	An unclosed code fence (unmatched by the fenced-block regex) left a stray backtick.	Strip residual backticks after inline-code unwrap.

A third probe (filler false-positive on "Surely…") passed — the \b boundary correctly rejects it.

Axes

Axis	Result	Evidence
2.1 Semantic correctness	✅	Pipeline does what the contract claims; D-01/D-02 fixed and regression-tested.
2.3 / 2.8 Security & adversarial surface	✅	No new injection sink — buffer text was already model/user-sourced and injected; summarization only shortens/cleans. RE2 regexes (no ReDoS).
2.4 Structural integrity	✅	Single centralized funnel; invariant (summarize on write, not on restore) documented on both AppendBuffer/RestoreBuffer and guarded by a verbatim-restore test.
2.5 Behavioral exploration	✅	Adversarial probes for early-terminator, unclosed fence, filler edge cases, multibyte/hard-cut boundaries.
2.9 Documentation coverage	✅	No exported-symbol changes (summarizeForBuffer/maxBufferSummaryRunes unexported; AppendBuffer/RestoreBuffer signatures unchanged). No public API/doc surface affected.

Signals

go build ./... ✅ · go vet ./... ✅ · gofmt -l empty ✅ · go test ./internal/memory/... -race ✅ · go test ./... ✅ (all pass).

Verdict: AutoApprove after remediation. Both findings fixed and covered by tests before merge.