docs: memory build-vs-gate decision (Position C) + universal gate pattern + arch.md index

[hanwencheng]

Summary

Pre-implementation research + decision + docs. No code changes. Three threads, plus source-of-truth hygiene:

1. Decision: memory = engine / store / gate (Position C)

docs/research/memory-build-vs-gate-decision.md records the call: AgentKeys owns the gated, K3-encrypted, per-actor, audited memory STORE + the GATE; the ranking/extraction ENGINE is delegated to the ecosystem (mem0-self-hosted / Claude memory tool / Hermes-native / agentmemory).

• Rejects "pure gate, use their store" — it breaks the founding privacy requirement on 3 of 5 axes (bytes not in our S3; vendor/LLM sees plaintext; Claude-memory couples to the LLM you wanted swappable).
• Claude's own memory tool validates the split: the engine is closed, but the store is yours (BetaAbstractMemoryTool backs it with "S3, encrypted files"). AgentKeys can literally be a Claude memory backend.
• Durability argument: the engine is per-agent and disposable; the user's memory + access policy is per-user and durable — so own the durable layer.

2. Reframe: agentkeys-memory-design.md → "gated memory backend"

• Engine sections (/search, /rebuild-index, vector index, BM25, embedding rotation) demoted to optional pluggable stages E1/E2 with 🔌 ENGINE banners.
• §9 stages split into CORE (store+gate: M-1 → M0 → M1 → M1.5 namespace gate → M2 → M3 → M4) and ENGINE (E1/E2, deferred).
• The build shrinks; the moat (cap-gated encrypted per-actor store) sharpens.

3. Generalization: the universal gate pattern

docs/research/universal-gate-pattern.md — the engine/store/gate split generalizes to every worker (memory / email / payment / home-IoT). The cap-token is the universal policy carrier with four primitives:

Primitive	Example
Operation (read/write/execute)	"read email but not send"
Resource scope (string-set)	"only the kids' room IoT devices"
Quantitative limit	"no single spend > $20"
Attribute constraint (allow/deny set)	"no fast food" → deny_attrs:["mcc:5814"]

Plus the determinism principle: the gate decides authorization deterministically on structured attributes; semantic judgment ("is this a burger?") happens upstream and reduces to an attribute the gate checks — never an LLM judgment at the gate (non-deterministic, prompt-injectable, unauditable). New workers (e.g. agentkeys-worker-iot) become template instances.

4. arch.md indexing + source-of-truth hygiene

Per the architecture-as-source-of-truth policy, arch.md now links outward to the new docs, and I fixed pre-existing defects found while anchoring:

• Added a "Design records & research" index block (decision, gated-backend spec, universal-gate pattern, survey).
• §15.2 memory-service: filled the literal [TODO link] placeholder with the Position-C anchors + store-vs-engine framing + the 4 types × namespaces.
• Fixed broken companion links — signer-protocol.md / threat-model-key-custody.md pointed at docs/ but live in docs/spec/.
• Removed stray committed reasoning ("Wait —"/"Hmm, line 21 is a stray" leaked into the header).
• §17.5: collapsed a corrupted cap-endpoint table — /v1/cap/memory-search was duplicated 41× (88 rows total) — down to the 5 canonical rows; fixed the intro count. arch.md: 2160 → 2077 lines.

0 broken internal links remain in arch.md.

New / changed files

• New: docs/research/memory-build-vs-gate-decision.md, docs/research/universal-gate-pattern.md, docs/research/agentmemory-and-claw-code-followup.md
• Modified: docs/plan/agentkeys-memory-design.md (reframe), docs/arch.md (index + defect cleanup), docs/research/README.md (index rows)

Test plan

• arch.md internal links: 0 broken (Python link-resolver over all ](...) targets).
• §17.5 cap table: 5 canonical rows, memory-search duplication removed.
• All docs/research/*.md present in docs/research/README.md index.
• Reframed plan is internally consistent (CORE/ENGINE numbering, ENGINE banners, no dangling M5/M6 in live content).
• Reviewer gut-check on Position C before any M-stage code starts.

Notes for reviewers

• This supersedes the framing of the memory plan as merged in docs: AI memory worker design + reorganize docs/ to canonical layout #106 — same file, now reframed to the gated-backend scope. The store/gate half is unchanged; the engine half is now explicitly pluggable/out-of-build.
• Per the /create-pr policy: committed from the Claude Code worktree (git), pushed via jj from the main repo; no Co-Authored-By line.
• The agentkeys-docs skill referenced in CLAUDE.md does not exist on disk — the docs organization here was done manually against the documented layout policy. Worth either creating that skill or dropping the reference (out of scope for this PR).