feat(coding-agents): retainTags / retainMetadata, with HINDSIGHT_RETAIN_TAGS (#3269, #2896) (#3346)

* feat(coding-agents): retainTags / retainMetadata with template placeholders

Closes #3269.

Every conversation retain carries `source:chat` and `harness:<id>` — what wrote
the memory, but nothing about where it came from. That is fine while each repo
has its own bank, since the bank is the answer. It stops being fine on a
deliberately shared bank, the setup in the issue: one bank holding cross-project
knowledge so facts recall everywhere, where a retained fact then carries no
record of the repository it came out of.

Both settings take `{placeholder}` templates resolved per retain, against the
vocabulary the dynamic bank id already uses plus what only a retain knows:

  {gitProject} {project} {harness} {bankId} {sessionId} {timestamp}
  {channel} {user}

  { "retainTags": ["project:{gitProject}"], "retainMetadata": {"repo": "{gitProject}"} }

{gitProject} is worktree-aware here too, so linked worktrees of one repo stamp a
single name rather than project:app and project:app-wt2.

The substitution itself moves to core/template.ts, shared with bank.ts rather
than duplicated — each call site keeps its own resolver map, because the valid
placeholders genuinely differ (a bank id cannot reference {bankId}).

Two things are deliberately not user-controllable. Built-in metadata is written
last and wins, and retainTags entries in the `source:`/`harness:` namespaces are
dropped with a warning: the documents list filters on those and resolves each
document's agent logo from them, so a template that could forge them would break
attribution for everyone reading the list.

Unconfigured, this adds nothing to a retain.

* docs(coding-agents): document retainTags/retainMetadata in the README, not the generated page

The docs page is generated from the integration's README by
hindsight-docs/scripts/sync-coding-agents-doc.mjs, and build-docs runs it with
--check. The first pass edited the generated page, so the build failed with
"docs page is out of date with the README".

Same content, moved to the source and re-synced (README, generated page and the
docs skill mirror). The row's cross-reference is plain text rather than an anchor
link because the generator flattens links.

* feat(coding-agents): HINDSIGHT_RETAIN_TAGS env override

Closes #2896.

The old Claude Code plugin had HINDSIGHT_RECALL_TAGS but no retain counterpart,
so per-project retain tagging could only be configured globally in the file. Now
that retainTags exists here, it joins the env surface on the same convention:
HINDSIGHT_ + the field in SCREAMING_SNAKE, still a FALLBACK the file wins over.

It is a list rather than a scalar, so a new ENV_LISTS branch splits on commas and
trims — blank entries dropped, so a trailing comma or "a,,b" is a typo rather
than an empty tag reaching the API.

  HINDSIGHT_RETAIN_TAGS="project:{gitProject},env:work"

retainMetadata deliberately gets no env form: it is map-valued, and per-key
branching doesn't survive flattening into one variable — the same rule already
applied to mapPathToBank, harnesses and banks.

Also corrects this file's header, which still claimed the plugin reads no
environment variables at all — untrue since ENV_KEYS was added.

Author: nicoloboschi

hindsight-docs/docs-integrations/coding-agents.md

@@ -270,8 +270,10 @@ Configuration is **one JSON file**: `~/.hindsight/coding-agent.json`. Layering,
 4. its `harnesses.<name>` section — per-agent override
 
 Environment variables are a **fallback**: the file wins wherever it sets a value, so adding env to
-an existing setup changes nothing. The map-valued settings (`mapPathToBank`, `harnesses`, `banks`)
-are file-only — nested branching doesn't survive flattening into one variable.
+an existing setup changes nothing. `retainTags` takes a comma-separated list
+(`HINDSIGHT_RETAIN_TAGS="project:{gitProject},env:work"`); entries are trimmed and blanks dropped.
+The map-valued settings (`mapPathToBank`, `harnesses`, `banks`, `retainMetadata`) are file-only —
+per-key branching doesn't survive flattening into one variable.
 
 There is deliberately no repo-carried config file — per-repo bank routing is `mapPathToBank`,
 per-agent differences are `harnesses.<name>`.
@@ -300,6 +302,8 @@ hook by Codex...), so one shared config serves several agents side by side:
 | `bankIdTemplate`        | `"coding-agent::{gitProject}"`       | dynamic bank id format; the default makes every agent share one bank per repo                                                                                                                                                       |
 | `mapPathToBank`         | —                                    | absolute path → bank; **longest prefix wins**; overrides everything                                                                                                                                                                 |
 | `resolveWorktrees`      | `true`                               | `{gitProject}`: linked worktrees share the main repo's bank                                                                                                                                                                         |
+| `retainTags`            | —                                    | extra tags on every session write-back, e.g. `["project:{gitProject}"]` — see **Recording where a memory came from** below                                                                                                          |
+| `retainMetadata`        | —                                    | extra metadata on every session write-back, e.g. `{"repo": "{gitProject}"}`                                                                                                                                                         |
 | `disabled`              | `false`                              | hard off-switch (inert plugin/hook — a no-memory baseline)                                                                                                                                                                          |
 | `reflectTimeoutMs`      | `120000`                             | session-reflect timeout (hook harnesses additionally cap it at 25s to fit the host's hook window); on timeout the session runs without reflect (recorded)                                                                           |
 | `pageRefreshEveryTurns` | `10`                                 | refetch the knowledge pages and re-inject the page roster + tool guide every N user turns                                                                                                                                           |
@@ -382,6 +386,29 @@ Coding memory is **per repository**. Resolution order for the working directory:
 The default `"coding-agent::{gitProject}"` is **harness-neutral**, so opencode, Claude Code, and Codex
 all share one memory per repo — use `"{harness}-{gitProject}"` to split per agent instead.
 
+### Recording where a memory came from
+
+With a bank per repo, the bank _is_ the answer to "where did this come from". On a deliberately
+**shared** bank — one bank holding cross-project knowledge so facts recall everywhere — it isn't:
+every memory looks alike. `retainTags` and `retainMetadata` stamp that provenance onto each session
+write-back:
+
+```jsonc
+{
+  "bankId": "shared", // one bank for everything
+  "retainTags": ["project:{gitProject}", "env:work"],
+  "retainMetadata": { "repo": "{gitProject}" },
+}
+```
+
+Recalls can then filter by `project:<repo>`, and every document shows which repository it came out
+of. Both accept the same placeholders as `bankIdTemplate` — `{gitProject}`, `{project}`,
+`{harness}`, `{channel}`, `{user}` — plus `{bankId}`, `{sessionId}` and `{timestamp}`.
+`{gitProject}` is worktree-aware here too, so every linked worktree of a repo stamps one name.
+
+The plugin's own `source:` and `harness:` tags are reserved: entries in those namespaces are ignored
+with a warning, so a document's agent attribution always reflects the agent that actually wrote it.
+
 ## Diagnostics & logging
 
 Two files, two audiences:

hindsight-integrations/coding-agents/README.md

@@ -263,8 +263,10 @@ Configuration is **one JSON file**: `~/.hindsight/coding-agent.json`. Layering,
 4. its `harnesses.<name>` section — per-agent override
 
 Environment variables are a **fallback**: the file wins wherever it sets a value, so adding env to
-an existing setup changes nothing. The map-valued settings (`mapPathToBank`, `harnesses`, `banks`)
-are file-only — nested branching doesn't survive flattening into one variable.
+an existing setup changes nothing. `retainTags` takes a comma-separated list
+(`HINDSIGHT_RETAIN_TAGS="project:{gitProject},env:work"`); entries are trimmed and blanks dropped.
+The map-valued settings (`mapPathToBank`, `harnesses`, `banks`, `retainMetadata`) are file-only —
+per-key branching doesn't survive flattening into one variable.
 
 There is deliberately no repo-carried config file — per-repo bank routing is `mapPathToBank`,
 per-agent differences are `harnesses.<name>`.
@@ -293,6 +295,8 @@ hook by Codex...), so one shared config serves several agents side by side:
 | `bankIdTemplate`        | `"coding-agent::{gitProject}"`       | dynamic bank id format; the default makes every agent share one bank per repo                                                                                                                                                       |
 | `mapPathToBank`         | —                                    | absolute path → bank; **longest prefix wins**; overrides everything                                                                                                                                                                 |
 | `resolveWorktrees`      | `true`                               | `{gitProject}`: linked worktrees share the main repo's bank                                                                                                                                                                         |
+| `retainTags`            | —                                    | extra tags on every session write-back, e.g. `["project:{gitProject}"]` — see **Recording where a memory came from** below                                                                                                          |
+| `retainMetadata`        | —                                    | extra metadata on every session write-back, e.g. `{"repo": "{gitProject}"}`                                                                                                                                                         |
 | `disabled`              | `false`                              | hard off-switch (inert plugin/hook — a no-memory baseline)                                                                                                                                                                          |
 | `reflectTimeoutMs`      | `120000`                             | session-reflect timeout (hook harnesses additionally cap it at 25s to fit the host's hook window); on timeout the session runs without reflect (recorded)                                                                           |
 | `pageRefreshEveryTurns` | `10`                                 | refetch the knowledge pages and re-inject the page roster + tool guide every N user turns                                                                                                                                           |
@@ -375,6 +379,29 @@ Coding memory is **per repository**. Resolution order for the working directory:
 The default `"coding-agent::{gitProject}"` is **harness-neutral**, so opencode, Claude Code, and Codex
 all share one memory per repo — use `"{harness}-{gitProject}"` to split per agent instead.
 
+### Recording where a memory came from
+
+With a bank per repo, the bank _is_ the answer to "where did this come from". On a deliberately
+**shared** bank — one bank holding cross-project knowledge so facts recall everywhere — it isn't:
+every memory looks alike. `retainTags` and `retainMetadata` stamp that provenance onto each session
+write-back:
+
+```jsonc
+{
+  "bankId": "shared", // one bank for everything
+  "retainTags": ["project:{gitProject}", "env:work"],
+  "retainMetadata": { "repo": "{gitProject}" },
+}
+```
+
+Recalls can then filter by `project:<repo>`, and every document shows which repository it came out
+of. Both accept the same placeholders as `bankIdTemplate` — `{gitProject}`, `{project}`,
+`{harness}`, `{channel}`, `{user}` — plus `{bankId}`, `{sessionId}` and `{timestamp}`.
+`{gitProject}` is worktree-aware here too, so every linked worktree of a repo stamps one name.
+
+The plugin's own `source:` and `harness:` tags are reserved: entries in those namespaces are ignored
+with a warning, so a document's agent attribution always reflects the agent that actually wrote it.
+
 ## Ingestion internals (no CLI)
 
 There is no user-facing ingest command — the deepen engine (`dist/deepen.js`) is spawned by every

hindsight-integrations/coding-agents/src/cline.ts

@@ -193,7 +193,7 @@ function createRuntime(workspaceRoot: string | undefined): RuntimeCore | undefin
     apiToken: cfg.apiToken,
     bank: resolved.bankId,
   });
-  return new RuntimeCore(client, resolved.bankId, cfg, HARNESS);
+  return new RuntimeCore(client, resolved.bankId, cfg, HARNESS, workspaceRoot || process.cwd());
 }
 
 const plugin: ClinePlugin = {

hindsight-integrations/coding-agents/src/core/bank.ts

@@ -23,6 +23,7 @@
 import { execFileSync } from "node:child_process";
 import { homedir } from "node:os";
 import { basename, dirname, join, normalize, sep } from "node:path";
+import { applyTemplate } from "./template";
 
 export interface BankConfig {
   bankId?: string;
@@ -38,7 +39,6 @@ const DEFAULT_BANK_NAME = "coding";
 // banks and avoid collisions with other Hindsight banks. Deliberately NOT `{harness}::…` (that would
 // split memory per agent, defeating cross-agent sharing).
 const DEFAULT_TEMPLATE = "coding-agent::{gitProject}";
-const PLACEHOLDER = /\{([a-zA-Z]+)\}/g;
 
 /** Main-worktree root for a directory inside a git repo (worktree- and bare-repo-aware), else null. */
 export function getProjectRootFromGit(directory: string): string | null {
@@ -103,18 +103,5 @@ export function deriveBankId(config: BankConfig, directory: string, harness = "c
     channel: () => process.env.HINDSIGHT_CHANNEL_ID || "default",
     user: () => process.env.HINDSIGHT_USER_ID || "anonymous",
   };
-  return (config.bankIdTemplate || DEFAULT_TEMPLATE).replace(PLACEHOLDER, (_, name: string) => {
-    const r = resolvers[name];
-    if (!r) {
-      console.error(
-        `hindsight: unknown bankIdTemplate placeholder "{${name}}" — valid: ` +
-          Object.keys(resolvers)
-            .sort()
-            .map((k) => `{${k}}`)
-            .join(", ")
-      );
-      return "unknown";
-    }
-    return r();
-  });
+  return applyTemplate(config.bankIdTemplate || DEFAULT_TEMPLATE, resolvers, "bankIdTemplate");
 }

hindsight-integrations/coding-agents/src/core/chat.test.ts

@@ -270,6 +270,45 @@ describe("retainLiveSession — incremental write-back", () => {
     ]);
   });
 
+  it("stamps configured tags and metadata onto the write-back", async () => {
+    const { retain, client } = stubClient();
+    await retainLiveSession(client, "s1", turns(2), "2026-01-01T00:00:00Z", "codex", {
+      cursors: memoryCursorStore(),
+      stamp: { tags: ["project:acme-api", "env:work"], metadata: { repo: "acme-api" } },
+    });
+    expect(retain.mock.calls[0][3]).toEqual([
+      "project:acme-api",
+      "env:work",
+      "source:chat",
+      "harness:codex",
+    ]);
+    expect(retain.mock.calls[0][5].metadata).toMatchObject({
+      repo: "acme-api",
+      source: "chat",
+      harness: "codex",
+    });
+  });
+
+  it("keeps built-in metadata authoritative and does not double a tag", async () => {
+    // The documents list filters on `source:chat` and draws its agent logo from `metadata.harness`,
+    // so the built-ins are written last and win. (retainTags entries in those namespaces are
+    // dropped earlier, at the source — see retain-stamp.test.ts.)
+    const { retain, client } = stubClient();
+    await retainLiveSession(client, "s1", turns(2), "2026-01-01T00:00:00Z", "codex", {
+      cursors: memoryCursorStore(),
+      stamp: {
+        tags: ["source:chat", "env:work"],
+        metadata: { harness: "not-codex", source: "elsewhere", session_id: "spoofed" },
+      },
+    });
+    expect(retain.mock.calls[0][3]).toEqual(["source:chat", "env:work", "harness:codex"]);
+    expect(retain.mock.calls[0][5].metadata).toMatchObject({
+      harness: "codex",
+      source: "chat",
+      session_id: "s1",
+    });
+  });
+
   it("keeps the write-back when the capability probe itself fails", async () => {
     const retain = vi.fn().mockResolvedValue(undefined);
     const client = {

hindsight-integrations/coding-agents/src/core/chat.ts

@@ -5,6 +5,7 @@
  */
 import type { HindsightClient } from "./hindsight";
 import { fingerprintTurns, planRetain, type RetainCursorStore } from "./retain-cursor";
+import type { RetainStamp } from "./retain-stamp";
 import type { ChatSession } from "./types";
 import { uuidV5 } from "./uuid";
 import { pool } from "./util";
@@ -152,13 +153,13 @@ export async function retainLiveSession(
   turns: TransportTurn[],
   startTs: string,
   harness?: string,
-  opts: { cursors?: RetainCursorStore } = {}
+  opts: { cursors?: RetainCursorStore; stamp?: RetainStamp } = {}
 ): Promise<void> {
   const cursors = opts.cursors;
-  if (!cursors) return writeSession(client, sessionId, turns, startTs, harness);
+  if (!cursors) return writeSession(client, sessionId, turns, startTs, harness, opts.stamp);
   // Serialised so the plan is made against the previous write-back's CONFIRMED cursor (see above).
   return serialize(cursors, sessionId, () =>
-    writeSession(client, sessionId, turns, startTs, harness, cursors)
+    writeSession(client, sessionId, turns, startTs, harness, opts.stamp, cursors)
   );
 }
 
@@ -168,6 +169,7 @@ async function writeSession(
   turns: TransportTurn[],
   startTs: string,
   harness?: string,
+  stamp?: RetainStamp,
   cursors?: RetainCursorStore
 ): Promise<void> {
   const refId = `conversation:${sessionId}`;
@@ -196,7 +198,15 @@ async function writeSession(
     content,
     "coding agent session",
     refId,
-    ["source:chat", ...(harness ? [`harness:${harness}`] : [])],
+    // Configured tags first, built-ins last and deduped: `source:chat` and `harness:<id>` are what
+    // the documents list filters and draws its agent logo from, so a template cannot displace them.
+    [
+      ...new Set([
+        ...(stamp?.tags ?? []),
+        "source:chat",
+        ...(harness ? [`harness:${harness}`] : []),
+      ]),
+    ],
     "conversation",
     {
       timestamp: startTs,
@@ -205,6 +215,7 @@ async function writeSession(
       // the original operation instead of extracting (or appending) twice.
       operationId: uuidV5(`${client.bank}\n${refId}\n${plan.mode}\n${content}`),
       metadata: {
+        ...stamp?.metadata, // configured first: the built-ins below win on any key collision
         source: "chat",
         session_id: sessionId,
         ref_id: refId,

hindsight-integrations/coding-agents/src/core/config.test.ts

@@ -2,7 +2,7 @@ import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from "node:fs";
 import { tmpdir } from "node:os";
 import { join } from "node:path";
 import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
-import { loadConfig, applyBankConfig, resolveConfig } from "./config";
+import { loadConfig, applyBankConfig, readEnvConfig, resolveConfig } from "./config";
 
 let root: string;
 let globalCfg: string;
@@ -212,3 +212,52 @@ describe("environment fallback", () => {
     expect(cfg.surveyModel).toBe("haiku");
   });
 });
+
+describe("retainTags / retainMetadata", () => {
+  it("default to empty, so a retain is unchanged unless configured", () => {
+    const cfg = resolveConfig({});
+    expect(cfg.retainTags).toEqual([]);
+    expect(cfg.retainMetadata).toEqual({});
+  });
+
+  it("carries templates through verbatim — resolution happens per retain", () => {
+    const cfg = resolveConfig({
+      retainTags: ["project:{gitProject}"],
+      retainMetadata: { repo: "{gitProject}" },
+    });
+    expect(cfg.retainTags).toEqual(["project:{gitProject}"]);
+    expect(cfg.retainMetadata).toEqual({ repo: "{gitProject}" });
+  });
+
+  it("ignores non-string entries rather than failing the whole retain", () => {
+    // A config typo (a number, a nested object) would otherwise reach the API as a tag.
+    const cfg = resolveConfig({
+      retainTags: ["ok", 42, null, "  "] as unknown as string[],
+      retainMetadata: { good: "x", bad: { nested: true } } as unknown as Record<string, string>,
+    });
+    expect(cfg.retainTags).toEqual(["ok"]);
+    expect(cfg.retainMetadata).toEqual({ good: "x" });
+  });
+});
+
+describe("HINDSIGHT_RETAIN_TAGS", () => {
+  it("reads a comma-separated list — the env form of retainTags (#2896)", () => {
+    expect(
+      readEnvConfig({ HINDSIGHT_RETAIN_TAGS: "project:{gitProject},env:work" }).retainTags
+    ).toEqual(["project:{gitProject}", "env:work"]);
+  });
+
+  it("trims entries and drops empties, so a trailing comma is not an empty tag", () => {
+    expect(readEnvConfig({ HINDSIGHT_RETAIN_TAGS: " a , ,b, " }).retainTags).toEqual(["a", "b"]);
+  });
+
+  it("is absent when unset or empty, leaving the file value alone", () => {
+    expect(readEnvConfig({}).retainTags).toBeUndefined();
+    expect(readEnvConfig({ HINDSIGHT_RETAIN_TAGS: "" }).retainTags).toBeUndefined();
+    expect(readEnvConfig({ HINDSIGHT_RETAIN_TAGS: " , " }).retainTags).toBeUndefined();
+  });
+
+  it("has no retainMetadata counterpart — map-valued settings stay file-only", () => {
+    expect(readEnvConfig({ HINDSIGHT_RETAIN_METADATA: "repo=x" }).retainMetadata).toBeUndefined();
+  });
+});