feat(memory-core): archiveMemoriesByAgentIdentity tombstone + recall-exclusion (#13384) (#13385)

* feat(memory-core): archiveMemoriesByAgentIdentity tombstone + recall-exclusion (#13384)

The Memory-Core primitive the Fleet agent-removal reconciliation consumes via the
archiveMemoriesFn seam (split from #13190 per the 1-ticket-1-PR contract).

- archiveMemoriesByAgentIdentity({agentIdentity, reason, dryRun}): dual-store tombstone
  (Chroma metadata + graph-node SQL + in-memory node-cache), idempotent, keyed on the
  stamped @<github-username> write-identity.
- unarchiveMemoriesByAgentIdentity: reversible (Chroma empty-string + graph json_remove + cache).
- archivedAt-exclusion across recall paths: queryMemories + listMemories (Chroma),
  queryRecentTurns (graph SQL), getContextFrontier frontier (GraphService).
- Leak-test: 5 cases (exclusion on both Chroma paths, dryRun, idempotency, identity-scoping,
  fail-closed) + 59 existing memory-core specs green (no regression).

Refs #13190.

* fix(memory-core): durable tombstone marker survives graph-projection lag (#13384)

archiveMemoriesByAgentIdentity stamped archivedAt on the Chroma rows and on ALREADY-projected graph nodes, but a record embedded into Chroma while still graph-pending (graphProjectionVersion:1, no .graph.jsonl marker) had no node to stamp. The deferred drainPendingGraphProjections -> _projectMemoryToGraph later minted a fresh AGENT_MEMORY node from the pre-archive WAL snapshot, silently re-introducing the row un-tombstoned. Chroma embed and graph projection are independent derived states that catch up on different clocks.

Fix: a durable, globally-visible RLS-exempt ARCHIVED_AGENT_IDENTITY:<identity> marker (upsertGlobalNode), written by the archive op and removed by unarchive, plus _archivedIdentityState + _withArchiveState helpers. _projectMemoryToGraph now replays the tombstone onto the projected node when the identity is archived, so a lag-window archive survives projection (and a restart).

Test: a real (non-stubbed) projection-lag behavior test + durable-marker round-trip exercising the actual _projectMemoryToGraph path; 8/8 green. The existing recall-exclusion suite is unaffected (its db stub gains transaction/removeNode for the marker ops).

Remaining before re-request (NOT this commit): the other AGENT_MEMORY mint/overlay sites — the buildMiniSummary re-upsert, mergeMiniSummary, and the _readPendingWalRecencyRows recency overlay — get the same archive-awareness, then the leak test extends across all four sites.

* fix(memory-core): archive-aware projection across all four mint/overlay sites (#13384)

Extends the durable-marker core to the remaining three AGENT_MEMORY sites the projection-lag analysis mapped, so a tombstone set during projection lag survives every (re)mint and recall surface — not just the drain path:

- buildMiniSummary inline re-upsert: wrapped in _withArchiveState (archive-vs-summary race no longer drops archivedAt). - updateMemoryMiniSummary: the addNodes overwrite now replays the tombstone from the durable marker (merge-vs-archive race). - _readPendingWalRecencyRows: identity-level short-circuit excludes an archived identity's graph-PENDING rows from recency recall (the graph-SQL path already excluded archivedAt; this closes the WAL-pending overlay gap).

Test: the projection-lag suite extends with a recency-overlay short-circuit assertion (9/9 green). Full memory-core unit suite regression-free (13 failed = pre-existing local-LM-dependent specs on clean dev; 510 passed = clean + the 4 new tests).

* test(memory-core): public graph-recall leak coverage for archive — queryRecentTurns + getContextFrontier (#13384)

Addresses the cycle-2 CHANGES_REQUESTED: the prior tests proved the projection-lag mechanism via private helpers, but the source issue's ACs name queryRecentTurns + getContextFrontier (the public graph-backed recall surfaces), which now have direct coverage against the real in-memory SQLite graph (UNIT_TEST_MODE ':memory:', mirroring QueryRecentTurns.spec).

queryRecentTurns: full end-to-end — addMemory -> archiveMemoriesByAgentIdentity (real op: Chroma sweep + graph-SQL stamp + durable marker) -> the row is excluded from the public recency surface -> unarchive restores it.

getContextFrontier: its frontier is agent-identity RLS-scoped (isRlsVisible keys on the acting agentIdentityNodeId, never a tenant userId), so a tenant memory node is never one of its strategic neighbors. The test exercises the public surface's archivedAt exclusion directly with a globally-visible frontier neighbor carrying the same node-cache archivedAt state the archive op's coherence sweep stamps (the op's stamping itself is pinned in the sibling spec). 2/2 green; 22 passed alongside ArchiveByIdentity + QueryRecentTurns (no cross-spec interference).

* test(memory-core): point ArchiveByIdentity header at the public-recall spec — gpt cycle-2 RA3 (#13384)

The header claimed the graph-SQL (queryRecentTurns) + frontier (getContextFrontier) exclusion was 'covered by a separate integration assertion'. That assertion now exists — MemoryService.ArchiveByIdentity.PublicRecall.spec.mjs — so the comment points at it directly. Completes the third of gpt's cycle-2 Required Actions; RA1 (queryRecentTurns public test) + RA2 (getContextFrontier public test) landed in 51d4561.

Author: neo-opus-grace

ai/services/memory-core/GraphService.mjs

@@ -869,7 +869,7 @@ class GraphService extends Base {
                 let node       = this.db.nodes.get(adjacentId);
 
                 // Actively filter out CLOSED structural paths plus RLS-invisible nodes/edges.
-                if (node && isRlsVisible(node, rlsUserId) && isRlsVisible(e, rlsUserId) && node.properties?.state !== 'CLOSED') {
+                if (node && isRlsVisible(node, rlsUserId) && isRlsVisible(e, rlsUserId) && node.properties?.state !== 'CLOSED' && !node.properties?.archivedAt) {
                     topology.strategicNeighbors.push({
                         id              : node.id,
                         type            : node.label,

ai/services/memory-core/MemoryService.mjs

@@ -527,7 +527,9 @@ class MemoryService extends Base {
                         name: `Memory: ${timestamp}`,
                         description: `Agent thought flow inside session ${sessionId}.`,
                         semanticVectorId: memoryId,
-                        properties: {...memoryProperties, miniSummary}
+                        // Archive-aware re-upsert: a tombstone set between the initial projection and this
+                        // post-summary re-mint must not be dropped (see _withArchiveState).
+                        properties: this._withArchiveState({...memoryProperties, miniSummary})
                     });
                 }
             }).catch(() => {});
@@ -621,7 +623,10 @@ class MemoryService extends Base {
             name: `Memory: ${timestamp}`,
             description: `Agent thought flow inside session ${sessionId}.`,
             semanticVectorId: memoryId,
-            properties: memoryProperties
+            // Archive-aware: a tombstone set while this record was graph-pending (the projection-lag
+            // window) is replayed onto the node here, so a deferred projection cannot reintroduce an
+            // archived row un-tombstoned. See _withArchiveState + the durable ARCHIVED_AGENT_IDENTITY marker.
+            properties: this._withArchiveState(memoryProperties)
         });
 
         // Stamp write-time provenance when the transport resolved a real AgentIdentity.
@@ -806,6 +811,14 @@ class MemoryService extends Base {
      */
     async _readPendingWalRecencyRows({identity, userId, before, excludeIds = new Set()} = {}) {
         try {
+            // Identity-level tombstone short-circuit: archiveMemoriesByAgentIdentity sweeps ALL of an
+            // identity's memories + writes a durable marker, and the graph-SQL recency path already
+            // excludes archivedAt — so an archived identity's graph-PENDING rows must be excluded here
+            // too, else they leak into recency recall during projection lag.
+            if (this._archivedIdentityState(identity)) {
+                return [];
+            }
+
             // No raw read-limit here: readPendingWalRecords walks each daily segment in append
             // (oldest-first) order, so capping the raw count would drop the NEWEST graph-pending rows
             // — exactly the just-written, not-yet-projected turns the read-after-write overlay must
@@ -913,6 +926,9 @@ class MemoryService extends Base {
             let records = result.ids.map((id, index) => {
                 const metadata = result.metadatas[index] || {};
 
+                // Tombstone exclusion: archived rows are dropped from recall.
+                if (metadata.archivedAt) return null;
+
                 return {
                     id,
                     sessionId: metadata.sessionId,
@@ -927,7 +943,7 @@ class MemoryService extends Base {
                     toolsUsed: metadata.toolsUsed || null,
                     _userId  : metadata.userId
                 };
-            });
+            }).filter(Boolean); // Tombstone exclusion (archived rows returned null above)
 
             if (userId && policy === 'legacy') {
                 records = records.filter(r => !r._userId || r._userId === userId || r._userId === SHARED_USER_ID);
@@ -958,6 +974,215 @@ class MemoryService extends Base {
         }
     }
 
+    /**
+     * @summary Tombstones every memory stamped with `agentIdentity` so it stops surfacing in recall,
+     * while RETAINING the rows (forensics) and staying operator-reversible — the Memory-Core
+     * primitive the Fleet agent-removal reconciliation consumes via the `archiveMemoriesFn` seam.
+     *
+     * Sets `archivedAt` (+ `archivedReason`) across BOTH stores — the Chroma row metadata (read by
+     * `queryMemories` / `listMemories`) and the graph `AGENT_MEMORY` node (read by `queryRecentTurns`
+     * and the topology frontier) — so the recall paths that exclude `archivedAt` stop returning the
+     * rows. Idempotent: already-tombstoned rows are skipped. Keys on the STAMPED write-identity
+     * (`@<github-username>` for FM stdio+PAT agents, per the `add_memory` stamping) — NOT a registry id.
+     *
+     * @param {Object}  options
+     * @param {String}  options.agentIdentity The stamped `metadata.agentIdentity` value to sweep.
+     * @param {String}  [options.reason] Tombstone provenance, stored per row as `archivedReason`.
+     * @param {Boolean} [options.dryRun=false] Preview `matchedCount` without tombstoning.
+     * @returns {Promise<Object>} `{agentIdentity, matchedCount, archivedCount, dryRun}`.
+     */
+    async archiveMemoriesByAgentIdentity({agentIdentity, reason, dryRun = false} = {}) {
+        if (!agentIdentity) {
+            return {error: 'Bad Request', message: "'agentIdentity' is required.", code: 'MISSING_AGENT_IDENTITY'};
+        }
+
+        try {
+            const collection  = await StorageRouter.getMemoryCollection(),
+                  matched     = await collection.get({where: {agentIdentity}, include: ['metadatas']}),
+                  matchedIds  = matched.ids       || [],
+                  matchedMeta = matched.metadatas || [],
+                  live        = []; // not-already-tombstoned rows (idempotent: a re-run archives 0 new)
+
+            for (let i = 0; i < matchedIds.length; i++) {
+                if (!(matchedMeta[i] && matchedMeta[i].archivedAt)) {
+                    live.push({id: matchedIds[i], meta: matchedMeta[i] || {}});
+                }
+            }
+
+            if (dryRun) {
+                return {agentIdentity, matchedCount: matchedIds.length, archivedCount: 0, dryRun: true};
+            }
+
+            const archivedAt     = new Date().toISOString(),
+                  archivedReason = reason || '';
+
+            // Chroma side: stamp archivedAt onto the live rows' metadata (full-metadata-preserving).
+            if (live.length > 0) {
+                await collection.update({
+                    ids      : live.map(r => r.id),
+                    metadatas: live.map(r => ({...r.meta, archivedAt, archivedReason}))
+                });
+            }
+
+            // Graph side: stamp the projected AGENT_MEMORY nodes. The `archivedAt IS NULL` guard keeps
+            // it idempotent + scoped to the live rows. Mirrors the MailboxService `archivedAt` model.
+            const sqlite = GraphService.db?.storage?.db;
+            if (sqlite) {
+                sqlite.prepare(`
+                    UPDATE Nodes
+                    SET data = json_set(json_set(data, '$.properties.archivedAt', ?), '$.properties.archivedReason', ?)
+                    WHERE json_extract(data, '$.label') = 'AGENT_MEMORY'
+                      AND json_extract(data, '$.properties.agentIdentity') = ?
+                      AND json_extract(data, '$.properties.archivedAt') IS NULL
+                `).run(archivedAt, archivedReason, agentIdentity);
+            }
+
+            // In-memory node-cache coherence: getContextFrontier reads GraphService.db.nodes (the
+            // cache), which the SQL UPDATE above does NOT touch. The graph node id === the memory id
+            // (_projectMemoryToGraph), so mirror archivedAt onto any cached node so the topology
+            // frontier excludes it too.
+            const nodeCache = GraphService.db?.nodes;
+            if (nodeCache) {
+                for (const {id} of live) {
+                    const cached = nodeCache.get(id);
+                    if (cached && cached.properties) {
+                        cached.properties.archivedAt     = archivedAt;
+                        cached.properties.archivedReason = archivedReason;
+                    }
+                }
+            }
+
+            // Durable tombstone marker — survives graph-projection lag. The SQL UPDATE + node-cache sweep
+            // above only reach ALREADY-projected nodes; a row embedded into Chroma but still graph-pending
+            // (graphProjectionVersion:1, no .graph.jsonl marker) has no node to stamp yet. This global,
+            // RLS-exempt marker lets the deferred projection drain (_projectMemoryToGraph → _withArchiveState)
+            // and the recency overlay tombstone it once projection catches up — even across a restart.
+            // Set unconditionally (not gated on live.length) so an identity archived entirely during
+            // projection lag is still covered.
+            GraphService.upsertGlobalNode({
+                id         : `ARCHIVED_AGENT_IDENTITY:${agentIdentity}`,
+                type       : 'ARCHIVED_AGENT_IDENTITY',
+                name       : `Archived identity: ${agentIdentity}`,
+                description: 'Tombstone marker: deferred graph projection + recency overlay exclude this identity.',
+                properties : {agentIdentity, archivedAt, archivedReason}
+            });
+
+            logger.info(`[MemoryService] archiveMemoriesByAgentIdentity('${agentIdentity}'): matched ${matchedIds.length}, archived ${live.length}`);
+            return {agentIdentity, matchedCount: matchedIds.length, archivedCount: live.length, dryRun: false};
+        } catch (error) {
+            logger.error('[MemoryService] Error archiving memories by agentIdentity:', error);
+            return {error: 'Failed to archive memories', message: error.message, code: 'MEMORY_ARCHIVE_ERROR'};
+        }
+    }
+
+    /**
+     * @summary Reverses {@link archiveMemoriesByAgentIdentity} — clears `archivedAt` for `agentIdentity`
+     * so the rows recall again (operator re-provisioning). A hard purge is a separate
+     * explicit op, never the tombstone default.
+     *
+     * Chroma metadata cannot hold `null`, so the cleared marker is the empty string `''` (falsy → the
+     * recall-exclusions re-admit the row); the graph node's marker is removed via `json_remove`.
+     *
+     * @param {Object} options
+     * @param {String} options.agentIdentity The stamped identity to restore.
+     * @returns {Promise<Object>} `{agentIdentity, restoredCount}`.
+     */
+    async unarchiveMemoriesByAgentIdentity({agentIdentity} = {}) {
+        if (!agentIdentity) {
+            return {error: 'Bad Request', message: "'agentIdentity' is required.", code: 'MISSING_AGENT_IDENTITY'};
+        }
+
+        try {
+            const collection  = await StorageRouter.getMemoryCollection(),
+                  matched     = await collection.get({where: {agentIdentity}, include: ['metadatas']}),
+                  matchedIds  = matched.ids       || [],
+                  matchedMeta = matched.metadatas || [],
+                  restore     = [];
+
+            for (let i = 0; i < matchedIds.length; i++) {
+                if (matchedMeta[i] && matchedMeta[i].archivedAt) {
+                    restore.push({id: matchedIds[i], meta: matchedMeta[i]});
+                }
+            }
+
+            if (restore.length > 0) {
+                await collection.update({
+                    ids      : restore.map(r => r.id),
+                    metadatas: restore.map(r => ({...r.meta, archivedAt: '', archivedReason: ''}))
+                });
+            }
+
+            const sqlite = GraphService.db?.storage?.db;
+            if (sqlite) {
+                sqlite.prepare(`
+                    UPDATE Nodes
+                    SET data = json_remove(data, '$.properties.archivedAt', '$.properties.archivedReason')
+                    WHERE json_extract(data, '$.label') = 'AGENT_MEMORY'
+                      AND json_extract(data, '$.properties.agentIdentity') = ?
+                      AND json_extract(data, '$.properties.archivedAt') IS NOT NULL
+                `).run(agentIdentity);
+            }
+
+            // In-memory node-cache coherence (mirror of the archive sweep): clear the cached marker.
+            const nodeCache = GraphService.db?.nodes;
+            if (nodeCache) {
+                for (const {id} of restore) {
+                    const cached = nodeCache.get(id);
+                    if (cached && cached.properties) {
+                        delete cached.properties.archivedAt;
+                        delete cached.properties.archivedReason;
+                    }
+                }
+            }
+
+            // Remove the durable tombstone marker so future projections + the recency overlay re-admit
+            // the identity. Safe when absent (the identity was never archived) — removeNodes no-ops.
+            GraphService.removeNodes([`ARCHIVED_AGENT_IDENTITY:${agentIdentity}`]);
+
+            logger.info(`[MemoryService] unarchiveMemoriesByAgentIdentity('${agentIdentity}'): restored ${restore.length}`);
+            return {agentIdentity, restoredCount: restore.length};
+        } catch (error) {
+            logger.error('[MemoryService] Error unarchiving memories by agentIdentity:', error);
+            return {error: 'Failed to unarchive memories', message: error.message, code: 'MEMORY_UNARCHIVE_ERROR'};
+        }
+    }
+
+    /**
+     * @summary Looks up the durable tombstone marker for an agent identity.
+     *
+     * The archive op ({@link MemoryService#archiveMemoriesByAgentIdentity}) writes a global,
+     * RLS-exempt `ARCHIVED_AGENT_IDENTITY:<identity>` node; this reads it so the deferred
+     * graph-projection drain + the recency overlay can tombstone a record whose graph node did
+     * not exist when the archive ran (the projection-lag leak: Chroma embed and graph projection
+     * are independent derived states that catch up on different clocks).
+     * @param {String} agentIdentity The stamped write-identity.
+     * @returns {Object|null} `{archivedAt, archivedReason}` when archived, else `null`.
+     * @private
+     */
+    _archivedIdentityState(agentIdentity) {
+        if (!agentIdentity) {
+            return null;
+        }
+
+        const props = GraphService.getNodeRecord({id: `ARCHIVED_AGENT_IDENTITY:${agentIdentity}`})?.properties;
+        return props?.archivedAt ? {archivedAt: props.archivedAt, archivedReason: props.archivedReason || ''} : null;
+    }
+
+    /**
+     * @summary Returns `memoryProperties` augmented with `archivedAt`/`archivedReason` when the row's
+     * `agentIdentity` carries a durable tombstone marker — applied at every `AGENT_MEMORY` mint site so
+     * a tombstone set during projection lag survives a later (re)projection.
+     * @param {Object} memoryProperties The properties about to be projected onto the graph node.
+     * @returns {Object} The same object, or a tombstoned copy when the identity is archived.
+     * @private
+     */
+    _withArchiveState(memoryProperties) {
+        const archived = this._archivedIdentityState(memoryProperties?.agentIdentity);
+        return archived
+            ? {...memoryProperties, archivedAt: archived.archivedAt, archivedReason: archived.archivedReason}
+            : memoryProperties;
+    }
+
     /**
      * @summary Cross-session, reverse-chronological *recency* recall over `AGENT_MEMORY` graph nodes.
      *
@@ -1045,6 +1270,7 @@ class MemoryService extends Base {
                 WHERE json_extract(memory.data, '$.label') = 'AGENT_MEMORY'
                   AND json_extract(memory.data, '$.properties.agentIdentity') = ?
                   AND json_extract(memory.data, '$.properties.userId')        = ?
+                  AND json_extract(memory.data, '$.properties.archivedAt') IS NULL
                   ${cursorClause}
                 ORDER BY json_extract(memory.data, '$.properties.timestamp') DESC, memory.id DESC
                 LIMIT ?
@@ -1282,7 +1508,9 @@ class MemoryService extends Base {
         }
 
         const existing   = JSON.parse(row.data),
-              properties = {...(existing.properties || {}), miniSummary},
+              // Archive-aware: this addNodes overwrites the node, so a tombstone set after `existing` was
+              // read (the merge-vs-archive race) must be replayed from the durable marker (_withArchiveState).
+              properties = this._withArchiveState({...(existing.properties || {}), miniSummary}),
               nodeData   = {
                   id        : existing.id || id,
                   label     : existing.label || 'AGENT_MEMORY',
@@ -1502,6 +1730,19 @@ class MemoryService extends Base {
             let distances = searchResult.distances?.[0] || [];
             let metadatas = searchResult.metadatas?.[0] || [];
 
+            // Tombstone exclusion: archived rows (metadata.archivedAt set) are dropped from
+            // recall. UNCONDITIONAL — the legacy/trust post-filter below only runs for some policies,
+            // so the exclusion cannot live there. A dropped archived row reads as a genuine no-match.
+            if (metadatas.some(m => m && m.archivedAt)) {
+                const live = [];
+                for (let i = 0; i < metadatas.length; i++) {
+                    if (!(metadatas[i] && metadatas[i].archivedAt)) live.push(i);
+                }
+                ids       = live.map(i => ids[i]);
+                distances = live.map(i => distances[i]);
+                metadatas = live.map(i => metadatas[i]);
+            }
+
             if ((userId && policy === 'legacy') || minTrustTier) {
                 const filteredIndices = [];
                 for (let i = 0; i < metadatas.length; i++) {