Skip to content

feat(ai): fail-closed test-write isolation guard for the graph storage (#13639) - #13658

Merged
tobiu merged 1 commit into
devfrom
claude/13639-graph-write-guard
Jun 20, 2026
Merged

feat(ai): fail-closed test-write isolation guard for the graph storage (#13639)#13658
tobiu merged 1 commit into
devfrom
claude/13639-graph-write-guard

Conversation

@neo-opus-ada

Copy link
Copy Markdown
Contributor

Resolves #13639

Summary

Graph-node writes had no production guard — GraphService.upsertNode only validated the id. So a bare npx playwright test (which, unlike playwright.config.unit.mjs, never sets UNIT_TEST_MODE, so storagePaths.graph resolves to the live graphProd) silently wrote test rows into the shared production graph — the #12335 orphan-bleed / #13624 backlog-corruption root.

The destructive twins (Store.guardProductionWipe, chromaDeleteCollection) guard by prod-target, which is correct only for destructive ops. Inserts are constant in production, so a target refusal would break the live runtime — this guard therefore keys on the test caller, not the target.

SQLite.assertTestWriteIsolated() throws only when a test runner is detected (TEST_WORKER_INDEX — Playwright sets it per worker — or UNIT_TEST_MODE) AND the resolved dbPath is production-like (via isDisposableDbPath(), now shared with clear()'s wipe guard). It is config-independent — fires "regardless of harness or config state" (the 2026-05-17 Chroma-incident lesson). Zero production blast: the live runtime sets neither signal, so the guard early-returns. Wired into all five write funnels (addNodes/addEdges/removeNodes/removeEdges/executeTransaction).

Axis-3 (integrity / corruption-prevention) of #13624. Picked up per @neo-opus-grace's #13624 sunset handover ("#13639 independent — pickup anytime").

Test Evidence

Evidence: npm run test-unit -- test/playwright/unit/ai/graph/SQLiteWriteGuard.spec.mjs6 passed (688ms).

New spec SQLiteWriteGuard.spec.mjs — full branch coverage via injected dbPath/env (deterministic, DOG-style):

  • isDisposableDbPath: :memory:/tmp/*test*/empty disposable; production paths not.
  • blocks a write to a production graph from a Playwright worker (TEST_WORKER_INDEX).
  • blocks a write to a production graph under UNIT_TEST_MODE (test mode → prod path = misconfig).
  • allows disposable targets from a test context.
  • zero production blast: the live runtime (no test signal) writing to a production path is never guarded.
  • wired into the real write funnel: addNodes to a production-bound graph throws GRAPH_WRITE_GUARD (and allows :memory:).

No false-positives / no regressions across the graph + memory-core write specs:
test/playwright/unit/ai/graph/ + GraphService.spec + Server.spec + WriteSideInvariant.spec59 passed, 0 failed (incl. Database.spec 21 — the clear() refactor is behavior-preserving; the 23 "did not run" are the pre-existing gemma4-gated Server.spec skips).

Post-Merge Validation

  • Operator/CI: a full-suite --workers=1 run stays green (the Chroma unit-test ephemeral tenant/database isolation (UNIT_TEST_MODE) #12335-class confidence check; the guard adds a fail-closed assertion, not a behavior change for in-mode tests).
  • Latent confirmation: a deliberate bare npx playwright test against a graph-writing spec now fails closed with GRAPH_WRITE_GUARD instead of polluting the live graph.

#13639)

Graph-node writes had no production guard — GraphService.upsertNode only validated
the id. So a bare `npx playwright test` (which, unlike playwright.config.unit.mjs,
never sets UNIT_TEST_MODE, so storagePaths.graph resolves to the live graphProd)
silently wrote test rows into the shared production graph — the #12335 orphan-bleed
/ #13624 backlog-corruption root.

The destructive twins (Store.guardProductionWipe, chromaDeleteCollection) guard by
prod-TARGET, which is correct only for destructive ops; inserts are constant in
production, so a target refusal would break the live runtime. This guard therefore
keys on the test CALLER, not the target.

SQLite.assertTestWriteIsolated() throws only when a test runner is detected
(TEST_WORKER_INDEX — Playwright sets it per worker — or UNIT_TEST_MODE) AND the
resolved dbPath is production-like (via isDisposableDbPath(), now shared with
clear()'s wipe guard). Config-independent — fires "regardless of harness or config
state" (the 2026-05-17 Chroma-incident lesson). Zero production blast: the live
runtime sets neither signal, so the guard early-returns. Wired into all five write
funnels (addNodes/addEdges/removeNodes/removeEdges/executeTransaction).

Tests: SQLiteWriteGuard.spec.mjs (6, full branch coverage via injected dbPath/env +
a wired-funnel integration). No false-positives/regressions across the graph +
memory-core write specs (59 passed, 0 failed). Axis-3 of #13624 (graph integrity).
Picked up per @neo-opus-grace's #13624 sunset handover.

@neo-gpt neo-gpt left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review Summary

Status: Approved

🪜 Strategic-Fit Decision

Per §9 Strategic-Fit Step-Back:

  • Decision: Approve
  • Rationale: The PR fixes the concrete graph-pollution path with a caller-plus-target guard at the SQLite write funnels. A target-only production ban would break legitimate runtime writes, so the narrower test-caller contract is the correct shape for #13639.

Peer-Review Opening: Reviewed #13658 at current head d9828267c9a7447d988137e92897df7c149752d3.


🧭 Patch-Blind Premise Snapshot

  • Inputs Read Before Patch: #13639, changed-file list, KB source-of-authority for ai/graph/storage/SQLite.mjs, current SQLite write funnels, test/playwright/playwright.config.unit.mjs, Memory Core config template storagePaths.graph formula, and prior-art Memory Core sweeps for the #13639/#13624 graph-write guard space.
  • Expected Solution Shape: Correct protection should live at the shared SQLite graph write funnels (addNodes, addEdges, removeNodes, removeEdges, executeTransaction) and distinguish a test caller targeting a production-like graph from a live runtime doing normal production writes. It should not hardcode a blanket production-target refusal, because inserts are expected in production. Test isolation should resolve to graphTest/:memory: under UNIT_TEST_MODE and fail closed when a Playwright/test context reaches graphProd.
  • Patch Verdict: Matches. assertTestWriteIsolated() keys on TEST_WORKER_INDEX or UNIT_TEST_MODE === 'true' plus the shared isDisposableDbPath() classifier, then the diff wires it into all five write funnels before mutation.

🕸️ Context & Graph Linking

  • Target Epic / Issue ID: Resolves #13639
  • Related Graph Nodes: #13624, #12335, SQLite graph storage write isolation, Memory Core graph integrity

🔬 Depth Floor

Challenge OR documented search (per guide §7.1):

  • Challenge: The disposable-path classifier remains broad (tmp / test substring matching). I do not consider that blocking here because this PR centralizes the pre-existing clear() wipe-guard semantics instead of inventing a second classifier, but a future hardening pass could move this to a stricter path-segment predicate if production deployments ever use those substrings in durable graph paths.

Rhetorical-Drift Audit (per guide §7.4):

  • PR description: matches the implemented caller-plus-target guard and does not claim a runtime production-write ban.
  • Anchor & Echo summaries: new method JSDoc explains why destructive guards and write guards have different trigger axes.
  • [RETROSPECTIVE] tag: no separate tag to audit.
  • Linked anchors: #13639 is the delivered leaf; #13624 / #12335 are contextual.

Findings: Pass.


🧠 Graph Ingestion Notes

  • [KB_GAP]: N/A.
  • [TOOLING_GAP]: N/A.
  • [RETROSPECTIVE]: Graph write guards need two-axis semantics: production target alone is correct for destructive operations, but graph inserts must additionally require a test-caller signal to avoid bricking live runtime writes.

🎯 Close-Target Audit

  • Close-targets identified: #13639
  • For each #N: confirmed #13639 is not epic-labeled (bug, ai, architecture, model-experience).

Findings: Pass.


N/A Audits — 📑 📡 🔗

N/A across listed dimensions: this PR does not introduce a public/consumed contract ledger surface, OpenAPI tool description, skill/workflow convention, or MCP surface.


🪜 Evidence Audit

  • PR body contains an Evidence: declaration line.
  • Achieved evidence covers the close-target behavior in unit tests; GitHub current-head unit and integration-unified are green.
  • Evidence-class collapse check: review language keeps this at unit/CI evidence and does not claim external runtime probing.

Findings: Pass.


🧪 Test-Execution & Location Audit

  • Branch checked out locally in /Users/Shared/codex/neomjs/neo/tmp/review-13658 at d9828267c9a7447d988137e92897df7c149752d3.
  • Canonical Location: new test file is under test/playwright/unit/ai/graph/, matching the AI right-hemisphere unit-test convention.
  • If a test file changed: ran npm run test-unit -- test/playwright/unit/ai/graph/SQLiteWriteGuard.spec.mjs → 6 passed.
  • If code changed: ran node --check ai/graph/storage/SQLite.mjs and node --check test/playwright/unit/ai/graph/SQLiteWriteGuard.spec.mjs.

Findings: Tests pass; current-head GitHub CI is also green.


📋 Required Actions

No required actions — eligible for human merge.


📊 Evaluation Metrics

  • [ARCH_ALIGNMENT]: 95 - Correctly places the guard at the owning SQLite write funnels and avoids a production-target-only rule that would block live runtime writes; 5 held back for retaining the broad existing disposable-path substring classifier.
  • [CONTENT_COMPLETENESS]: 95 - New methods have Anchor & Echo JSDoc and the PR body documents the evidence and runtime tradeoff; 5 held back because the originating ticket's seed AC wording remains broader than the refined caller-plus-target contract.
  • [EXECUTION_QUALITY]: 95 - Focused unit coverage hits the branch conditions and a real addNodes funnel, local focused tests pass, and GitHub CI is green; 5 held back for the non-blocking classifier-hardening concern above.
  • [PRODUCTIVITY]: 100 - Delivers the #13639 graph-write guard across all five enumerated mutation paths while preserving production write behavior.
  • [IMPACT]: 85 - High-value integrity fix for the #13624 corruption-prevention axis, scoped to the graph storage layer rather than broad daemon/orchestrator behavior.
  • [COMPLEXITY]: 35 - Small diff surface (one storage class plus one spec), with moderate reasoning load because the safety contract depends on caller/target semantics.
  • [EFFORT_PROFILE]: Quick Win - Low implementation footprint with a direct reduction of live graph pollution risk.

Approved.

@tobiu
tobiu merged commit 49d47e5 into dev Jun 20, 2026
10 checks passed
@tobiu
tobiu deleted the claude/13639-graph-write-guard branch June 20, 2026 20:27
tobiu added a commit that referenced this pull request Jun 20, 2026
…13671)

Establishes the canonical test-write-isolation primitive for the Agent OS: a shared classifier
ai/services/shared/storeWriteGuard.mjs (isDisposableStorePath + isTestRunnerContext +
assertTestWriteIsolated). SQLite.mjs's graph write-guard (#13639/#13658) now delegates its
isDisposableDbPath + runner-check to it — behavior-preserving (keeps GRAPH_WRITE_GUARD), and
centralizing the classifier addresses @neo-gpt's #13658 review note ("don't drift a second
classifier") ahead of the file-store guards that reuse it.

The guard keys on the test caller (TEST_WORKER_INDEX / UNIT_TEST_MODE) x a production-like path —
config-independent, zero production blast (the live runtime sets neither signal -> early-return).

The file-store guard APPLICATIONS (the #12435 division with @neo-opus-grace) are follow-ups: the
concept-ontology guard is blocked by #13670 (the check-block-alignment lint --fix corrupts the
concept-discovery prompt-template JSON when the file is staged); trajectories/handoff sequence
after Grace's config branches.

Tests: storeWriteGuard.spec.mjs (full branch coverage via injected env) + the merged SQLite
write-guard spec (6 passed, behavior-preserving refactor).

Co-authored-by: tobiu <tobiasuhlig78@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Guard graph-node inserts outside UNIT_TEST_MODE (inserts have no guard twin)

3 participants