diff --git a/docs/design/phase-1a-composed-boot-parity-test.md b/docs/design/phase-1a-composed-boot-parity-test.md new file mode 100644 index 0000000..2c01e66 --- /dev/null +++ b/docs/design/phase-1a-composed-boot-parity-test.md @@ -0,0 +1,121 @@ +# Phase 1A — Minimum Missing Integration Test: Composed Boot Parity + +**Date:** 2026-04-16 +**Status:** Gap identified (codex2 review of `0da468e`); test not yet written. +Test-plan artifact only — this memo does not introduce code. +**Scope:** `Atomicmemory-core`, branch `feat/phase-1a-runtime-container`. + +--- + +## What is missing + +One live-HTTP integration test that boots + +``` +createApp(createCoreRuntime({ pool })) +``` + +against an ephemeral port and proves the composed app's config-facing +HTTP behavior matches the runtime backed by the module-level config +singleton end-to-end. Today no such test exists on this branch. + +## Why `runtime-container.test.ts` alone is insufficient + +`src/app/__tests__/runtime-container.test.ts` (126 lines) is a +**composition-shape test**, not a parity test. Specifically: + +- **Stubbed pool, no DB round-trip.** `stubPool()` at line 17 returns a + minimal `{ query: vi.fn(...) }` object. No SQL actually runs; no + repo→DB seam is exercised. +- **`createApp` test is type-shape only.** Line 119–126 asserts + `typeof app.use === 'function'` and `typeof app.listen === 'function'` + on the returned app. No request is issued, no route is hit, no + response is observed. A bug that binds `/memories` to the wrong + service — or silently drops the route mount — would pass this test. +- **No observed config-driven behavior.** Lines 42–80 verify that + `runtime.config === config` and that `repos.entities` / + `repos.lessons` track `entityGraphEnabled` / `lessonsEnabled` at the + container level. None of that proves the flags surface correctly + into the HTTP layer once routes are mounted. +- **No parity assertion.** There is no comparison of composed-boot + behavior against a reference. If someone later threads config + through routes/services and introduces a subtle divergence between + the composed path and the singleton-backed path, the current suite + would not detect it. + +The existing `src/__tests__/route-validation.test.ts` is a +**route-behavior test**, not a composition-parity test: it constructs +its own `express()` app (line 41), its own `new MemoryService(repo, +claimRepo)` (line 51), and mounts `createMemoryRouter` directly. It +deliberately bypasses `createCoreRuntime` and `createApp`. Useful for +route logic; silent on whether the Phase 1A composition seam actually +produces an equivalent app. + +## The minimum missing test — shape (not implementation) + +A single integration test file under `src/app/__tests__/` that: + +1. Boots a live Express listener via + `createApp(createCoreRuntime({ pool })).listen(0)` where `pool` is + the shared test pool (same pattern as + `route-validation.test.ts:54-61`). +2. Applies `schema.sql` to the test DB once in `beforeAll` (match the + existing integration-test pattern; do not reinvent setup). +3. Issues a small number of HTTP requests against the ephemeral port + that collectively exercise at least one config-sensitive path. + Minimum request set: one write (ingest or verbatim store) and one + read that surfaces the write. This proves routes → services → + repos → pool all connected correctly through the composition seam. +4. Asserts responses match what a reference request against a + hand-wired runtime (the `route-validation.test.ts` 2-arg + `MemoryService` shape, or a second composed runtime built the same + way) produces. Parity is the point; the specific endpoint matters + less than the observation that the composed path and the + known-working path agree. +5. Closes the listener and releases any ephemeral resources in + `afterAll`. + +The test does not need to be exhaustive. It needs to prove, with at +least one live round-trip, that the composition seam does not drop +fidelity between the runtime container and the HTTP surface. + +## Narrow acceptance criteria + +The test is considered sufficient iff: + +1. It boots `createApp(createCoreRuntime({ pool }))` — not a hand-wired + Express app, not a manually-constructed `MemoryService`. +2. It binds `app.listen(0)` (ephemeral port) inside the test lifecycle, + not a hardcoded port. +3. It issues at least one HTTP request whose response depends on code + paths reached through the runtime container (routes, services, + repos). A `GET /health` alone does not satisfy this — `/health` + does not traverse the runtime graph. +4. It asserts equivalence against a reference behavior, either by + (a) constructing a second runtime the same way and comparing + responses, or (b) asserting concrete expected values that a + singleton-backed server would also produce under identical config. +5. It runs green under the existing `npm test` harness with no new + dependencies and no new env requirements beyond `.env.test`. +6. It lives under `src/app/__tests__/` — adjacent to the composition + code it covers, not under `src/__tests__/`, and not under + `src/services/__tests__/`. + +## What this test deliberately does not cover + +- **Route-level input validation** — already covered by + `route-validation.test.ts` (UUID guards, filter params). +- **Per-endpoint business logic** — covered by service-level and + repo-level integration tests. +- **Config-threading correctness across 53 `import { config }` sites** + — that is Phase 1B scope and requires its own test strategy. +- **Startup-check behavior** — already covered in + `runtime-container.test.ts:83-116`. +- **Process lifecycle** (SIGTERM, SIGINT, uncaught handlers) — covered + in Phase 1C when `bootstrap()` is extracted. + +## One-line framing + +Prove that `createApp(createCoreRuntime(...))` produces an HTTP server +whose observable behavior matches the singleton-backed reference on +at least one config-sensitive round-trip. No more; no less. diff --git a/docs/design/phase-1a-singleton-hazards.md b/docs/design/phase-1a-singleton-hazards.md new file mode 100644 index 0000000..7eb348f --- /dev/null +++ b/docs/design/phase-1a-singleton-hazards.md @@ -0,0 +1,192 @@ +# Phase 1A — Singleton & Global Hazard Audit + Follow-on Sequence + +**Date:** 2026-04-16 +**Status:** Phase 1A landed (`ff57540`, `0da468e`); this memo captures the +source-grounded audit that informed the refactor and the minimum safe +sequence for Phase 1B and beyond. +**Scope:** `Atomicmemory-core` only. + +--- + +## Why Phase 1A was needed + +Two independent singleton/global patterns were entangling startup, +tests, and research harnesses, making it impossible to boot two +runtimes in one process (e.g. for `/compare` style evaluations) or to +unit-test the search pipeline without module-load side effects. + +### Hazard 1 — `src/services/search-pipeline.ts` module-level entity repo + +A mutable module global and a dedicated setter: + +- `let entityRepo: EntityRepository | null = null;` at module scope +- `export function setEntityRepository(repo)` as the only way to mutate +- ~11 read sites inside the pipeline dereferencing it directly + (`runSearchPipelineWithTrace`, `generateLinks`, query-expansion, + query-augmentation, link-expansion branches) + +Production flow: `server.ts` called `setEntityRepository(entityRepo)` +at module load, gated by `config.entityGraphEnabled`. Tests that +imported anything pulling in `server.ts` inherited that global state. +Two tests mocked the whole module via `createSearchPipelineMockFactory`, +locking in three exported names (`runSearchPipelineWithTrace`, +`generateLinks`, `setEntityRepository`) — any rename silently broke +the mock. + +### Hazard 2 — `src/server.ts` as a singleton composition root + +- `repo`, `claimRepo`, `trustRepo`, `linkRepo`, `entityRepo`, + `lessonRepo`, `service`, and `app` all declared at module scope +- `checkEmbeddingDimensions().then(() => app.listen(config.port, …))` + fired at module evaluation — **importing `./server.ts` bound a port** +- Process-global handlers (`uncaughtException`, `unhandledRejection`, + `SIGTERM`, `SIGINT`) registered at import time +- Exports (`app, service, repo, claimRepo, trustRepo, linkRepo`) had + zero in-repo readers; the shape advertised a reusable API that no + one could actually use without triggering the side effects + +### Hazard 3 — parallel DI paths for the entity repo + +`MemoryService` accepted `entities` as an optional constructor arg +(`memory-service.ts:33-48`) and threaded it through `deps.entities` +for storage/dedup/audn paths. The search pipeline, meanwhile, read +its entity repo from the module global. Two independent sources of +truth for "is the entity graph wired?" — one constructor-injected, +one module-global. + +--- + +## What Phase 1A resolved + +Commits on `feat/phase-1a-runtime-container`: + +| Commit | Scope | +|---|---| +| `ff57540` refactor(core): introduce runtime container composition root | Extracted `runtime-container.ts`, `create-app.ts`, `startup-checks.ts`; server.ts now just bootstraps | +| `0da468e` refactor(app): remove misleading config override from createCoreRuntime | Removed the nascent `config` override parameter that would have been silently ignored by the 53 downstream `import { config }` sites | + +### Concrete changes + +- **`src/app/runtime-container.ts`** (new, 107 lines) — `createCoreRuntime({ pool })` returns an explicit `{ config, pool, repos, services }` graph. No hidden singletons. +- **`src/app/create-app.ts`** (new, 39 lines) — `createApp(runtime)` is the only way to build the Express app. CORS, body parsing, routes, health are bound to the passed-in runtime. +- **`src/app/startup-checks.ts`** (new, 70 lines) — embedding-dimension guard extracted behind a plain function, no process exits from inside the check. +- **`src/app/__tests__/runtime-container.test.ts`** (new, 126 lines) — coverage of the composition root. +- **`src/server.ts`** shrunk from 101 → 70 lines — now just process lifecycle (boot, listen, shutdown) over the composed runtime. +- **`src/services/search-pipeline.ts`** — module-level `entityRepo` and `setEntityRepository` removed; `runSearchPipelineWithTrace` and `generateLinks` now accept `entityRepo: EntityRepository | null` as an explicit function parameter. +- **`src/services/__tests__/test-fixtures.ts`** — `createSearchPipelineMockFactory` no longer needs to stub `setEntityRepository`. +- **`src/services/__tests__/current-state-retrieval-regression.test.ts`** — the `setEntityRepository(null)` reset in `beforeEach` is gone (not needed without the global). + +### What Phase 1A deliberately did **not** do + +- **Config override in the runtime.** An early draft of `createCoreRuntime` accepted a `config` override. Removed in `0da468e` because 53 files still `import { config }` directly from `../config.js`; an override would have influenced only repo construction (`entityGraphEnabled`, `lessonsEnabled`) and been silently ignored everywhere else. Reintroducing this override is a Phase 1B deliverable, not a Phase 1A one. +- **Process-global handlers.** `uncaughtException`/`unhandledRejection`/`SIGTERM`/`SIGINT` registrations still live at import time in `server.ts`. Safe while `server.ts` is purely an entrypoint — becomes unsafe if anyone imports `server.ts` from library code. +- **`server.ts` re-exports.** `{ app, service, repo, claimRepo, trustRepo, linkRepo, runtime }` are still exported. Kept for back-compat with any external consumer; zero in-repo readers. + +--- + +## Hazards that remain (post Phase 1A) + +### Remaining 1 — config as a 53-reader module singleton + +- 53 files `import { config }` from `../config.js` directly. +- `createCoreRuntime` reflects this honestly: `runtime.config` is the + same reference as the module singleton. You cannot construct two + runtimes with different configs in one process. +- Any attempt to override per-runtime config without first threading + config through routes/services/pipeline is a silent no-op for + every flag not named `entityGraphEnabled` or `lessonsEnabled`. + +### Remaining 2 — `server.ts` side-effects-on-import + +- `bootstrap()` fires at module load. +- Four `process.on(...)` handlers register at module load. +- Importing `./server.ts` still has a nontrivial runtime contract. + +### Remaining 3 — zero-reader back-compat export surface on `server.ts` + +- `export { app, service, repo, claimRepo, trustRepo, linkRepo, runtime };` +- No in-repo importers; external consumers (if any) would inherit the + import-time side effects above. + +--- + +## Files and tests to watch during Phase 1B+ + +### Production code (expect churn) + +- `src/config.ts` — the singleton that will grow a per-runtime construction path +- `src/services/search-pipeline.ts` — currently reads `config` inline (e.g. `config.entityGraphEnabled`, `config.queryExpansionEnabled`, `config.mmrEnabled`); candidate for threading config through rather than module-level reads +- `src/services/memory-search.ts` — same pattern +- `src/services/memory-ingest.ts`, `memory-audn.ts`, `memory-storage.ts` — 53-site config audit lands here +- `src/routes/memories.ts`, `src/routes/agents.ts` — routes also read config directly in places +- `src/server.ts` — when `bootstrap()` becomes a function callable by tests/harnesses, the import-time side effects go with it + +### Tests (fragile surfaces if Phase 1A exports shift) + +- `src/app/__tests__/runtime-container.test.ts` — locks the shape of `createCoreRuntime` and the `CoreRuntime` interface +- `src/__tests__/route-validation.test.ts` — builds its own `express()` app, calls `new MemoryService(repo, claimRepo)` (2-arg ctor); if `MemoryService` ctor changes, update here +- `src/__tests__/smoke.test.ts` — consumes `createServiceTestContext` (2-arg `MemoryService`) +- `src/db/__tests__/test-fixtures.ts` — `createServiceTestContext` locks the 2-arg `MemoryService` ctor shape +- `src/services/__tests__/test-fixtures.ts` — `createSearchPipelineMockFactory` locks two exported names now (`runSearchPipelineWithTrace`, `generateLinks`); any rename is a silent mock break +- `src/services/__tests__/stale-composite-retrieval.test.ts`, `current-state-composite-packaging.test.ts` — both use `createSearchPipelineMockFactory` +- `src/services/__tests__/current-state-retrieval-regression.test.ts` — no longer imports `setEntityRepository`; should now pass an explicit entity repo (or leave as null) when it invokes the pipeline + +--- + +## Minimum safe follow-on sequence + +Each step is independently testable and reversible. Don't skip ahead. + +### Phase 1B — thread config through the service layer + +**Goal:** `runtime.config` becomes the one config source; reintroduce a genuine `config` override on `createCoreRuntime`. + +1. **Audit the 53 `import { config }` sites.** Classify each as (a) flag read at request time, (b) flag read at construction time, or (c) constant-use-as-pseudo-env. The counts inform Phase 1B's scope. +2. **Thread config through `MemoryService` construction.** Add a config argument to `MemoryService` and `MemoryServiceDeps`; remove inline `import { config }` from the service-level modules called from the facade. +3. **Thread config through the search pipeline.** `runSearchPipelineWithTrace` and `generateLinks` already take `entityRepo` explicitly; extend the same treatment to the config fields they read (`queryExpansionEnabled`, `mmrEnabled`, `rerankSkipTopSimilarity`, `rerankSkipMinGap`, etc.). +4. **Thread config through routes.** Route factories (`createMemoryRouter`, `createAgentRouter`) already accept services; add a config parameter where routes read `config` inline today (spot-check first — some routes may not need it). +5. **Reintroduce `config` override on `createCoreRuntime`.** Once steps 2–4 are landed, a `CoreRuntimeDeps.config` override is honest again: every flag passed in will actually be consulted by downstream code. +6. **Test:** extend `runtime-container.test.ts` to assert two concurrently-constructed runtimes with different configs behave independently on at least one flag. + +**Exit criterion:** `grep -r "import { config } from" src | wc -l` drops meaningfully; the remaining sites are deliberate (e.g. `config.ts` itself, bootstrap files, truly environment-scoped reads). + +### Phase 1C — extract `bootstrap()` to `src/app/bootstrap.ts` + +**Goal:** `server.ts` becomes a thin `bin/`-style entrypoint; `bootstrap(runtime)` is callable from tests and harnesses. + +1. Move `bootstrap()` body, the `app.listen` call, and the four `process.on` handlers into `src/app/bootstrap.ts`. Export `startRuntime(runtime): Promise<() => Promise>` returning a shutdown function. +2. `server.ts` becomes: `const runtime = createCoreRuntime({ pool }); const app = createApp(runtime); startRuntime({ runtime, app });` — no exports. +3. Delete the back-compat re-exports from `server.ts`. Verified above: zero in-repo readers. +4. **Test:** unit test `startRuntime` can bind to an ephemeral port (`listen(0)`) and be cleanly shut down by its returned function. + +**Exit criterion:** `server.ts` has no exports and no side effects beyond the bootstrap call; importing it from a test does not bind a port or register process handlers. + +### Phase 1D — `createCoreRuntime` accepts override hooks for testing + +**Goal:** tests can substitute individual repositories or services without reconstructing the whole graph. + +1. Add optional `repos?: Partial` and `services?: Partial` to `CoreRuntimeDeps`. +2. Merge overrides after default construction; add a runtime-container test that verifies an injected mock `MemoryService` is what `createApp` binds to `/memories`. +3. Migrate `route-validation.test.ts` and `smoke.test.ts` to use `createCoreRuntime` instead of constructing `MemoryService` directly. + +**Exit criterion:** no test file reaches past `createCoreRuntime` to import repositories or the `MemoryService` class directly. + +### Phase 1E — lifecycle for `pool` + +**Goal:** the process-global `pool` singleton (imported only by `server.ts` now) becomes an explicit parameter that tests can replace with an isolated pool. + +1. Audit importers of `src/db/pool.ts`. If `server.ts` is the only one, Phase 1E is a 2-line change to thread `pool` via a constructor arg to any remaining importer. +2. If tests still import `pool` directly for setup (`route-validation.test.ts`, `test-fixtures.ts`), add a `createTestPool()` helper so test and production paths don't share the same pg connection pool by accident. + +**Exit criterion:** `pool` is never imported outside `server.ts` (production) and the test-pool helper (tests). + +--- + +## One-line summary + +Phase 1A eliminated the search-pipeline module global and the +server.ts composition root; what remains is config-as-singleton +(53 sites), bootstrap-on-import in `server.ts`, and the zero-reader +re-export surface. Phase 1B threads config, Phase 1C extracts +bootstrap, Phase 1D enables test overrides, Phase 1E isolates pool. +Each step is independently landable. diff --git a/src/__tests__/deployment-config.test.ts b/src/__tests__/deployment-config.test.ts index 26783f4..503d66c 100644 --- a/src/__tests__/deployment-config.test.ts +++ b/src/__tests__/deployment-config.test.ts @@ -42,6 +42,19 @@ function extractComposeEnvVar(composeContent: string, varName: string): string | return match ? match[1].trim().replace(/^["']|["']$/g, '') : null; } +/** + * Build a regex that matches a docker-compose `ports:` list entry binding + * an external host port to the given internal container port. Accepts both + * a literal external port (`"3050:3050"`) and a shell-variable substitution + * (`"${APP_PORT:-3050}:3050"`). The substitution form is the side-by-side-CI + * shape introduced in PR #6. + */ +function composePortBindingRegex(internalPort: number): RegExp { + return new RegExp( + `ports:\\s*\\n\\s*-\\s*["']?(?:\\d+|\\$\\{[A-Z_]+:-\\d+\\}):${internalPort}`, + ); +} + describe('deployment configuration', () => { describe('docker-compose.yml', () => { it('app service uses host.docker.internal for OLLAMA_BASE_URL, not localhost', () => { @@ -102,7 +115,7 @@ describe('deployment configuration', () => { it('app port is exposed', () => { const compose = readComposeRaw(); - expect(compose).toMatch(/ports:\s*\n\s*-\s*["']?(\$\{APP_PORT:-\d+\}|\d+):3050/); + expect(compose).toMatch(composePortBindingRegex(3050)); }); }); diff --git a/src/app/__tests__/composed-boot-parity.test.ts b/src/app/__tests__/composed-boot-parity.test.ts new file mode 100644 index 0000000..90e3e9f --- /dev/null +++ b/src/app/__tests__/composed-boot-parity.test.ts @@ -0,0 +1,162 @@ +/** + * Phase 1A composed-boot parity test. + * + * Boots `createApp(createCoreRuntime({ pool }))` against an ephemeral + * port and proves that the composed app's HTTP behavior matches a + * hand-wired singleton-backed reference. This closes the gap left by + * `runtime-container.test.ts` (composition-shape only, no live HTTP) + * and `route-validation.test.ts` (route logic, deliberately bypasses + * the composition seam). + * + * The test is narrow by design — Phase 1A is about proving the + * composition seam doesn't drop fidelity, not about covering every + * config-threading scenario. That's Phase 1B scope. + * + * See docs/design/phase-1a-composed-boot-parity-test.md for the + * acceptance criteria this test satisfies. + */ + +import { describe, it, expect, beforeAll, afterAll } from 'vitest'; +import express from 'express'; +import { readFileSync } from 'node:fs'; +import { resolve, dirname } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { pool } from '../../db/pool.js'; +import { config } from '../../config.js'; +import { MemoryRepository } from '../../db/memory-repository.js'; +import { ClaimRepository } from '../../db/claim-repository.js'; +import { MemoryService } from '../../services/memory-service.js'; +import { createMemoryRouter } from '../../routes/memories.js'; +import { createCoreRuntime } from '../runtime-container.js'; +import { createApp } from '../create-app.js'; + +const __dirname = dirname(fileURLToPath(import.meta.url)); +const TEST_USER = 'composed-boot-parity-user'; + +interface BootedApp { + baseUrl: string; + close: () => Promise; +} + +/** + * Bind an Express app to an ephemeral port and return its base URL plus + * a close handle. Mirrors the listen pattern in route-validation.test.ts + * but isolated here so each test app shuts down independently. + */ +async function bindEphemeral(app: ReturnType): Promise { + const server = app.listen(0); + await new Promise((resolve) => server.once('listening', () => resolve())); + const addr = server.address(); + const port = typeof addr === 'object' && addr ? addr.port : 0; + return { + baseUrl: `http://localhost:${port}`, + close: () => new Promise((resolve) => server.close(() => resolve())), + }; +} + +/** + * Build the singleton-backed reference app — what server.ts looked like + * before Phase 1A. Used as the parity baseline. + */ +function buildReferenceApp(): ReturnType { + const app = express(); + app.use(express.json()); + const repo = new MemoryRepository(pool); + const claimRepo = new ClaimRepository(pool); + const service = new MemoryService(repo, claimRepo); + app.use('/memories', createMemoryRouter(service)); + return app; +} + +describe('composed boot parity', () => { + let composed: BootedApp; + let reference: BootedApp; + + beforeAll(async () => { + const raw = readFileSync(resolve(__dirname, '../../db/schema.sql'), 'utf-8'); + const sql = raw.replace(/\{\{EMBEDDING_DIMENSIONS\}\}/g, String(config.embeddingDimensions)); + await pool.query(sql); + + composed = await bindEphemeral(createApp(createCoreRuntime({ pool }))); + reference = await bindEphemeral(buildReferenceApp()); + }); + + afterAll(async () => { + await composed.close(); + await reference.close(); + await pool.end(); + }); + + it('GET /memories/health returns the same config payload from both apps', async () => { + const composedRes = await fetch(`${composed.baseUrl}/memories/health`); + const referenceRes = await fetch(`${reference.baseUrl}/memories/health`); + + expect(composedRes.status).toBe(200); + expect(referenceRes.status).toBe(200); + + const composedBody = await composedRes.json(); + const referenceBody = await referenceRes.json(); + + expect(composedBody).toEqual(referenceBody); + expect(composedBody.status).toBe('ok'); + expect(composedBody.config.embedding_provider).toBe(config.embeddingProvider); + expect(composedBody.config.entity_graph_enabled).toBe(config.entityGraphEnabled); + }); + + it('GET /memories/stats traverses routes → services → repos → pool through the composition seam', async () => { + const composedRes = await fetch(`${composed.baseUrl}/memories/stats?user_id=${TEST_USER}`); + const referenceRes = await fetch(`${reference.baseUrl}/memories/stats?user_id=${TEST_USER}`); + + expect(composedRes.status).toBe(200); + expect(referenceRes.status).toBe(200); + + const composedBody = await composedRes.json(); + const referenceBody = await referenceRes.json(); + + // Both queries hit the same DB with no preceding writes — counts + // must match. If the composed seam silently dropped a layer, this + // would either 500 or return a different shape. + expect(composedBody).toEqual(referenceBody); + expect(typeof composedBody.count).toBe('number'); + expect(composedBody.sourceDistribution).toBeDefined(); + }); + + // Runs last so any failure of the finally{} cleanup cannot bleed into + // the GET parity tests above. Cleanup mutates the config singleton + // directly rather than via a follow-up PUT so it does not depend on + // either server still being healthy at teardown. + it('PUT /memories/config mutation is observable via GET /memories/health on both composed and reference apps', async () => { + const originalMaxResults = config.maxSearchResults; + const sentinel = originalMaxResults + 17; + + try { + const putRes = await fetch(`${composed.baseUrl}/memories/config`, { + method: 'PUT', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ max_search_results: sentinel }), + }); + expect(putRes.status).toBe(200); + const putBody = await putRes.json(); + expect(putBody.applied).toContain('maxSearchResults'); + expect(putBody.config.max_search_results).toBe(sentinel); + + // The mutation goes through the composed app's PUT route into the + // module-level config singleton. The reference app reads the same + // singleton, so its /health must reflect the change too — that + // parity is the proof the composed write seam is honestly wired + // to the same config the rest of the runtime reads from. + const composedHealthRes = await fetch(`${composed.baseUrl}/memories/health`); + const referenceHealthRes = await fetch(`${reference.baseUrl}/memories/health`); + const composedHealth = await composedHealthRes.json(); + const referenceHealth = await referenceHealthRes.json(); + + expect(composedHealth.config.max_search_results).toBe(sentinel); + expect(referenceHealth.config.max_search_results).toBe(sentinel); + expect(composedHealth).toEqual(referenceHealth); + } finally { + // Restore the singleton directly so a server hiccup cannot leak + // the sentinel into subsequent test files in the same worker. + (config as { maxSearchResults: number }).maxSearchResults = originalMaxResults; + } + }); +}); diff --git a/src/app/__tests__/runtime-container.test.ts b/src/app/__tests__/runtime-container.test.ts new file mode 100644 index 0000000..ae786a7 --- /dev/null +++ b/src/app/__tests__/runtime-container.test.ts @@ -0,0 +1,126 @@ +/** + * Phase 1A composition tests. + * + * Verifies the runtime container boots cleanly with explicit deps, and + * that startup checks return a structured result instead of exiting the + * process. These tests don't depend on a live database — they exercise + * the composition seam itself. + */ + +import { describe, it, expect, vi } from 'vitest'; +import pg from 'pg'; +import { createCoreRuntime } from '../runtime-container.js'; +import { checkEmbeddingDimensions } from '../startup-checks.js'; +import { createApp } from '../create-app.js'; +import { config } from '../../config.js'; + +function stubPool(rows: Array<{ typmod: number }> = []): pg.Pool { + return { query: vi.fn(async () => ({ rows })) } as unknown as pg.Pool; +} + +/** + * Temporarily mutate a config flag, run a function, and restore. + * Used to exercise repo-construction branches without accepting a config + * override on createCoreRuntime (which would be dishonest — most code + * reads config from the module singleton directly). + */ +function withConfigFlag( + key: K, + value: typeof config[K], + run: () => void, +): void { + const previous = config[key]; + (config as unknown as Record)[key] = value; + try { + run(); + } finally { + (config as unknown as Record)[key] = previous; + } +} + +describe('createCoreRuntime', () => { + it('composes a runtime with explicit pool dep', () => { + const pool = stubPool(); + const runtime = createCoreRuntime({ pool }); + expect(runtime.pool).toBe(pool); + expect(runtime.config).toBe(config); + expect(runtime.repos.memory).toBeDefined(); + expect(runtime.repos.claims).toBeDefined(); + expect(runtime.repos.trust).toBeDefined(); + expect(runtime.repos.links).toBeDefined(); + expect(runtime.services.memory).toBeDefined(); + }); + + it('runtime.config references the module-level config singleton', () => { + // Phase 1A.5 truthfulness: the container does not accept a config + // override because routes/services still read the singleton. + const pool = stubPool(); + const runtime = createCoreRuntime({ pool }); + expect(runtime.config).toBe(config); + }); + + it('entity repo tracks config.entityGraphEnabled', () => { + const pool = stubPool(); + withConfigFlag('entityGraphEnabled', false, () => { + expect(createCoreRuntime({ pool }).repos.entities).toBeNull(); + }); + withConfigFlag('entityGraphEnabled', true, () => { + expect(createCoreRuntime({ pool }).repos.entities).not.toBeNull(); + }); + }); + + it('lesson repo tracks config.lessonsEnabled', () => { + const pool = stubPool(); + withConfigFlag('lessonsEnabled', false, () => { + expect(createCoreRuntime({ pool }).repos.lessons).toBeNull(); + }); + withConfigFlag('lessonsEnabled', true, () => { + expect(createCoreRuntime({ pool }).repos.lessons).not.toBeNull(); + }); + }); +}); + +describe('checkEmbeddingDimensions', () => { + it('returns ok=false when memories.embedding column is missing', async () => { + const pool = stubPool([]); + const result = await checkEmbeddingDimensions(pool, config); + expect(result.ok).toBe(false); + expect(result.message).toContain('run npm run migrate'); + }); + + it('returns ok=false when DB dims differ from config', async () => { + const pool = stubPool([{ typmod: 1024 }]); + const cfg = { ...config, embeddingDimensions: 1536 }; + const result = await checkEmbeddingDimensions(pool, cfg); + expect(result.ok).toBe(false); + expect(result.dbDims).toBe(1024); + expect(result.configDims).toBe(1536); + expect(result.message).toContain('1024 dimensions'); + expect(result.message).toContain('EMBEDDING_DIMENSIONS=1536'); + }); + + it('returns ok=true when DB dims match config', async () => { + const pool = stubPool([{ typmod: 1024 }]); + const cfg = { ...config, embeddingDimensions: 1024 }; + const result = await checkEmbeddingDimensions(pool, cfg); + expect(result.ok).toBe(true); + expect(result.dbDims).toBe(1024); + }); + + it('returns ok=true when DB typmod is unset (0 or negative)', async () => { + const pool = stubPool([{ typmod: -1 }]); + const result = await checkEmbeddingDimensions(pool, config); + expect(result.ok).toBe(true); + expect(result.dbDims).toBeNull(); + }); +}); + +describe('createApp', () => { + it('returns an Express app wired from a runtime container', () => { + const pool = stubPool(); + const runtime = createCoreRuntime({ pool }); + const app = createApp(runtime); + expect(typeof app.use).toBe('function'); + expect(typeof app.listen).toBe('function'); + }); +}); diff --git a/src/app/create-app.ts b/src/app/create-app.ts new file mode 100644 index 0000000..2f2a989 --- /dev/null +++ b/src/app/create-app.ts @@ -0,0 +1,39 @@ +/** + * Express application factory — wires routers onto a runtime container. + * + * Separates composition (done in `runtime-container.ts`) from HTTP + * transport concerns. Tests and harnesses can create an Express app from + * any runtime container without touching the server bootstrap. + */ + +import express from 'express'; +import { createAgentRouter } from '../routes/agents.js'; +import { createMemoryRouter } from '../routes/memories.js'; +import type { CoreRuntime } from './runtime-container.js'; + +/** + * Build an Express application from a composed runtime container. The + * runtime owns all deps; this module only wires HTTP concerns (CORS, body + * parsing, routes, health). + */ +export function createApp(runtime: CoreRuntime): ReturnType { + const app = express(); + + app.use((_req, res, next) => { + res.header('Access-Control-Allow-Origin', '*'); + res.header('Access-Control-Allow-Methods', 'GET, POST, PUT, DELETE, OPTIONS'); + res.header('Access-Control-Allow-Headers', 'Content-Type'); + next(); + }); + + app.use(express.json({ limit: '1mb' })); + + app.use('/memories', createMemoryRouter(runtime.services.memory)); + app.use('/agents', createAgentRouter(runtime.repos.trust)); + + app.get('/health', (_req, res) => { + res.json({ status: 'ok' }); + }); + + return app; +} diff --git a/src/app/runtime-container.ts b/src/app/runtime-container.ts new file mode 100644 index 0000000..fa2386f --- /dev/null +++ b/src/app/runtime-container.ts @@ -0,0 +1,107 @@ +/** + * Core runtime container — the explicit composition root for Atomicmemory-core. + * + * Owns the construction of config, pool, repositories, and services so + * startup (`server.ts`), tests, and in-process research harnesses all boot + * through the same seam. Replaces the hidden singleton wiring that used to + * live inline in `server.ts`. + * + * Phase 1A of the rearchitecture — see + * atomicmemory-research/docs/atomicmemory-core-rearchitecture-plan-2026-04-16.md. + */ + +import pg from 'pg'; +import { config } from '../config.js'; +import { AgentTrustRepository } from '../db/agent-trust-repository.js'; +import { ClaimRepository } from '../db/claim-repository.js'; +import { LinkRepository } from '../db/link-repository.js'; +import { MemoryRepository } from '../db/memory-repository.js'; +import { EntityRepository } from '../db/repository-entities.js'; +import { LessonRepository } from '../db/repository-lessons.js'; +import { MemoryService } from '../services/memory-service.js'; + +/** + * Public runtime configuration subset. Phase 1A exposes the full config + * object for compatibility; later phases will split public runtime config + * from internal policy flags. + * + * NOTE (phase 1a.5): `runtime.config` currently references the same + * module-level singleton that routes, services, and the search pipeline + * all read from directly. There is no per-runtime config copy yet — + * consumers cannot construct two runtimes with different configs because + * the deeper service code (25+ `import { config }` sites across + * routes/, services/) reads the module singleton regardless. Phase 1B + * will thread config through properly and reintroduce a genuine override. + */ +export type CoreRuntimeConfig = typeof config; + +/** Repositories constructed by the runtime container. */ +export interface CoreRuntimeRepos { + memory: MemoryRepository; + claims: ClaimRepository; + trust: AgentTrustRepository; + links: LinkRepository; + entities: EntityRepository | null; + lessons: LessonRepository | null; +} + +/** Services constructed on top of repositories. */ +export interface CoreRuntimeServices { + memory: MemoryService; +} + +/** + * Explicit dependency bundle accepted by `createCoreRuntime`. + * + * `pool` is required — the composition root never reaches around to + * import the singleton `pg.Pool` itself. + * + * A `config` override is deliberately NOT accepted here. Most downstream + * route and service code still reads config directly from the module + * singleton, so any override passed here would only influence repo + * construction (`entityGraphEnabled`, `lessonsEnabled`) and silently be + * ignored everywhere else — a dishonest contract. Phase 1B will thread + * config through routes and services and reintroduce a genuine override. + */ +export interface CoreRuntimeDeps { + pool: pg.Pool; +} + +/** The composed runtime — single source of truth for route registration. */ +export interface CoreRuntime { + config: CoreRuntimeConfig; + pool: pg.Pool; + repos: CoreRuntimeRepos; + services: CoreRuntimeServices; +} + +/** + * Compose the core runtime. Instantiates repositories and the memory + * service from an explicit pool. Reads the module-level config singleton + * for repo-construction flags so there is a single source of truth + * between the container and the rest of the codebase. No mutation. + */ +export function createCoreRuntime(deps: CoreRuntimeDeps): CoreRuntime { + const { pool } = deps; + + const memory = new MemoryRepository(pool); + const claims = new ClaimRepository(pool); + const trust = new AgentTrustRepository(pool); + const links = new LinkRepository(pool); + const entities = config.entityGraphEnabled ? new EntityRepository(pool) : null; + const lessons = config.lessonsEnabled ? new LessonRepository(pool) : null; + + const service = new MemoryService( + memory, + claims, + entities ?? undefined, + lessons ?? undefined, + ); + + return { + config, + pool, + repos: { memory, claims, trust, links, entities, lessons }, + services: { memory: service }, + }; +} diff --git a/src/app/startup-checks.ts b/src/app/startup-checks.ts new file mode 100644 index 0000000..7045706 --- /dev/null +++ b/src/app/startup-checks.ts @@ -0,0 +1,70 @@ +/** + * Startup guard functions run before the HTTP server accepts traffic. + * + * Extracted from `server.ts` so tests can exercise individual checks and + * future startup guards can be added without growing the server bootstrap + * module. Phase 1A of the rearchitecture. + */ + +import pg from 'pg'; +import type { CoreRuntimeConfig } from './runtime-container.js'; + +/** + * Result of an embedding dimension check against the `memories.embedding` + * column. `ok=false` means the process should exit before serving traffic. + */ +export interface EmbeddingDimensionCheckResult { + ok: boolean; + dbDims: number | null; + configDims: number; + message: string; +} + +/** + * Verify DB embedding column dimensions match the configured embedding size. + * Catches the "expected N dimensions, not M" class of errors at startup, + * which would otherwise surface as opaque insert failures during ingest. + * + * This function never throws or exits — it returns a structured result so + * the caller decides how to react (log + exit at boot, throw in tests). + */ +export async function checkEmbeddingDimensions( + pool: pg.Pool, + config: CoreRuntimeConfig, +): Promise { + const { rows } = await pool.query<{ typmod: number }>( + `SELECT atttypmod AS typmod + FROM pg_attribute a + JOIN pg_class c ON a.attrelid = c.oid + WHERE c.relname = 'memories' AND a.attname = 'embedding'`, + ); + + if (rows.length === 0) { + return { + ok: false, + dbDims: null, + configDims: config.embeddingDimensions, + message: 'memories.embedding column not found — run npm run migrate first', + }; + } + + const dbDims = rows[0].typmod > 0 ? rows[0].typmod : null; + + if (dbDims !== null && dbDims !== config.embeddingDimensions) { + return { + ok: false, + dbDims, + configDims: config.embeddingDimensions, + message: + `DB vector column is ${dbDims} dimensions but EMBEDDING_DIMENSIONS=${config.embeddingDimensions}. ` + + `Fix: set EMBEDDING_DIMENSIONS=${dbDims} or run 'npm run migrate' to recreate the schema.`, + }; + } + + return { + ok: true, + dbDims, + configDims: config.embeddingDimensions, + message: `Embedding dimensions OK: config=${config.embeddingDimensions}, DB=${dbDims ?? 'unset'}`, + }; +} diff --git a/src/server.ts b/src/server.ts index e6f0c59..3b938ce 100644 --- a/src/server.ts +++ b/src/server.ts @@ -1,83 +1,52 @@ /** - * AtomicMemory Core API Server. - * Express server with memory ingest, search, and agent trust endpoints. + * AtomicMemory Core API Server — bootstrap entry point. + * + * Composes the runtime container, runs startup guards, builds the Express + * app, and starts listening. All composition logic lives in `./app/`; + * this file only owns the process lifecycle (boot → listen → shutdown). + * + * The `runtime` is the single source of truth for config, pool, repos, + * and services. Nothing in this file reaches around it to import + * singletons directly — if a consumer bootstraps with custom deps later, + * shutdown and lifecycle still act on the right graph. */ -import express from 'express'; import { pool } from './db/pool.js'; -import { AgentTrustRepository } from './db/agent-trust-repository.js'; -import { ClaimRepository } from './db/claim-repository.js'; -import { LinkRepository } from './db/link-repository.js'; -import { MemoryRepository } from './db/memory-repository.js'; -import { EntityRepository } from './db/repository-entities.js'; -import { LessonRepository } from './db/repository-lessons.js'; -import { MemoryService } from './services/memory-service.js'; -import { setEntityRepository } from './services/search-pipeline.js'; -import { createAgentRouter } from './routes/agents.js'; -import { createMemoryRouter } from './routes/memories.js'; -import { config } from './config.js'; +import { createCoreRuntime } from './app/runtime-container.js'; +import { createApp } from './app/create-app.js'; +import { checkEmbeddingDimensions } from './app/startup-checks.js'; -const app: ReturnType = express(); -app.use((_req, res, next) => { - res.header('Access-Control-Allow-Origin', '*'); - res.header('Access-Control-Allow-Methods', 'GET, POST, PUT, DELETE, OPTIONS'); - res.header('Access-Control-Allow-Headers', 'Content-Type'); - next(); -}); -app.use(express.json({ limit: '1mb' })); - -const repo = new MemoryRepository(pool); -const claimRepo = new ClaimRepository(pool); -const trustRepo = new AgentTrustRepository(pool); -const linkRepo = new LinkRepository(pool); -const entityRepo = config.entityGraphEnabled ? new EntityRepository(pool) : undefined; -if (entityRepo) setEntityRepository(entityRepo); -const lessonRepo = config.lessonsEnabled ? new LessonRepository(pool) : undefined; -const service = new MemoryService(repo, claimRepo, entityRepo, lessonRepo); +// Compose the runtime from explicit deps. The singleton pool is +// imported here (and only here) so the composition root itself has no +// side-effectful singleton dependencies. +const runtime = createCoreRuntime({ pool }); +const app = createApp(runtime); -app.use('/memories', createMemoryRouter(service)); -app.use('/agents', createAgentRouter(trustRepo)); - -app.get('/health', (_req, res) => { - res.json({ status: 'ok' }); -}); +// Re-export composed pieces for existing consumers (tests, research harnesses). +// These preserve the public surface from the previous server.ts. +const service = runtime.services.memory; +const repo = runtime.repos.memory; +const claimRepo = runtime.repos.claims; +const trustRepo = runtime.repos.trust; +const linkRepo = runtime.repos.links; -/** - * Verify DB embedding column dimensions match config before accepting traffic. - * Catches the "expected N dimensions, not M" class of errors at startup. - */ -async function checkEmbeddingDimensions(): Promise { - const { rows } = await pool.query<{ typmod: number }>( - `SELECT atttypmod AS typmod - FROM pg_attribute a - JOIN pg_class c ON a.attrelid = c.oid - WHERE c.relname = 'memories' AND a.attname = 'embedding'`, - ); - if (rows.length === 0) { - console.error('[startup] memories.embedding column not found — run npm run migrate first'); - process.exit(1); - } - const dbDims = rows[0].typmod > 0 ? rows[0].typmod : null; - if (dbDims !== null && dbDims !== config.embeddingDimensions) { - console.error( - `[startup] FATAL: DB vector column is ${dbDims} dimensions but EMBEDDING_DIMENSIONS=${config.embeddingDimensions}.\n` + - ` Fix: set EMBEDDING_DIMENSIONS=${dbDims} or run 'npm run migrate' to recreate the schema.`, - ); +async function bootstrap(): Promise { + const check = await checkEmbeddingDimensions(runtime.pool, runtime.config); + if (!check.ok) { + console.error(`[startup] FATAL: ${check.message}`); process.exit(1); } - console.log(`[startup] Embedding dimensions OK: config=${config.embeddingDimensions}, DB=${dbDims ?? 'unset'}`); -} + console.log(`[startup] ${check.message}`); -checkEmbeddingDimensions() - .then(() => { - app.listen(config.port, () => { - console.log(`AtomicMemory Core running on http://localhost:${config.port}`); - }); - }) - .catch((err) => { - console.error('[startup] Embedding dimension check failed:', err); - process.exit(1); + app.listen(runtime.config.port, () => { + console.log(`AtomicMemory Core running on http://localhost:${runtime.config.port}`); }); +} + +bootstrap().catch((err) => { + console.error('[startup] bootstrap failed:', err); + process.exit(1); +}); process.on('uncaughtException', (err) => { console.error('[FATAL] Uncaught exception:', err); @@ -90,12 +59,12 @@ process.on('unhandledRejection', (reason) => { process.on('SIGTERM', () => { console.log('[shutdown] Received SIGTERM, closing...'); - pool.end().then(() => process.exit(0)); + runtime.pool.end().then(() => process.exit(0)); }); process.on('SIGINT', () => { console.log('[shutdown] Received SIGINT, closing...'); - pool.end().then(() => process.exit(0)); + runtime.pool.end().then(() => process.exit(0)); }); -export { app, service, repo, claimRepo, trustRepo, linkRepo }; +export { app, service, repo, claimRepo, trustRepo, linkRepo, runtime }; diff --git a/src/services/__tests__/current-state-retrieval-regression.test.ts b/src/services/__tests__/current-state-retrieval-regression.test.ts index 96c1d20..7718f8d 100644 --- a/src/services/__tests__/current-state-retrieval-regression.test.ts +++ b/src/services/__tests__/current-state-retrieval-regression.test.ts @@ -31,7 +31,6 @@ vi.mock('../embedding.js', async () => { import { config } from '../../config.js'; import { pool } from '../../db/pool.js'; import { createServiceTestContext, unitVector, offsetVector } from '../../db/__tests__/test-fixtures.js'; -import { setEntityRepository } from '../search-pipeline.js'; const TEST_USER = 'current-state-retrieval-regression-user'; const OLD_CONVERSATION = [ @@ -60,7 +59,6 @@ describe('current-state retrieval regression', () => { beforeEach(async () => { mockChat.mockReset(); embeddingOverrides.clear(); - setEntityRepository(null); await claimRepo.deleteAll(); await repo.deleteAll(); registerEmbeddings(); diff --git a/src/services/__tests__/test-fixtures.ts b/src/services/__tests__/test-fixtures.ts index 1e30956..a35b41e 100644 --- a/src/services/__tests__/test-fixtures.ts +++ b/src/services/__tests__/test-fixtures.ts @@ -103,7 +103,6 @@ export function createSearchPipelineMockFactory( return { runSearchPipelineWithTrace: (...args: unknown[]) => mockRunSearchPipelineWithTrace(...args), generateLinks: () => {}, - setEntityRepository: () => {}, }; } diff --git a/src/services/memory-search.ts b/src/services/memory-search.ts index 2e3af0b..d9152c1 100644 --- a/src/services/memory-search.ts +++ b/src/services/memory-search.ts @@ -70,7 +70,7 @@ async function executeSearchStep( trace.stage('as-of-search', memories, { asOf }); return { memories, activeTrace: trace }; } - const pipelineResult = await runSearchPipelineWithTrace(deps.repo, userId, query, effectiveLimit, sourceSite, referenceTime, { + const pipelineResult = await runSearchPipelineWithTrace(deps.repo, deps.entities, userId, query, effectiveLimit, sourceSite, referenceTime, { namespaceScope, retrievalMode: retrievalOptions?.retrievalMode, searchStrategy: retrievalOptions?.searchStrategy, diff --git a/src/services/search-pipeline.ts b/src/services/search-pipeline.ts index 26b9ead..91e954f 100644 --- a/src/services/search-pipeline.ts +++ b/src/services/search-pipeline.ts @@ -53,9 +53,6 @@ function shouldAutoSkipReranking(results: SearchResult[]): boolean { return topSim >= config.rerankSkipTopSimilarity && (topSim - secondSim) >= config.rerankSkipMinGap; } -/** Optional entity repository for entity-aware retrieval expansion. */ -let entityRepo: EntityRepository | null = null; - export interface SearchPipelineOptions { namespaceScope?: string; retrievalMode?: RetrievalMode; @@ -66,15 +63,12 @@ export interface SearchPipelineOptions { skipReranking?: boolean; } -export function setEntityRepository(repo: EntityRepository | null): void { - entityRepo = repo; -} - /** * Core search pipeline implementation with explicit trace collection. */ export async function runSearchPipelineWithTrace( repo: MemoryRepository, + entityRepo: EntityRepository | null, userId: string, query: string, limit: number, @@ -91,21 +85,21 @@ export async function runSearchPipelineWithTrace( // Phase 2: Entity-grounded query augmentation (zero-LLM) const augmentation = await timed('search.augmentation', () => applyQueryAugmentation( - userId, query, rawQueryEmbedding, trace, + entityRepo, userId, query, rawQueryEmbedding, trace, )); const queryEmbedding = augmentation.augmentedEmbedding; const searchQuery = augmentation.searchQuery; const initialResults = await timed('search.vector', () => runInitialRetrieval( - repo, userId, searchQuery, queryEmbedding, candidateDepth, sourceSite, referenceTime, options.searchStrategy, + repo, entityRepo, userId, searchQuery, queryEmbedding, candidateDepth, sourceSite, referenceTime, options.searchStrategy, )); const seededResults = await timed('search.hybrid-fallback', () => maybeApplyAbstractHybridFallback( - repo, userId, query, searchQuery, queryEmbedding, candidateDepth, sourceSite, referenceTime, + repo, entityRepo, userId, query, searchQuery, queryEmbedding, candidateDepth, sourceSite, referenceTime, options.retrievalMode, options.searchStrategy, initialResults, trace, )); console.log(`[search] Query: "${query}", Results: ${seededResults.length}`); - + trace.stage('initial', seededResults, { candidateDepth, hybrid: config.hybridSearchEnabled, @@ -117,7 +111,7 @@ export async function runSearchPipelineWithTrace( // Entity name co-retrieval const withCoRetrieval = await timed('search.co-retrieval', () => applyEntityNameCoRetrieval( - repo, userId, query, queryEmbedding, seededResults, candidateDepth, trace, + repo, entityRepo, userId, query, queryEmbedding, seededResults, candidateDepth, trace, )); const withSubjectExpansion = await timed('search.subject-query-expansion', () => applySubjectQueryExpansion( @@ -134,13 +128,14 @@ export async function runSearchPipelineWithTrace( // Query expansion const withExpansion = await timed('search.query-expansion', () => applyQueryExpansion( - repo, userId, query, queryEmbedding, temporalExpansion.memories, candidateDepth, trace, + repo, entityRepo, userId, query, queryEmbedding, temporalExpansion.memories, candidateDepth, trace, )); const repaired = options.skipRepairLoop ? { memories: withExpansion, queryText: searchQuery } : await timed('search.repair-loop', () => applyRepairLoop( repo, + entityRepo, query, queryEmbedding, withExpansion, @@ -192,6 +187,7 @@ export async function runSearchPipelineWithTrace( const selected = await timed('search.expansion-reranking', () => applyExpansionAndReranking( repo, + entityRepo, userId, searchQuery, results, @@ -222,6 +218,7 @@ export async function runSearchPipelineWithTrace( async function runInitialRetrieval( repo: MemoryRepository, + entityRepo: EntityRepository | null, userId: string, searchQuery: string, queryEmbedding: number[], @@ -242,6 +239,7 @@ async function runInitialRetrieval( } return runMemoryRrfRetrieval( repo, + entityRepo, userId, searchQuery, queryEmbedding, @@ -254,6 +252,7 @@ async function runInitialRetrieval( async function maybeApplyAbstractHybridFallback( repo: MemoryRepository, + entityRepo: EntityRepository | null, userId: string, rawQuery: string, searchQuery: string, @@ -273,6 +272,7 @@ async function maybeApplyAbstractHybridFallback( } const fallbackResults = await runMemoryRrfRetrieval( repo, + entityRepo, userId, searchQuery, queryEmbedding, @@ -290,6 +290,7 @@ async function maybeApplyAbstractHybridFallback( */ async function applyRepairLoop( repo: MemoryRepository, + entityRepo: EntityRepository | null, query: string, queryEmbedding: number[], initialResults: SearchResult[], @@ -316,6 +317,7 @@ async function applyRepairLoop( ? await repo.searchAtomicFactsHybrid(userId, rewrittenQuery, rewrittenEmbedding, candidateDepth, sourceSite, referenceTime) : await runMemoryRrfRetrieval( repo, + entityRepo, userId, rewrittenQuery, rewrittenEmbedding, @@ -363,6 +365,7 @@ async function applyRepairLoop( */ async function applyQueryExpansion( repo: MemoryRepository, + entityRepo: EntityRepository | null, userId: string, query: string, queryEmbedding: number[], @@ -405,6 +408,7 @@ async function applyQueryExpansion( * If disabled or no entities match, returns the original query and embedding unchanged. */ async function applyQueryAugmentation( + entityRepo: EntityRepository | null, userId: string, query: string, queryEmbedding: number[], @@ -443,6 +447,7 @@ async function applyQueryAugmentation( */ async function applyEntityNameCoRetrieval( repo: MemoryRepository, + entityRepo: EntityRepository | null, userId: string, query: string, queryEmbedding: number[], @@ -603,6 +608,7 @@ async function applySubjectQueryExpansion( */ async function applyExpansionAndReranking( repo: MemoryRepository, + entityRepo: EntityRepository | null, userId: string, query: string, results: SearchResult[], @@ -645,7 +651,7 @@ async function applyExpansionAndReranking( candidates = applyConcisenessPenalty(candidates); if (config.linkExpansionBeforeMMR && config.linkExpansionEnabled && config.mmrEnabled) { - const preExpanded = await expandWithLinks(repo, userId, candidates.slice(0, limit), queryEmbedding, referenceTime); + const preExpanded = await expandWithLinks(repo, entityRepo, userId, candidates.slice(0, limit), queryEmbedding, referenceTime); trace.stage('link-expansion', preExpanded, { order: 'before-mmr' }); const selected = preserveProtectedResults( applyMMR(preExpanded, queryEmbedding, limit, config.mmrLambda), @@ -665,13 +671,13 @@ async function applyExpansionAndReranking( limit, ); trace.stage('mmr', mmrResults, { lambda: config.mmrLambda }); - const expanded = await expandWithLinks(repo, userId, mmrResults, queryEmbedding, referenceTime); + const expanded = await expandWithLinks(repo, entityRepo, userId, mmrResults, queryEmbedding, referenceTime); trace.stage('link-expansion', expanded, { order: 'after-mmr' }); return expanded; } const sliced = preserveProtectedResults(candidates.slice(0, limit), candidates, protectedFingerprints, limit); - const expanded = await expandWithLinks(repo, userId, sliced, queryEmbedding, referenceTime); + const expanded = await expandWithLinks(repo, entityRepo, userId, sliced, queryEmbedding, referenceTime); trace.stage('link-expansion', expanded, { order: 'no-mmr' }); return expanded; } @@ -682,6 +688,7 @@ async function applyExpansionAndReranking( */ async function expandWithLinks( repo: MemoryRepository, + entityRepo: EntityRepository | null, userId: string, results: SearchResult[], queryEmbedding: number[], @@ -715,7 +722,7 @@ async function expandWithLinks( const dedupedTemporal = temporalNeighbors.filter((m) => !seen.has(m.id)); // Entity graph expansion: find entities matching the query and pull in their linked memories - const entityMemories = await expandViaEntities(repo, userId, queryEmbedding, seen, budget); + const entityMemories = await expandViaEntities(repo, entityRepo, userId, queryEmbedding, seen, budget); const expansions = [...linkedMemories, ...dedupedTemporal, ...entityMemories] .sort((a, b) => b.score - a.score) @@ -726,6 +733,7 @@ async function expandWithLinks( async function runMemoryRrfRetrieval( repo: MemoryRepository, + entityRepo: EntityRepository | null, userId: string, queryText: string, queryEmbedding: number[], @@ -746,7 +754,7 @@ async function runMemoryRrfRetrieval( ]; if (config.entityGraphEnabled && entityRepo) { - const entityResults = await expandViaEntities(repo, userId, queryEmbedding, new Set(), limit); + const entityResults = await expandViaEntities(repo, entityRepo, userId, queryEmbedding, new Set(), limit); if (entityResults.length > 0) { channels.push({ name: 'entity', weight: ENTITY_RRF_WEIGHT, results: entityResults }); } @@ -865,6 +873,7 @@ export async function generateLinks( */ async function expandViaEntities( repo: MemoryRepository, + entityRepo: EntityRepository | null, userId: string, queryEmbedding: number[], excludeIds: Set,