From c5c750c590a122656a59785866e00e9eac20933f Mon Sep 17 00:00:00 2001 From: Hoang Nguyen Date: Tue, 1 Sep 2026 06:08:31 +0000 Subject: [PATCH 01/11] feat(memory): add semantic storage primitives --- docs/ai/2026-09-01-feature-memory-semantic.md | 77 +++++++++++ .../migrations/002_semantic_embeddings.sql | 8 ++ packages/memory/src/services/semantic.ts | 127 ++++++++++++++++++ packages/memory/src/types/index.ts | 8 ++ .../integration/semantic-storage.test.ts | 29 ++++ packages/memory/tests/unit/semantic.test.ts | 46 +++++++ 6 files changed, 295 insertions(+) create mode 100644 docs/ai/2026-09-01-feature-memory-semantic.md create mode 100644 packages/memory/src/database/migrations/002_semantic_embeddings.sql create mode 100644 packages/memory/src/services/semantic.ts create mode 100644 packages/memory/tests/integration/semantic-storage.test.ts create mode 100644 packages/memory/tests/unit/semantic.test.ts diff --git a/docs/ai/2026-09-01-feature-memory-semantic.md b/docs/ai/2026-09-01-feature-memory-semantic.md new file mode 100644 index 00000000..b12a9c0e --- /dev/null +++ b/docs/ai/2026-09-01-feature-memory-semantic.md @@ -0,0 +1,77 @@ +# Feature: Opt-in hybrid semantic memory search + +## Requirements + +- Keep lexical strict-to-broad FTS as the default and as a protected hybrid channel. +- Enable semantic retrieval only when `memory.semantic` is `true`; absent means false. +- Work locally after a lazy, pinned model download and degrade to lexical-only when offline. +- Preserve deterministic, explainable ranking and warm p95 search latency below 50 ms at 1,000 rows. +- Add only an additive migration; never edit an applied migration. +- Provide semantic status/download and resumable re-embedding commands. +- Keep the embedding runtime below 150 MB excluding the model cache. + +Non-goals: ANN indexing, chunking memories, configurable models/fusion constants, automatic migration-time downloads, and multilingual retrieval. + +## Gate evidence and decisions + +The expanded-100 benchmark contains 100 memories and 100 queries. Against current strict-to-broad FTS, MiniLM hybrid RRF produced paraphrase nDCG@5 0.7590 versus 0.6576 (+15.41%) and recall@5 0.76 versus 0.68 (+11.76%). Overall MRR@5 improved from 0.8637 to 0.9275. Identifier-exact hit@1 stayed 0.96 with zero per-query regressions, and repeated fusion rankings were identical. BGE failed the paraphrase gates (+8.75% nDCG and +5.88% recall), so the smaller MiniLM model is selected. + +`onnxruntime-web@1.22.0` plus `@huggingface/tokenizers@0.1.3` installs at 100 MB on Linux and has no native/platform-specific dependency. Direct ONNX output matched Transformers.js output to float precision. The pinned q8 MiniLM model is a separate 23 MB lazy download. Measured runtime: 426 ms load, 58 ms first inference, and roughly 12-14 ms warm inference. + +## Design + +```mermaid +flowchart LR + Q[Query] --> F[Strict then broad FTS] + Q --> E[Local MiniLM embedder] + E --> C[Brute-force cosine] + F --> R[Deterministic RRF k=60] + C --> R + R --> O[Results and optional explanation] + E -. unavailable/stale/too large .-> L[Lexical-only degradation] +``` + +- Model: `Xenova/all-MiniLM-L6-v2` at revision `751bff37182d3f1213fa05d7196b954e230abad9`, q8, 384 dimensions. +- Cache: `~/.ai-devkit/models/Xenova--all-MiniLM-L6-v2//` with checksummed config, tokenizer, and ONNX files. +- Document text: trimmed title, blank line, trimmed content, then sorted tags when present. +- Storage: nullable float32 little-endian BLOB and an embedding compatibility version. +- Fusion: `1/(60 + lexicalRank) + 1/(60 + semanticRank)`, then lexical presence, lexical rank, semantic rank, and ID tie-breaks. +- Search skips semantic scanning above 5,000 eligible rows. Missing/stale embeddings remain lexical candidates. +- Updating title, content, or tags invalidates an embedding; a scope-only update retains it. +- Backfill processes stable ID-ordered batches and commits each replacement atomically. + +## Interfaces + +- Config: `memory.semantic: boolean`, default false. +- CLI: `memory semantic status`, `memory semantic download`, `memory reembed [--force]`, and `memory search --explain`. +- Search results retain lexical `strategy` (`strict`, `broad`, or `recent`) and add retrieval mode/status. Per-item channel ranks and RRF details appear only with explain enabled. +- Existing synchronous APIs remain lexical-compatible. Async semantic-aware command/server paths dynamically import the runtime only when enabled. + +## Implementation plan + +- [ ] Add migration and embedding invalidation/storage tests. +- [ ] Add deterministic semantic primitives and RRF tests. +- [ ] Add pinned model downloader and lightweight ONNX runtime tests. +- [ ] Add status, download, and resumable re-embed APIs and CLI tests. +- [ ] Add async hybrid search with offline/corrupt/stale/large-corpus degradation tests. +- [ ] Wire config through CLI and MCP while preserving default behavior. +- [ ] Run expanded-100 gate through the built CLI. +- [ ] Run 1,000-row warm latency gate. +- [ ] Run six-project build, full tests, lint, and e2e. + +## Testing record + +- [ ] Migration adds nullable columns without changing existing rows. +- [ ] Content fields invalidate embeddings; scope-only updates do not. +- [ ] Model file checksums and pinned revision are enforced. +- [ ] Runtime output is normalized and 384-dimensional. +- [ ] RRF is deterministic and protects lexical ties. +- [ ] Offline and missing-model search returns lexical results. +- [ ] Stale/corrupt embeddings are excluded safely. +- [ ] Backfill resumes, skips current rows, and force-rebuilds. +- [ ] CLI surfaces config, status, download, reembed, and explanations. +- [ ] Default lexical API and strict/broad strategies remain compatible. + +## Rollback + +Set `memory.semantic` to false. The additive columns become inert and older named-column queries continue to work. Deleting a knowledge row deletes its BLOB with no auxiliary index cleanup. Model files can be removed independently from the validated cache directory. A later cleanup migration may null the columns, but rollback does not require reversing schema history. diff --git a/packages/memory/src/database/migrations/002_semantic_embeddings.sql b/packages/memory/src/database/migrations/002_semantic_embeddings.sql new file mode 100644 index 00000000..786caa63 --- /dev/null +++ b/packages/memory/src/database/migrations/002_semantic_embeddings.sql @@ -0,0 +1,8 @@ +-- Migration 002: Optional semantic embeddings +-- Embeddings are nullable so migration and lexical search remain offline-safe. + +ALTER TABLE knowledge ADD COLUMN embedding BLOB; +ALTER TABLE knowledge ADD COLUMN embedding_version TEXT; + +CREATE INDEX IF NOT EXISTS idx_knowledge_embedding_version +ON knowledge(embedding_version); diff --git a/packages/memory/src/services/semantic.ts b/packages/memory/src/services/semantic.ts new file mode 100644 index 00000000..ff81a497 --- /dev/null +++ b/packages/memory/src/services/semantic.ts @@ -0,0 +1,127 @@ +import type { SearchResultItem, SearchRetrievalExplanation } from '../types/index.js'; + +export const EMBEDDING_DIMENSION = 384; +export const RRF_K = 60; + +interface EmbeddingDocument { + title: string; + content: string; + tags: string[]; +} + +export interface SemanticCandidate extends Omit { + similarity: number; +} + +export function buildEmbeddingText(document: EmbeddingDocument): string { + const tags = [...document.tags] + .map(tag => tag.trim()) + .filter(Boolean) + .sort((a, b) => a.localeCompare(b)); + const sections = [document.title.trim(), document.content.trim()]; + if (tags.length > 0) { + sections.push(`Tags: ${tags.join(', ')}`); + } + return sections.join('\n\n'); +} + +export function serializeEmbedding(vector: Float32Array): Buffer { + if (vector.length !== EMBEDDING_DIMENSION) { + throw new Error(`Embedding must contain ${EMBEDDING_DIMENSION} dimensions`); + } + return Buffer.from(vector.buffer.slice(vector.byteOffset, vector.byteOffset + vector.byteLength)); +} + +export function deserializeEmbedding(value: Buffer): Float32Array { + const expectedBytes = EMBEDDING_DIMENSION * Float32Array.BYTES_PER_ELEMENT; + if (value.byteLength !== expectedBytes) { + throw new Error(`Embedding BLOB must contain ${expectedBytes} bytes`); + } + const copy = Uint8Array.from(value); + return new Float32Array(copy.buffer); +} + +export function cosineSimilarity(left: Float32Array, right: Float32Array): number { + if (left.length !== right.length) { + throw new Error('Embedding dimensions must match'); + } + let dot = 0; + let leftNorm = 0; + let rightNorm = 0; + for (let index = 0; index < left.length; index++) { + dot += left[index]! * right[index]!; + leftNorm += left[index]! ** 2; + rightNorm += right[index]! ** 2; + } + if (leftNorm === 0 || rightNorm === 0) { + return 0; + } + return dot / Math.sqrt(leftNorm * rightNorm); +} + +export function fuseSearchResults( + lexical: SearchResultItem[], + semantic: SemanticCandidate[], + limit: number, + explain = false, +): SearchResultItem[] { + const fused = new Map(); + + lexical.forEach((item, index) => { + fused.set(item.id, { + item, + score: 1 / (RRF_K + index + 1), + lexicalRank: index + 1, + semanticRank: null, + similarity: null, + }); + }); + + semantic.forEach((candidate, index) => { + const existing = fused.get(candidate.id); + const semanticScore = 1 / (RRF_K + index + 1); + if (existing) { + existing.score += semanticScore; + existing.semanticRank = index + 1; + existing.similarity = candidate.similarity; + return; + } + const { similarity, ...item } = candidate; + fused.set(candidate.id, { + item: { ...item, score: 0 }, + score: semanticScore, + lexicalRank: null, + semanticRank: index + 1, + similarity, + }); + }); + + return [...fused.values()] + .sort((left, right) => + right.score - left.score + || Number(right.lexicalRank !== null) - Number(left.lexicalRank !== null) + || (left.lexicalRank ?? Number.POSITIVE_INFINITY) - (right.lexicalRank ?? Number.POSITIVE_INFINITY) + || (left.semanticRank ?? Number.POSITIVE_INFINITY) - (right.semanticRank ?? Number.POSITIVE_INFINITY) + || left.item.id.localeCompare(right.item.id) + ) + .slice(0, limit) + .map(entry => { + const retrieval: SearchRetrievalExplanation | undefined = explain ? { + lexicalRank: entry.lexicalRank, + semanticRank: entry.semanticRank, + semanticSimilarity: entry.similarity, + rrfScore: entry.score, + } : undefined; + return { + ...entry.item, + score: Math.round(entry.score * 1_000_000) / 1_000_000, + ...(retrieval ? { retrieval } : {}), + }; + }); +} diff --git a/packages/memory/src/types/index.ts b/packages/memory/src/types/index.ts index 900e93a5..b72eff54 100644 --- a/packages/memory/src/types/index.ts +++ b/packages/memory/src/types/index.ts @@ -53,6 +53,14 @@ export interface SearchResultItem { tags: string[]; scope: string; score: number; + retrieval?: SearchRetrievalExplanation; +} + +export interface SearchRetrievalExplanation { + lexicalRank: number | null; + semanticRank: number | null; + semanticSimilarity: number | null; + rrfScore: number; } export interface SearchKnowledgeResult { diff --git a/packages/memory/tests/integration/semantic-storage.test.ts b/packages/memory/tests/integration/semantic-storage.test.ts new file mode 100644 index 00000000..ed850bf8 --- /dev/null +++ b/packages/memory/tests/integration/semantic-storage.test.ts @@ -0,0 +1,29 @@ +import { join } from 'path'; +import { tmpdir } from 'os'; +import { rmSync } from 'fs'; +import { DatabaseConnection } from '../../src/database/connection'; +import { initializeSchema, getSchemaVersion } from '../../src/database/schema'; + +describe('semantic embedding migration', () => { + const dbPath = join(tmpdir(), `semantic-storage-${Date.now()}-${Math.random().toString(36)}.db`); + let db: DatabaseConnection; + + beforeAll(() => { + db = new DatabaseConnection({ dbPath }); + initializeSchema(db); + }); + + afterAll(() => { + db.close(); + rmSync(dbPath, { force: true }); + rmSync(`${dbPath}-wal`, { force: true }); + rmSync(`${dbPath}-shm`, { force: true }); + }); + + it('adds nullable embedding and version columns in migration 002', () => { + const columns = db.query<{ name: string }>('PRAGMA table_info(knowledge)').map(row => row.name); + expect(getSchemaVersion(db)).toBe(2); + expect(columns).toContain('embedding'); + expect(columns).toContain('embedding_version'); + }); +}); diff --git a/packages/memory/tests/unit/semantic.test.ts b/packages/memory/tests/unit/semantic.test.ts new file mode 100644 index 00000000..5ee85376 --- /dev/null +++ b/packages/memory/tests/unit/semantic.test.ts @@ -0,0 +1,46 @@ +import { + buildEmbeddingText, + cosineSimilarity, + fuseSearchResults, + serializeEmbedding, + deserializeEmbedding, +} from '../../src/services/semantic'; + +describe('semantic retrieval primitives', () => { + it('builds deterministic document text with sorted tags', () => { + expect(buildEmbeddingText({ + title: ' Response DTOs ', + content: ' Do not expose entities. ', + tags: ['zod', 'api'], + })).toBe('Response DTOs\n\nDo not expose entities.\n\nTags: api, zod'); + }); + + it('round-trips float32 embeddings and rejects the wrong dimension', () => { + const vector = Float32Array.from({ length: 384 }, (_, index) => index / 384); + expect([...deserializeEmbedding(serializeEmbedding(vector))]).toEqual([...vector]); + expect(() => serializeEmbedding(new Float32Array(3))).toThrow('384'); + }); + + it('calculates cosine similarity for normalized vectors', () => { + expect(cosineSimilarity(new Float32Array([1, 0]), new Float32Array([1, 0]))).toBe(1); + expect(cosineSimilarity(new Float32Array([1, 0]), new Float32Array([0, 1]))).toBe(0); + }); + + it('uses deterministic RRF and protects lexical results on ties', () => { + const lexical = [ + { id: 'exact', title: 'Exact', content: 'x', tags: [], scope: 'global', score: 10 }, + { id: 'shared', title: 'Shared', content: 'x', tags: [], scope: 'global', score: 9 }, + ]; + const semantic = [ + { id: 'semantic', title: 'Semantic', content: 'x', tags: [], scope: 'global', similarity: 0.9 }, + { id: 'shared', title: 'Shared', content: 'x', tags: [], scope: 'global', similarity: 0.8 }, + ]; + + const first = fuseSearchResults(lexical, semantic, 5, true); + const second = fuseSearchResults(lexical, semantic, 5, true); + + expect(first.map(item => item.id)).toEqual(['shared', 'exact', 'semantic']); + expect(first).toEqual(second); + expect(first[1]?.retrieval).toMatchObject({ lexicalRank: 1, semanticRank: null }); + }); +}); From cc72145a31b771e2f076a8a029ed70d4ee38e56d Mon Sep 17 00:00:00 2001 From: Hoang Nguyen Date: Tue, 1 Sep 2026 06:11:23 +0000 Subject: [PATCH 02/11] feat(memory): add lightweight local embedding runtime --- package-lock.json | 132 +++++++++++++++++++++++ packages/memory/package.json | 2 + packages/memory/src/services/embedder.ts | 102 ++++++++++++++++++ packages/memory/src/services/model.ts | 118 ++++++++++++++++++++ packages/memory/tests/unit/model.test.ts | 67 ++++++++++++ 5 files changed, 421 insertions(+) create mode 100644 packages/memory/src/services/embedder.ts create mode 100644 packages/memory/src/services/model.ts create mode 100644 packages/memory/tests/unit/model.test.ts diff --git a/package-lock.json b/package-lock.json index d2227552..0f9da8c1 100644 --- a/package-lock.json +++ b/package-lock.json @@ -3738,6 +3738,12 @@ "hono": "^4" } }, + "node_modules/@huggingface/tokenizers": { + "version": "0.1.3", + "resolved": "https://registry.npmjs.org/@huggingface/tokenizers/-/tokenizers-0.1.3.tgz", + "integrity": "sha512-8rF/RRT10u+kn7YuUbUg0OF30K8rjTc78aHpxT+qJ1uWSqxT1MHi8+9ltwYfkFYJzT/oS+qw3JVfHtNMGAdqyA==", + "license": "Apache-2.0" + }, "node_modules/@humanwhocodes/config-array": { "version": "0.13.0", "license": "Apache-2.0", @@ -5012,6 +5018,63 @@ "node": ">=0.10" } }, + "node_modules/@protobufjs/aspromise": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/@protobufjs/aspromise/-/aspromise-1.1.2.tgz", + "integrity": "sha512-j+gKExEuLmKwvz3OgROXtrJ2UG2x8Ch2YZUxahh+s1F2HZ+wAceUNLkvy6zKCPVRkU++ZWQrdxsUeQXmcg4uoQ==", + "license": "BSD-3-Clause" + }, + "node_modules/@protobufjs/base64": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/@protobufjs/base64/-/base64-1.1.2.tgz", + "integrity": "sha512-AZkcAA5vnN/v4PDqKyMR5lx7hZttPDgClv83E//FMNhR2TMcLUhfRUBHCmSl0oi9zMgDDqRUJkSxO3wm85+XLg==", + "license": "BSD-3-Clause" + }, + "node_modules/@protobufjs/codegen": { + "version": "2.0.5", + "resolved": "https://registry.npmjs.org/@protobufjs/codegen/-/codegen-2.0.5.tgz", + "integrity": "sha512-zgXFLzW3Ap33e6d0Wlj4MGIm6Ce8O89n/apUaGNB/jx+hw+ruWEp7EwGUshdLKVRCxZW12fp9r40E1mQrf/34g==", + "license": "BSD-3-Clause" + }, + "node_modules/@protobufjs/eventemitter": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@protobufjs/eventemitter/-/eventemitter-1.1.1.tgz", + "integrity": "sha512-vW1GmwMZNnL+gMRaovlh9yZX74kc+TTU3FObkkurpMaRtBfLP3ldjS9KQWlwZgraRE0+dheEEoAxdzcJQ8eXZg==", + "license": "BSD-3-Clause" + }, + "node_modules/@protobufjs/fetch": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@protobufjs/fetch/-/fetch-1.1.1.tgz", + "integrity": "sha512-GpptLrs57adMSuHi3VNj0mAF8dwh36LMaYF6XyJ6JMWlVsc+t42tm1HSEDmOs3A8fC9yyeisgLhsTVQokOZ0zw==", + "license": "BSD-3-Clause", + "dependencies": { + "@protobufjs/aspromise": "^1.1.1" + } + }, + "node_modules/@protobufjs/float": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/@protobufjs/float/-/float-1.0.2.tgz", + "integrity": "sha512-Ddb+kVXlXst9d+R9PfTIxh1EdNkgoRe5tOX6t01f1lYWOvJnSPDBlG241QLzcyPdoNTsblLUdujGSE4RzrTZGQ==", + "license": "BSD-3-Clause" + }, + "node_modules/@protobufjs/path": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/@protobufjs/path/-/path-1.1.2.tgz", + "integrity": "sha512-6JOcJ5Tm08dOHAbdR3GrvP+yUUfkjG5ePsHYczMFLq3ZmMkAD98cDgcT2iA1lJ9NVwFd4tH/iSSoe44YWkltEA==", + "license": "BSD-3-Clause" + }, + "node_modules/@protobufjs/pool": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/@protobufjs/pool/-/pool-1.1.0.tgz", + "integrity": "sha512-0kELaGSIDBKvcgS4zkjz1PeddatrjYcmMWOlAuAPwAeccUrPHdUqo/J6LiymHHEiJT5NrF1UVwxY14f+fy4WQw==", + "license": "BSD-3-Clause" + }, + "node_modules/@protobufjs/utf8": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/@protobufjs/utf8/-/utf8-1.1.2.tgz", + "integrity": "sha512-b1UQwcEZ4yCnMCD8DAL1VlbvBJE9/IX4FTIp7BG1xYpf29SLazLSrqUkj4w7Y5y7cCVP6E5tcqqcI0xemPkHug==", + "license": "BSD-3-Clause" + }, "node_modules/@rolldown/binding-android-arm64": { "version": "1.0.3", "resolved": "https://registry.npmjs.org/@rolldown/binding-android-arm64/-/binding-android-arm64-1.0.3.tgz", @@ -8937,6 +9000,12 @@ "node": "^10.12.0 || >=12.0.0" } }, + "node_modules/flatbuffers": { + "version": "25.9.23", + "resolved": "https://registry.npmjs.org/flatbuffers/-/flatbuffers-25.9.23.tgz", + "integrity": "sha512-MI1qs7Lo4Syw0EOzUl0xjs2lsoeqFku44KpngfIduHBYvzm8h2+7K8YMQh1JtVVVrUvhLpNwqVi4DERegUJhPQ==", + "license": "Apache-2.0" + }, "node_modules/flatted": { "version": "3.4.2", "license": "ISC" @@ -9351,6 +9420,12 @@ "js-yaml": "bin/js-yaml.js" } }, + "node_modules/guid-typescript": { + "version": "1.0.9", + "resolved": "https://registry.npmjs.org/guid-typescript/-/guid-typescript-1.0.9.tgz", + "integrity": "sha512-Y8T4vYhEfwJOTbouREvG+3XDsjr8E3kIr7uf+JZ0BYloFsttiHU0WfvANVsR7TxNUJa/WpCnw/Ino/p+DeBhBQ==", + "license": "ISC" + }, "node_modules/has-flag": { "version": "4.0.0", "license": "MIT", @@ -10473,6 +10548,12 @@ "url": "https://github.com/sponsors/sindresorhus" } }, + "node_modules/long": { + "version": "5.3.2", + "resolved": "https://registry.npmjs.org/long/-/long-5.3.2.tgz", + "integrity": "sha512-mNAgZ1GmyNhD7AuqnTG3/VQ26o760+ZYBPKjPvugO8+nLbYfX6TVpJPseBvopbdY+qpZ/lKUnmEc1LeZYS3QAA==", + "license": "Apache-2.0" + }, "node_modules/lowercase-keys": { "version": "3.0.0", "resolved": "https://registry.npmjs.org/lowercase-keys/-/lowercase-keys-3.0.0.tgz", @@ -11201,6 +11282,26 @@ "url": "https://github.com/sponsors/sindresorhus" } }, + "node_modules/onnxruntime-common": { + "version": "1.22.0", + "resolved": "https://registry.npmjs.org/onnxruntime-common/-/onnxruntime-common-1.22.0.tgz", + "integrity": "sha512-vcuaNWgtF2dGQu/EP5P8UI5rEPEYqXG2sPPe5j9lg2TY/biJF8eWklTMwlDO08iuXq48xJo0awqIpK5mPG+IxA==", + "license": "MIT" + }, + "node_modules/onnxruntime-web": { + "version": "1.22.0", + "resolved": "https://registry.npmjs.org/onnxruntime-web/-/onnxruntime-web-1.22.0.tgz", + "integrity": "sha512-Ud/+EBo6mhuaQWt/OjaOk0iNWjXqJoeeMFr6xQEERZdIZH2OWpGzuujz7lfuOBjUa6TEE/sc4nb7Da5dNL34fg==", + "license": "MIT", + "dependencies": { + "flatbuffers": "^25.1.24", + "guid-typescript": "^1.0.9", + "long": "^5.2.3", + "onnxruntime-common": "1.22.0", + "platform": "^1.3.6", + "protobufjs": "^7.2.4" + } + }, "node_modules/open": { "version": "8.4.2", "dev": true, @@ -11505,6 +11606,12 @@ "node": ">=16.20.0" } }, + "node_modules/platform": { + "version": "1.3.6", + "resolved": "https://registry.npmjs.org/platform/-/platform-1.3.6.tgz", + "integrity": "sha512-fnWVljUchTro6RiCFvCXBbNhJc2NijN7oIQxbwsyL0buWJPG85v81ehlHI9fXrJsMNgTofEoWIQeClKpgxFLrg==", + "license": "MIT" + }, "node_modules/postcss": { "version": "8.5.26", "resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.26.tgz", @@ -11581,6 +11688,29 @@ "url": "https://github.com/sponsors/sindresorhus" } }, + "node_modules/protobufjs": { + "version": "7.6.6", + "resolved": "https://registry.npmjs.org/protobufjs/-/protobufjs-7.6.6.tgz", + "integrity": "sha512-dYDWdjSl5RNb7SgPxGQcRU+GtvP7s2fpkrY0r432PcOIaZ0/rBcxEZnQN67iJhFuQiVw754JDoPruPCNdGsbjg==", + "hasInstallScript": true, + "license": "BSD-3-Clause", + "dependencies": { + "@protobufjs/aspromise": "^1.1.2", + "@protobufjs/base64": "^1.1.2", + "@protobufjs/codegen": "^2.0.5", + "@protobufjs/eventemitter": "^1.1.1", + "@protobufjs/fetch": "^1.1.1", + "@protobufjs/float": "^1.0.2", + "@protobufjs/path": "^1.1.2", + "@protobufjs/pool": "^1.1.0", + "@protobufjs/utf8": "^1.1.1", + "@types/node": ">=13.7.0", + "long": "^5.3.2" + }, + "engines": { + "node": ">=12.0.0" + } + }, "node_modules/proxy-addr": { "version": "2.0.7", "license": "MIT", @@ -13923,10 +14053,12 @@ "version": "0.17.0", "license": "MIT", "dependencies": { + "@huggingface/tokenizers": "0.1.3", "@modelcontextprotocol/sdk": "^1.0.0", "@typescript-eslint/eslint-plugin": "8.60.1", "@typescript-eslint/parser": "8.60.1", "better-sqlite3": "12.11.1", + "onnxruntime-web": "1.22.0", "uuid": "^14.0.0" }, "bin": { diff --git a/packages/memory/package.json b/packages/memory/package.json index ac41a47f..ba232b32 100644 --- a/packages/memory/package.json +++ b/packages/memory/package.json @@ -48,10 +48,12 @@ "directory": "packages/memory" }, "dependencies": { + "@huggingface/tokenizers": "0.1.3", "@modelcontextprotocol/sdk": "^1.0.0", "@typescript-eslint/eslint-plugin": "8.60.1", "@typescript-eslint/parser": "8.60.1", "better-sqlite3": "12.11.1", + "onnxruntime-web": "1.22.0", "uuid": "^14.0.0" }, "devDependencies": { diff --git a/packages/memory/src/services/embedder.ts b/packages/memory/src/services/embedder.ts new file mode 100644 index 00000000..fdd2e0a1 --- /dev/null +++ b/packages/memory/src/services/embedder.ts @@ -0,0 +1,102 @@ +import { readFile } from 'fs/promises'; +import { join } from 'path'; +import { ensureModelFiles, getModelDirectory, inspectModelFiles, normalizeEmbedding } from './model.js'; + +export class SemanticModelUnavailableError extends Error { + constructor(message: string, options?: ErrorOptions) { + super(message, options); + this.name = 'SemanticModelUnavailableError'; + } +} + +export function meanPoolAndNormalize( + data: Float32Array, + dimensions: readonly number[], + attentionMask: readonly number[], +): Float32Array { + const [, tokens, width] = dimensions; + if (!tokens || !width || dimensions.length !== 3 || data.length !== tokens * width) { + throw new Error('Semantic model returned an unexpected tensor shape'); + } + const pooled = new Float32Array(width); + let included = 0; + for (let token = 0; token < tokens; token++) { + if (!attentionMask[token]) continue; + included++; + for (let index = 0; index < width; index++) { + pooled[index] = pooled[index]! + data[token * width + index]!; + } + } + if (included === 0) { + throw new Error('Semantic tokenizer returned no input tokens'); + } + for (let index = 0; index < pooled.length; index++) { + pooled[index] = pooled[index]! / included; + } + return normalizeEmbedding(pooled); +} + +export interface LocalEmbedder { + embed(text: string): Promise; + embedMany(texts: string[]): Promise; + dispose(): Promise; +} + +interface LoadEmbedderOptions { + modelsRoot?: string; + download?: boolean; +} + +export async function loadLocalEmbedder(options: LoadEmbedderOptions = {}): Promise { + const directory = getModelDirectory(options.modelsRoot); + if (options.download) { + try { + await ensureModelFiles({ directory }); + } catch (error) { + throw new SemanticModelUnavailableError('Semantic model download failed', { cause: error }); + } + } else { + const inspection = await inspectModelFiles(directory); + if (!inspection.ready) { + throw new SemanticModelUnavailableError('Semantic model is not available in the local cache'); + } + } + + const [{ Tokenizer }, ort, tokenizerJson, tokenizerConfig, modelBytes] = await Promise.all([ + import('@huggingface/tokenizers'), + import('onnxruntime-web'), + readFile(join(directory, 'tokenizer.json'), 'utf8').then(value => JSON.parse(value) as object), + readFile(join(directory, 'tokenizer_config.json'), 'utf8').then(value => JSON.parse(value) as object), + readFile(join(directory, 'onnx/model_quantized.onnx')), + ]); + ort.env.wasm.numThreads = 1; + const tokenizer = new Tokenizer(tokenizerJson, tokenizerConfig); + const session = await ort.InferenceSession.create(modelBytes, { executionProviders: ['wasm'] }); + + const embed = async (text: string): Promise => { + const encoded = tokenizer.encode(text); + const inputIds = BigInt64Array.from(encoded.ids, BigInt); + const attentionMask = BigInt64Array.from(encoded.attention_mask, BigInt); + const tokenTypeIds = new BigInt64Array(inputIds.length); + const outputs = await session.run({ + input_ids: new ort.Tensor('int64', inputIds, [1, inputIds.length]), + attention_mask: new ort.Tensor('int64', attentionMask, [1, attentionMask.length]), + token_type_ids: new ort.Tensor('int64', tokenTypeIds, [1, tokenTypeIds.length]), + }); + const hidden = outputs.last_hidden_state; + if (!hidden || !(hidden.data instanceof Float32Array)) { + throw new Error('Semantic model did not return last_hidden_state'); + } + return meanPoolAndNormalize(hidden.data, hidden.dims, encoded.attention_mask); + }; + + return { + embed, + embedMany: async texts => { + const results: Float32Array[] = []; + for (const text of texts) results.push(await embed(text)); + return results; + }, + dispose: async () => session.release(), + }; +} diff --git a/packages/memory/src/services/model.ts b/packages/memory/src/services/model.ts new file mode 100644 index 00000000..bd9428ea --- /dev/null +++ b/packages/memory/src/services/model.ts @@ -0,0 +1,118 @@ +import { createHash } from 'crypto'; +import { homedir } from 'os'; +import { dirname, join } from 'path'; +import { mkdir, readFile, rename, rm, writeFile } from 'fs/promises'; + +export const MODEL_ID = 'Xenova/all-MiniLM-L6-v2'; +export const MODEL_REVISION = '751bff37182d3f1213fa05d7196b954e230abad9'; +export const MODEL_DIMENSION = 384; +export const MODEL_VERSION = `${MODEL_ID}@${MODEL_REVISION}:q8:${MODEL_DIMENSION}`; + +export interface ModelFile { + path: string; + sha256: string; + size: number; +} + +export const MODEL_FILES: readonly ModelFile[] = [ + { path: 'config.json', sha256: '7135149f7cffa1a573466c6e4d8423ed73b62fd2332c575bf738a0d033f70df7', size: 650 }, + { path: 'tokenizer.json', sha256: 'da0e79933b9ed51798a3ae27893d3c5fa4a201126cef75586296df9b4d2c62a0', size: 711_661 }, + { path: 'tokenizer_config.json', sha256: '9261e7d79b44c8195c1cada2b453e55b00aeb81e907a6664974b4d7776172ab3', size: 366 }, + { path: 'onnx/model_quantized.onnx', sha256: 'afdb6f1a0e45b715d0bb9b11772f032c399babd23bfc31fed1c170afc848bdb1', size: 22_972_370 }, +]; + +export interface ModelInspection { + ready: boolean; + missing: string[]; + corrupt: string[]; +} + +export function getModelDirectory(modelsRoot = join(homedir(), '.ai-devkit', 'models')): string { + return join(modelsRoot, MODEL_ID.replace('/', '--'), MODEL_REVISION); +} + +async function digestFile(path: string): Promise<{ sha256: string; size: number } | null> { + try { + const bytes = await readFile(path); + return { + sha256: createHash('sha256').update(bytes).digest('hex'), + size: bytes.byteLength, + }; + } catch (error) { + if ((error as NodeJS.ErrnoException).code === 'ENOENT') { + return null; + } + throw error; + } +} + +export async function inspectModelFiles( + directory: string, + manifest: readonly ModelFile[] = MODEL_FILES, +): Promise { + const missing: string[] = []; + const corrupt: string[] = []; + for (const file of manifest) { + const digest = await digestFile(join(directory, file.path)); + if (!digest) { + missing.push(file.path); + } else if (digest.sha256 !== file.sha256 || digest.size !== file.size) { + corrupt.push(file.path); + } + } + return { ready: missing.length === 0 && corrupt.length === 0, missing, corrupt }; +} + +interface EnsureModelFilesOptions { + directory?: string; + manifest?: readonly ModelFile[]; + fetchImpl?: typeof fetch; +} + +export async function ensureModelFiles(options: EnsureModelFilesOptions = {}): Promise { + const directory = options.directory ?? getModelDirectory(); + const manifest = options.manifest ?? MODEL_FILES; + const fetchImpl = options.fetchImpl ?? fetch; + const inspection = await inspectModelFiles(directory, manifest); + const needed = new Set([...inspection.missing, ...inspection.corrupt]); + + for (const file of manifest) { + if (!needed.has(file.path)) continue; + const url = `https://huggingface.co/${MODEL_ID}/resolve/${MODEL_REVISION}/${file.path}?download=true`; + const response = await fetchImpl(url); + if (!response.ok) { + throw new Error(`Failed to download semantic model file ${file.path}: HTTP ${response.status}`); + } + const bytes = Buffer.from(await response.arrayBuffer()); + const sha256 = createHash('sha256').update(bytes).digest('hex'); + if (bytes.byteLength !== file.size || sha256 !== file.sha256) { + throw new Error(`Semantic model checksum mismatch for ${file.path}`); + } + + const destination = join(directory, file.path); + const temporary = `${destination}.tmp-${process.pid}`; + await mkdir(dirname(destination), { recursive: true }); + try { + await writeFile(temporary, bytes, { mode: 0o600 }); + await rename(temporary, destination); + } catch (error) { + await rm(temporary, { force: true }); + throw error; + } + } + return directory; +} + +export function normalizeEmbedding(vector: Float32Array): Float32Array { + let squaredNorm = 0; + for (const value of vector) squaredNorm += value * value; + const norm = Math.sqrt(squaredNorm); + if (!Number.isFinite(norm) || norm === 0) { + throw new Error('Semantic model returned a zero or invalid embedding'); + } + const normalized = new Float32Array(vector.length); + for (let index = 0; index < vector.length; index++) { + normalized[index] = vector[index]! / norm; + } + return normalized; +} diff --git a/packages/memory/tests/unit/model.test.ts b/packages/memory/tests/unit/model.test.ts new file mode 100644 index 00000000..a886ad3a --- /dev/null +++ b/packages/memory/tests/unit/model.test.ts @@ -0,0 +1,67 @@ +import { createHash } from 'crypto'; +import { join } from 'path'; +import { tmpdir } from 'os'; +import { mkdtempSync, readFileSync, rmSync } from 'fs'; +import { + MODEL_DIMENSION, + MODEL_ID, + MODEL_REVISION, + MODEL_VERSION, + ensureModelFiles, + inspectModelFiles, + normalizeEmbedding, +} from '../../src/services/model'; +import { meanPoolAndNormalize } from '../../src/services/embedder'; + +describe('semantic model files', () => { + const directory = mkdtempSync(join(tmpdir(), 'memory-model-')); + const bytes = Buffer.from('pinned model fixture'); + const manifest = [{ + path: 'nested/model.onnx', + sha256: createHash('sha256').update(bytes).digest('hex'), + size: bytes.length, + }]; + + afterAll(() => rmSync(directory, { recursive: true, force: true })); + + it('pins model identity, revision, quantization, and dimension', () => { + expect(MODEL_ID).toBe('Xenova/all-MiniLM-L6-v2'); + expect(MODEL_REVISION).toMatch(/^[a-f0-9]{40}$/); + expect(MODEL_DIMENSION).toBe(384); + expect(MODEL_VERSION).toContain(':q8:384'); + }); + + it('downloads checked files atomically and recognizes the cache', async () => { + const fetchImpl = vi.fn(async () => new Response(bytes)); + + expect(await inspectModelFiles(directory, manifest)).toEqual({ ready: false, missing: ['nested/model.onnx'], corrupt: [] }); + await ensureModelFiles({ directory, manifest, fetchImpl }); + + expect(fetchImpl).toHaveBeenCalledOnce(); + expect(readFileSync(join(directory, 'nested/model.onnx'))).toEqual(bytes); + expect(await inspectModelFiles(directory, manifest)).toEqual({ ready: true, missing: [], corrupt: [] }); + }); + + it('rejects a download that does not match the pinned checksum', async () => { + await expect(ensureModelFiles({ + directory: join(directory, 'bad'), + manifest, + fetchImpl: async () => new Response('wrong'), + })).rejects.toThrow('checksum'); + }); + + it('normalizes embeddings and rejects invalid output', () => { + expect([...normalizeEmbedding(new Float32Array([3, 4]))]).toEqual([0.6000000238418579, 0.800000011920929]); + expect(() => normalizeEmbedding(new Float32Array([0, 0]))).toThrow('zero'); + }); + + it('mean-pools only unmasked token vectors', () => { + const pooled = meanPoolAndNormalize( + new Float32Array([1, 0, 0, 1, 10, 10]), + [1, 3, 2], + [1, 1, 0], + ); + expect(pooled[0]).toBeCloseTo(Math.SQRT1_2); + expect(pooled[1]).toBeCloseTo(Math.SQRT1_2); + }); +}); From 7f529bcc13cf6aa5d5b3f0e0d8998b48da41592a Mon Sep 17 00:00:00 2001 From: Hoang Nguyen Date: Tue, 1 Sep 2026 06:20:30 +0000 Subject: [PATCH 03/11] feat(memory): wire opt-in hybrid semantic search --- docs/ai/2026-09-01-feature-memory-semantic.md | 32 ++-- .../cli/src/__tests__/commands/memory.test.ts | 66 +++++++- packages/cli/src/__tests__/lib/Config.test.ts | 15 ++ packages/cli/src/commands/memory.ts | 78 +++++++-- packages/cli/src/lib/Config.ts | 6 + packages/cli/src/types.ts | 2 + packages/memory/src/api.ts | 95 ++++++++++- .../src/handlers/semantic-maintenance.ts | 148 ++++++++++++++++++ .../memory/src/handlers/semantic-search.ts | 117 ++++++++++++++ packages/memory/src/handlers/store.ts | 20 ++- packages/memory/src/handlers/update.ts | 15 +- packages/memory/src/server.ts | 20 ++- packages/memory/src/services/config.ts | 17 ++ packages/memory/src/types/index.ts | 12 ++ .../tests/integration/semantic-search.test.ts | 117 ++++++++++++++ packages/memory/tests/unit/config.test.ts | 22 +++ 16 files changed, 741 insertions(+), 41 deletions(-) create mode 100644 packages/memory/src/handlers/semantic-maintenance.ts create mode 100644 packages/memory/src/handlers/semantic-search.ts create mode 100644 packages/memory/src/services/config.ts create mode 100644 packages/memory/tests/integration/semantic-search.test.ts create mode 100644 packages/memory/tests/unit/config.test.ts diff --git a/docs/ai/2026-09-01-feature-memory-semantic.md b/docs/ai/2026-09-01-feature-memory-semantic.md index b12a9c0e..02aeb12a 100644 --- a/docs/ai/2026-09-01-feature-memory-semantic.md +++ b/docs/ai/2026-09-01-feature-memory-semantic.md @@ -49,28 +49,28 @@ flowchart LR ## Implementation plan -- [ ] Add migration and embedding invalidation/storage tests. -- [ ] Add deterministic semantic primitives and RRF tests. -- [ ] Add pinned model downloader and lightweight ONNX runtime tests. -- [ ] Add status, download, and resumable re-embed APIs and CLI tests. -- [ ] Add async hybrid search with offline/corrupt/stale/large-corpus degradation tests. -- [ ] Wire config through CLI and MCP while preserving default behavior. +- [x] Add migration and embedding invalidation/storage tests. +- [x] Add deterministic semantic primitives and RRF tests. +- [x] Add pinned model downloader and lightweight ONNX runtime tests. +- [x] Add status, download, and resumable re-embed APIs and CLI tests. +- [x] Add async hybrid search with offline/corrupt/stale degradation tests; large-corpus validation remains in the performance gate. +- [x] Wire config through CLI and MCP while preserving default behavior. - [ ] Run expanded-100 gate through the built CLI. - [ ] Run 1,000-row warm latency gate. - [ ] Run six-project build, full tests, lint, and e2e. ## Testing record -- [ ] Migration adds nullable columns without changing existing rows. -- [ ] Content fields invalidate embeddings; scope-only updates do not. -- [ ] Model file checksums and pinned revision are enforced. -- [ ] Runtime output is normalized and 384-dimensional. -- [ ] RRF is deterministic and protects lexical ties. -- [ ] Offline and missing-model search returns lexical results. -- [ ] Stale/corrupt embeddings are excluded safely. -- [ ] Backfill resumes, skips current rows, and force-rebuilds. -- [ ] CLI surfaces config, status, download, reembed, and explanations. -- [ ] Default lexical API and strict/broad strategies remain compatible. +- [x] Migration adds nullable columns without changing existing rows. +- [x] Content fields invalidate embeddings; scope-only updates do not. +- [x] Model file checksums and pinned revision are enforced. +- [x] Runtime output is normalized and 384-dimensional. +- [x] RRF is deterministic and protects lexical ties. +- [x] Offline and missing-model search returns lexical results. +- [x] Stale/corrupt embeddings are excluded safely. +- [x] Backfill resumes and skips current rows; force mode remains in final CLI validation. +- [x] CLI surfaces config, status, download, reembed, and explanations. +- [x] Default lexical API and strict/broad strategies remain compatible in targeted tests. ## Rollback diff --git a/packages/cli/src/__tests__/commands/memory.test.ts b/packages/cli/src/__tests__/commands/memory.test.ts index 868bf84b..29ca00a0 100644 --- a/packages/cli/src/__tests__/commands/memory.test.ts +++ b/packages/cli/src/__tests__/commands/memory.test.ts @@ -2,18 +2,36 @@ import type { MockedFunction } from 'vitest'; import { Command } from 'commander'; import { registerMemoryCommand } from '../../commands/memory.js'; -import { memorySearchCommand, memoryStoreCommand, memoryUpdateCommand } from '@ai-devkit/memory'; +import { + memoryDownloadSemanticCommand, + memoryReembedCommand, + memorySearchCommand, + memorySearchCommandAsync, + memorySemanticStatusCommand, + memoryStoreCommand, + memoryStoreCommandAsync, + memoryUpdateCommand, + memoryUpdateCommandAsync, +} from '@ai-devkit/memory'; import { ui } from '../../util/terminal-ui.js'; const mockGetMemoryDbPath = vi.fn<() => Promise>(); +const mockGetMemorySemanticEnabled = vi.fn<() => Promise>(); const mockConfigManager = { - getMemoryDbPath: mockGetMemoryDbPath + getMemoryDbPath: mockGetMemoryDbPath, + getMemorySemanticEnabled: mockGetMemorySemanticEnabled, }; vi.mock('@ai-devkit/memory', () => ({ memoryStoreCommand: vi.fn(), + memoryStoreCommandAsync: vi.fn(), memorySearchCommand: vi.fn(), - memoryUpdateCommand: vi.fn() + memorySearchCommandAsync: vi.fn(), + memoryUpdateCommand: vi.fn(), + memoryUpdateCommandAsync: vi.fn(), + memorySemanticStatusCommand: vi.fn(), + memoryDownloadSemanticCommand: vi.fn(), + memoryReembedCommand: vi.fn(), }), { virtual: true }); vi.mock('../../lib/Config.js', () => ({ @@ -38,6 +56,7 @@ describe('memory command', () => { beforeEach(() => { vi.clearAllMocks(); mockGetMemoryDbPath.mockResolvedValue(undefined); + mockGetMemorySemanticEnabled.mockResolvedValue(false); consoleLogSpy = vi.spyOn(console, 'log').mockImplementation(() => undefined); }); @@ -315,4 +334,45 @@ describe('memory command', () => { expect(mockedUi.error).toHaveBeenCalledWith('Failed to search knowledge: search failed'); expect(exitSpy).toHaveBeenCalledWith(1); }); + + it('uses hybrid search and explanations when semantic config is enabled', async () => { + mockGetMemorySemanticEnabled.mockResolvedValue(true); + vi.mocked(memorySearchCommandAsync).mockResolvedValue({ + results: [], totalMatches: 0, query: 'wire contracts', strategy: 'broad', + retrievalMode: 'hybrid', + semantic: { status: 'ready', embeddingVersion: 'model', eligibleCount: 2 }, + }); + + const program = new Command(); + registerMemoryCommand(program); + await program.parseAsync(['node', 'test', 'memory', 'search', '--query', 'wire contracts', '--explain']); + + expect(memorySearchCommandAsync).toHaveBeenCalledWith(expect.objectContaining({ + query: 'wire contracts', semantic: true, explain: true, + })); + expect(mockedMemorySearchCommand).not.toHaveBeenCalled(); + }); + + it('exposes semantic status, download, and force reembed commands', async () => { + vi.mocked(memorySemanticStatusCommand).mockResolvedValue({ + modelReady: false, modelDirectory: '/models', embeddingVersion: 'model', total: 0, current: 0, missing: 0, stale: 0, + }); + vi.mocked(memoryDownloadSemanticCommand).mockResolvedValue({ + modelReady: true, modelDirectory: '/models', embeddingVersion: 'model', total: 0, current: 0, missing: 0, stale: 0, + }); + vi.mocked(memoryReembedCommand).mockResolvedValue({ total: 0, embedded: 0, skipped: 0, failed: 0, embeddingVersion: 'model' }); + + for (const args of [ + ['memory', 'semantic', 'status'], + ['memory', 'semantic', 'download'], + ['memory', 'reembed', '--force'], + ]) { + const program = new Command(); registerMemoryCommand(program); + await program.parseAsync(['node', 'test', ...args]); + } + + expect(memorySemanticStatusCommand).toHaveBeenCalledWith({ dbPath: undefined }); + expect(memoryDownloadSemanticCommand).toHaveBeenCalledWith({ dbPath: undefined }); + expect(memoryReembedCommand).toHaveBeenCalledWith({ dbPath: undefined, force: true }); + }); }); diff --git a/packages/cli/src/__tests__/lib/Config.test.ts b/packages/cli/src/__tests__/lib/Config.test.ts index bcc44f84..d7af94be 100644 --- a/packages/cli/src/__tests__/lib/Config.test.ts +++ b/packages/cli/src/__tests__/lib/Config.test.ts @@ -121,6 +121,7 @@ describe('ConfigManager', () => { version: '1.0.0', environments: [], phases: [], + memory: { semantic: false }, createdAt: expect.any(String), }; @@ -897,4 +898,18 @@ describe('ConfigManager', () => { }); }); + describe('getMemorySemanticEnabled', () => { + it('defaults to false and accepts only explicit true', async () => { + (mockFs.pathExists as any).mockResolvedValue(false); + await expect(configManager.getMemorySemanticEnabled()).resolves.toBe(false); + + (mockFs.pathExists as any).mockResolvedValue(true); + (mockFs.readJson as any).mockResolvedValue({ memory: { semantic: true } }); + await expect(configManager.getMemorySemanticEnabled()).resolves.toBe(true); + + (mockFs.readJson as any).mockResolvedValue({ memory: { semantic: 'true' } }); + await expect(configManager.getMemorySemanticEnabled()).resolves.toBe(false); + }); + }); + }); diff --git a/packages/cli/src/commands/memory.ts b/packages/cli/src/commands/memory.ts index e8342266..9e1624d0 100644 --- a/packages/cli/src/commands/memory.ts +++ b/packages/cli/src/commands/memory.ts @@ -1,5 +1,15 @@ import type { Command } from 'commander'; -import { memoryStoreCommand, memorySearchCommand, memoryUpdateCommand } from '@ai-devkit/memory'; +import { + memoryDownloadSemanticCommand, + memoryReembedCommand, + memorySearchCommand, + memorySearchCommandAsync, + memorySemanticStatusCommand, + memoryStoreCommand, + memoryStoreCommandAsync, + memoryUpdateCommand, + memoryUpdateCommandAsync, +} from '@ai-devkit/memory'; import type { MemorySearchOptions, MemoryStoreOptions, MemoryUpdateOptions } from '@ai-devkit/memory'; import { ConfigManager } from '../lib/Config.js'; import { ui } from '../util/terminal-ui.js'; @@ -14,6 +24,11 @@ export function registerMemoryCommand(program: Command): void { return configManager.getMemoryDbPath(); }; + const resolveSemanticEnabled = async (): Promise => { + const configManager = new ConfigManager(); + return configManager.getMemorySemanticEnabled(); + }; + const memoryCommand = program .command('memory') .description('Interact with the knowledge memory service'); @@ -26,10 +41,13 @@ export function registerMemoryCommand(program: Command): void { .option('--tags ', 'Comma-separated tags (e.g., "api,backend")') .option('-s, --scope ', 'Scope: global, project:, or repo:', 'global') .action(withErrorHandler('store knowledge', async (options: MemoryStoreOptions) => { - const result = memoryStoreCommand({ + const commandOptions = { ...options, dbPath: await resolveMemoryDbPath() - } as MemoryStoreOptions); + } as MemoryStoreOptions; + const result = await resolveSemanticEnabled() + ? memoryStoreCommandAsync({ ...commandOptions, semantic: true }) + : memoryStoreCommand(commandOptions); console.log(JSON.stringify(result, null, 2)); })); @@ -42,10 +60,13 @@ export function registerMemoryCommand(program: Command): void { .option('--tags ', 'Comma-separated new tags (replaces existing)') .option('-s, --scope ', 'New scope: global, project:, or repo:') .action(withErrorHandler('update knowledge', async (options: MemoryUpdateOptions) => { - const result = memoryUpdateCommand({ + const commandOptions = { ...options, dbPath: await resolveMemoryDbPath() - } as MemoryUpdateOptions); + } as MemoryUpdateOptions; + const result = await resolveSemanticEnabled() + ? memoryUpdateCommandAsync({ ...commandOptions, semantic: true }) + : memoryUpdateCommand(commandOptions); console.log(JSON.stringify(result, null, 2)); })); @@ -57,13 +78,18 @@ export function registerMemoryCommand(program: Command): void { .option('-s, --scope ', 'Scope filter') .option('-l, --limit ', 'Maximum results (1-20)', '5') .option('--table', 'Display results as a table with id, title, and scope') - .action(withErrorHandler('search knowledge', async (options: MemorySearchOptions & { limit?: string; table?: boolean }) => { - const { table, limit, ...searchOptions } = options; - const result = memorySearchCommand({ + .option('--explain', 'Include lexical and semantic rank details') + .action(withErrorHandler('search knowledge', async (options: MemorySearchOptions & { limit?: string; table?: boolean; explain?: boolean }) => { + const { table, limit, explain, ...searchOptions } = options; + const commandOptions = { ...searchOptions, limit: limit ? parseInt(limit, 10) : 5, - dbPath: await resolveMemoryDbPath() - } as MemorySearchOptions); + dbPath: await resolveMemoryDbPath(), + ...(explain ? { explain: true } : {}), + } as MemorySearchOptions; + const result = await resolveSemanticEnabled() + ? await memorySearchCommandAsync({ ...commandOptions, semantic: true }) + : memorySearchCommand(commandOptions); if (table) { if (result.results.length === 0) { @@ -84,4 +110,36 @@ export function registerMemoryCommand(program: Command): void { console.log(JSON.stringify(result, null, 2)); })); + + const semanticCommand = memoryCommand + .command('semantic') + .description('Manage the optional local semantic model'); + + semanticCommand + .command('status') + .description('Show semantic model and embedding readiness') + .action(withErrorHandler('show semantic status', async () => { + const result = await memorySemanticStatusCommand({ dbPath: await resolveMemoryDbPath() }); + console.log(JSON.stringify(result, null, 2)); + })); + + semanticCommand + .command('download') + .description('Download and verify the pinned local semantic model') + .action(withErrorHandler('download semantic model', async () => { + const result = await memoryDownloadSemanticCommand({ dbPath: await resolveMemoryDbPath() }); + console.log(JSON.stringify(result, null, 2)); + })); + + memoryCommand + .command('reembed') + .description('Backfill missing or stale semantic embeddings') + .option('--force', 'Recompute all embeddings') + .action(withErrorHandler('re-embed memory', async (options: { force?: boolean }) => { + const result = await memoryReembedCommand({ + dbPath: await resolveMemoryDbPath(), + force: options.force === true, + }); + console.log(JSON.stringify(result, null, 2)); + })); } diff --git a/packages/cli/src/lib/Config.ts b/packages/cli/src/lib/Config.ts index 6ef7db91..90303fef 100644 --- a/packages/cli/src/lib/Config.ts +++ b/packages/cli/src/lib/Config.ts @@ -35,6 +35,7 @@ export class ConfigManager { version: packageJson.version, environments: [], phases: [], + memory: { semantic: false }, createdAt: new Date().toISOString() }; @@ -101,6 +102,11 @@ export class ConfigManager { return this.resolveConfiguredPath(config?.memory?.path); } + async getMemorySemanticEnabled(): Promise { + const config = await this.read(); + return config?.memory?.semantic === true; + } + private resolveConfiguredPath(configuredPath: unknown): string | undefined { if (typeof configuredPath !== 'string') { return undefined; diff --git a/packages/cli/src/types.ts b/packages/cli/src/types.ts index f0261013..49b2a23b 100644 --- a/packages/cli/src/types.ts +++ b/packages/cli/src/types.ts @@ -27,6 +27,7 @@ export interface DevKitConfig { }; memory?: { path?: string; + semantic?: boolean; }; environments: EnvironmentCode[]; phases: Phase[]; @@ -59,6 +60,7 @@ export interface GlobalDevKitConfig { plugins?: string[]; memory?: { path?: string; + semantic?: boolean; }; } diff --git a/packages/memory/src/api.ts b/packages/memory/src/api.ts index 5bd54121..a454715a 100644 --- a/packages/memory/src/api.ts +++ b/packages/memory/src/api.ts @@ -3,10 +3,19 @@ import { searchKnowledge } from './handlers/search.js'; import { updateKnowledge } from './handlers/update.js'; import { listKnowledge } from './handlers/list.js'; import { getKnowledgeSummary } from './handlers/summary.js'; +import { searchKnowledgeHybrid } from './handlers/semantic-search.js'; +import { + downloadSemanticModel, + getSemanticStatus, + reembedKnowledge, + storeKnowledgeSemantic, + updateKnowledgeSemantic, +} from './handlers/semantic-maintenance.js'; import { closeDatabase, getDatabase } from './database/index.js'; import type { StoreKnowledgeInput, SearchKnowledgeInput, StoreKnowledgeResult, SearchKnowledgeResult, UpdateKnowledgeInput, UpdateKnowledgeResult, ListKnowledgeInput, ListKnowledgeResult, ListKnowledgeSort, KnowledgeSummaryResult, KnowledgeItem } from './types/index.js'; +import type { ReembedResult, SemanticStatusResult } from './handlers/semantic-maintenance.js'; -export { storeKnowledge, searchKnowledge, updateKnowledge, listKnowledge, getKnowledgeSummary }; +export { storeKnowledge, searchKnowledge, searchKnowledgeHybrid, updateKnowledge, listKnowledge, getKnowledgeSummary }; export type { StoreKnowledgeInput, SearchKnowledgeInput, StoreKnowledgeResult, SearchKnowledgeResult, UpdateKnowledgeInput, UpdateKnowledgeResult, ListKnowledgeInput, ListKnowledgeResult, ListKnowledgeSort, KnowledgeSummaryResult, KnowledgeItem }; // CLI command handlers for integration with main ai-devkit CLI @@ -33,6 +42,8 @@ export interface MemorySearchOptions { scope?: string; limit?: number; dbPath?: string; + semantic?: boolean; + explain?: boolean; } export interface MemoryListOptions { @@ -49,6 +60,14 @@ export interface MemorySummaryOptions { dbPath?: string; } +export interface MemorySemanticOptions { + dbPath?: string; +} + +export interface MemoryReembedOptions extends MemorySemanticOptions { + force?: boolean; +} + export function memoryStoreCommand(options: MemoryStoreOptions): StoreKnowledgeResult { try { getDatabase({ dbPath: options.dbPath }); @@ -98,6 +117,80 @@ export function memorySearchCommand(options: MemorySearchOptions): SearchKnowled } } +export async function memorySearchCommandAsync(options: MemorySearchOptions): Promise { + try { + getDatabase({ dbPath: options.dbPath }); + const input: SearchKnowledgeInput = { + query: options.query, + contextTags: options.tags ? options.tags.split(',').map(t => t.trim()) : undefined, + scope: options.scope, + limit: options.limit, + explain: options.explain, + }; + return options.semantic ? await searchKnowledgeHybrid(input) : searchKnowledge(input); + } finally { + closeDatabase(); + } +} + +export async function memoryStoreCommandAsync(options: MemoryStoreOptions & { semantic?: boolean }): Promise { + try { + getDatabase({ dbPath: options.dbPath }); + const input: StoreKnowledgeInput = { + title: options.title, + content: options.content, + tags: options.tags ? options.tags.split(',').map(t => t.trim()) : undefined, + scope: options.scope, + }; + return options.semantic ? await storeKnowledgeSemantic(input) : storeKnowledge(input); + } finally { + closeDatabase(); + } +} + +export async function memoryUpdateCommandAsync(options: MemoryUpdateOptions & { semantic?: boolean }): Promise { + try { + getDatabase({ dbPath: options.dbPath }); + const input: UpdateKnowledgeInput = { + id: options.id, + title: options.title, + content: options.content, + tags: options.tags ? options.tags.split(',').map(t => t.trim()) : undefined, + scope: options.scope, + }; + return options.semantic ? await updateKnowledgeSemantic(input) : updateKnowledge(input); + } finally { + closeDatabase(); + } +} + +export async function memorySemanticStatusCommand(options: MemorySemanticOptions = {}): Promise { + try { + getDatabase({ dbPath: options.dbPath }); + return await getSemanticStatus(); + } finally { + closeDatabase(); + } +} + +export async function memoryDownloadSemanticCommand(options: MemorySemanticOptions = {}): Promise { + try { + getDatabase({ dbPath: options.dbPath }); + return await downloadSemanticModel(); + } finally { + closeDatabase(); + } +} + +export async function memoryReembedCommand(options: MemoryReembedOptions = {}): Promise { + try { + getDatabase({ dbPath: options.dbPath }); + return await reembedKnowledge({ force: options.force }); + } finally { + closeDatabase(); + } +} + export function memoryListCommand(options: MemoryListOptions = {}): ListKnowledgeResult { try { getDatabase({ dbPath: options.dbPath }); diff --git a/packages/memory/src/handlers/semantic-maintenance.ts b/packages/memory/src/handlers/semantic-maintenance.ts new file mode 100644 index 00000000..70cf43a5 --- /dev/null +++ b/packages/memory/src/handlers/semantic-maintenance.ts @@ -0,0 +1,148 @@ +import { getDatabase } from '../database/index.js'; +import { normalizeTags } from '../services/normalizer.js'; +import { loadLocalEmbedder, type LocalEmbedder } from '../services/embedder.js'; +import { ensureModelFiles, getModelDirectory, inspectModelFiles, MODEL_VERSION } from '../services/model.js'; +import { buildEmbeddingText, serializeEmbedding } from '../services/semantic.js'; +import { storeKnowledge } from './store.js'; +import { updateKnowledge } from './update.js'; +import type { KnowledgeRow, StoreKnowledgeInput, StoreKnowledgeResult, UpdateKnowledgeInput, UpdateKnowledgeResult } from '../types/index.js'; + +interface EmbedderOption { + embedder?: LocalEmbedder; + modelsRoot?: string; +} + +export interface SemanticStatusResult { + modelReady: boolean; + modelDirectory: string; + embeddingVersion: string; + total: number; + current: number; + missing: number; + stale: number; +} + +export interface ReembedResult { + total: number; + embedded: number; + skipped: number; + failed: number; + embeddingVersion: string; +} + +export async function getSemanticStatus(options: { modelsRoot?: string } = {}): Promise { + const db = getDatabase(); + const directory = getModelDirectory(options.modelsRoot); + const model = await inspectModelFiles(directory); + const counts = db.queryOne<{ total: number; current: number; missing: number; stale: number }>(` + SELECT + COUNT(*) AS total, + COALESCE(SUM(CASE WHEN embedding IS NOT NULL AND embedding_version = ? THEN 1 ELSE 0 END), 0) AS current, + COALESCE(SUM(CASE WHEN embedding IS NULL THEN 1 ELSE 0 END), 0) AS missing, + COALESCE(SUM(CASE WHEN embedding IS NOT NULL AND (embedding_version IS NULL OR embedding_version != ?) THEN 1 ELSE 0 END), 0) AS stale + FROM knowledge + `, [MODEL_VERSION, MODEL_VERSION]) ?? { total: 0, current: 0, missing: 0, stale: 0 }; + return { + modelReady: model.ready, + modelDirectory: directory, + embeddingVersion: MODEL_VERSION, + total: counts.total, + current: counts.current, + missing: counts.missing, + stale: counts.stale, + }; +} + +export async function downloadSemanticModel(options: { modelsRoot?: string } = {}): Promise { + await ensureModelFiles({ directory: getModelDirectory(options.modelsRoot) }); + return getSemanticStatus(options); +} + +export async function storeKnowledgeSemantic( + input: StoreKnowledgeInput, + options: EmbedderOption = {}, +): Promise { + let value: Buffer; + try { + const embedder = options.embedder ?? await loadLocalEmbedder({ modelsRoot: options.modelsRoot, download: true }); + const tags = normalizeTags(input.tags ?? []); + const vector = await embedder.embed(buildEmbeddingText({ title: input.title, content: input.content, tags })); + value = serializeEmbedding(vector); + } catch { + return storeKnowledge(input); + } + return storeKnowledge(input, { value, version: MODEL_VERSION }); +} + +export async function updateKnowledgeSemantic( + input: UpdateKnowledgeInput, + options: EmbedderOption = {}, +): Promise { + if (input.title === undefined && input.content === undefined && input.tags === undefined) { + return updateKnowledge(input); + } + const existing = getDatabase().queryOne('SELECT * FROM knowledge WHERE id = ?', [input.id]); + if (!existing) return updateKnowledge(input); + let value: Buffer; + try { + const embedder = options.embedder ?? await loadLocalEmbedder({ modelsRoot: options.modelsRoot, download: true }); + const vector = await embedder.embed(buildEmbeddingText({ + title: input.title ?? existing.title, + content: input.content ?? existing.content, + tags: input.tags !== undefined ? normalizeTags(input.tags) : JSON.parse(existing.tags) as string[], + })); + value = serializeEmbedding(vector); + } catch { + return updateKnowledge(input); + } + return updateKnowledge(input, { value, version: MODEL_VERSION }); +} + +interface ReembedOptions extends EmbedderOption { + force?: boolean; + batchSize?: number; +} + +export async function reembedKnowledge(options: ReembedOptions = {}): Promise { + const db = getDatabase(); + const all = db.query('SELECT * FROM knowledge ORDER BY id'); + const pending = options.force + ? all + : all.filter(row => !row.embedding || row.embedding_version !== MODEL_VERSION); + const embedder = options.embedder ?? await loadLocalEmbedder({ modelsRoot: options.modelsRoot, download: true }); + const batchSize = Math.max(1, options.batchSize ?? 32); + let embedded = 0; + let failed = 0; + + for (let offset = 0; offset < pending.length; offset += batchSize) { + const batch = pending.slice(offset, offset + batchSize); + try { + const texts = batch.map(row => buildEmbeddingText({ + title: row.title, + content: row.content, + tags: JSON.parse(row.tags) as string[], + })); + const vectors = await embedder.embedMany(texts); + if (vectors.length !== batch.length) throw new Error('Semantic model returned the wrong batch size'); + db.transaction(() => { + batch.forEach((row, index) => { + db.execute( + 'UPDATE knowledge SET embedding = ?, embedding_version = ? WHERE id = ?', + [serializeEmbedding(vectors[index]!), MODEL_VERSION, row.id], + ); + }); + }); + embedded += batch.length; + } catch { + failed += batch.length; + } + } + + return { + total: all.length, + embedded, + skipped: all.length - pending.length, + failed, + embeddingVersion: MODEL_VERSION, + }; +} diff --git a/packages/memory/src/handlers/semantic-search.ts b/packages/memory/src/handlers/semantic-search.ts new file mode 100644 index 00000000..e92194b6 --- /dev/null +++ b/packages/memory/src/handlers/semantic-search.ts @@ -0,0 +1,117 @@ +import { getDatabase } from '../database/index.js'; +import { loadLocalEmbedder, type LocalEmbedder } from '../services/embedder.js'; +import { MODEL_DIMENSION, MODEL_VERSION } from '../services/model.js'; +import { cosineSimilarity, deserializeEmbedding, fuseSearchResults, type SemanticCandidate } from '../services/semantic.js'; +import { searchKnowledge } from './search.js'; +import type { SearchKnowledgeInput, SearchKnowledgeResult } from '../types/index.js'; + +const MAX_SEMANTIC_CORPUS = 5_000; +const MAX_CANDIDATES = 20; +let defaultEmbedder: Promise | undefined; + +interface RawSemanticRow { + id: string; + title: string; + content: string; + tags: string; + scope: string; + embedding: Buffer; +} + +interface HybridSearchOptions { + embedder?: LocalEmbedder; +} + +function semanticStatus( + status: 'ready' | 'unavailable' | 'corpus-too-large', + eligibleCount: number, + reason?: string, +) { + return { + status, + embeddingVersion: MODEL_VERSION, + eligibleCount, + ...(reason ? { reason } : {}), + } as const; +} + +function getDefaultEmbedder(): Promise { + defaultEmbedder ??= loadLocalEmbedder({ download: true }).catch(error => { + defaultEmbedder = undefined; + throw error; + }); + return defaultEmbedder; +} + +export async function searchKnowledgeHybrid( + input: SearchKnowledgeInput, + options: HybridSearchOptions = {}, +): Promise { + const requestedLimit = Math.min(Math.max(input.limit ?? 5, 1), MAX_CANDIDATES); + const lexical = searchKnowledge({ ...input, limit: MAX_CANDIDATES, explain: false }); + const db = getDatabase(); + const scopeClause = input.scope ? ` AND (scope = ? OR scope = 'global')` : ''; + const scopeParams = input.scope ? [input.scope] : []; + const count = db.queryOne<{ count: number }>( + `SELECT COUNT(*) AS count FROM knowledge WHERE embedding_version = ? AND embedding IS NOT NULL${scopeClause}`, + [MODEL_VERSION, ...scopeParams], + )?.count ?? 0; + + if (count > MAX_SEMANTIC_CORPUS) { + return { + ...lexical, + results: lexical.results.slice(0, requestedLimit), + retrievalMode: 'lexical', + semantic: semanticStatus('corpus-too-large', count, `Semantic scan is limited to ${MAX_SEMANTIC_CORPUS} rows`), + }; + } + + try { + const embedder = options.embedder ?? await getDefaultEmbedder(); + const queryEmbedding = await embedder.embed(input.query.trim()); + if (queryEmbedding.length !== MODEL_DIMENSION) { + throw new Error(`Semantic model returned ${queryEmbedding.length} dimensions; expected ${MODEL_DIMENSION}`); + } + const rows = db.query( + `SELECT id, title, content, tags, scope, embedding + FROM knowledge + WHERE embedding_version = ? AND embedding IS NOT NULL${scopeClause}`, + [MODEL_VERSION, ...scopeParams], + ); + const candidates: SemanticCandidate[] = []; + for (const row of rows) { + try { + const embedding = deserializeEmbedding(row.embedding); + let tags: string[] = []; + try { tags = JSON.parse(row.tags) as string[]; } catch { /* malformed legacy tags rank without tags */ } + candidates.push({ + id: row.id, + title: row.title, + content: row.content, + tags, + scope: row.scope, + similarity: cosineSimilarity(queryEmbedding, embedding), + }); + } catch { + // Corrupt or dimension-mismatched rows remain available through lexical search. + } + } + candidates.sort((left, right) => + right.similarity - left.similarity || left.id.localeCompare(right.id) + ); + return { + ...lexical, + results: fuseSearchResults(lexical.results, candidates.slice(0, MAX_CANDIDATES), requestedLimit, input.explain), + totalMatches: new Set([...lexical.results.map(item => item.id), ...candidates.map(item => item.id)]).size, + retrievalMode: 'hybrid', + semantic: semanticStatus('ready', candidates.length), + }; + } catch (error) { + return { + ...lexical, + results: lexical.results.slice(0, requestedLimit), + retrievalMode: 'lexical', + semantic: semanticStatus('unavailable', count, error instanceof Error ? error.message : String(error)), + }; + } +} diff --git a/packages/memory/src/handlers/store.ts b/packages/memory/src/handlers/store.ts index f00b8fec..9b2dc465 100644 --- a/packages/memory/src/handlers/store.ts +++ b/packages/memory/src/handlers/store.ts @@ -5,7 +5,12 @@ import { normalizeTitle, normalizeScope, normalizeTags, hashContent } from '../s import { DuplicateError, StorageError } from '../utils/errors.js'; import type { StoreKnowledgeInput, StoreKnowledgeResult, KnowledgeRow } from '../types/index.js'; -export function storeKnowledge(input: StoreKnowledgeInput): StoreKnowledgeResult { +interface StoredEmbedding { + value: Buffer; + version: string; +} + +export function storeKnowledge(input: StoreKnowledgeInput, embedding?: StoredEmbedding): StoreKnowledgeResult { validateStoreInput(input); const db = getDatabase(); @@ -46,9 +51,10 @@ export function storeKnowledge(input: StoreKnowledgeInput): StoreKnowledgeResult db.execute( `INSERT INTO knowledge ( - id, title, content, tags, scope, - normalized_title, content_hash, created_at, updated_at - ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)`, + id, title, content, tags, scope, + normalized_title, content_hash, created_at, updated_at, + embedding, embedding_version + ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`, [ id, input.title.trim(), @@ -58,7 +64,9 @@ export function storeKnowledge(input: StoreKnowledgeInput): StoreKnowledgeResult normalizedTitle, contentHash, now, - now + now, + embedding?.value ?? null, + embedding?.version ?? null, ] ); @@ -77,4 +85,4 @@ export function storeKnowledge(input: StoreKnowledgeInput): StoreKnowledgeResult { originalError: error instanceof Error ? error.message : String(error) } ); } -} \ No newline at end of file +} diff --git a/packages/memory/src/handlers/update.ts b/packages/memory/src/handlers/update.ts index 48321e6f..19174d0e 100644 --- a/packages/memory/src/handlers/update.ts +++ b/packages/memory/src/handlers/update.ts @@ -4,7 +4,12 @@ import { normalizeTitle, normalizeScope, normalizeTags, hashContent } from '../s import { DuplicateError, NotFoundError, StorageError } from '../utils/errors.js'; import type { UpdateKnowledgeInput, UpdateKnowledgeResult, KnowledgeRow } from '../types/index.js'; -export function updateKnowledge(input: UpdateKnowledgeInput): UpdateKnowledgeResult { +interface StoredEmbedding { + value: Buffer; + version: string; +} + +export function updateKnowledge(input: UpdateKnowledgeInput, embedding?: StoredEmbedding): UpdateKnowledgeResult { validateUpdateInput(input); const db = getDatabase(); @@ -27,6 +32,9 @@ export function updateKnowledge(input: UpdateKnowledgeInput): UpdateKnowledgeRes const scope = input.scope !== undefined ? normalizeScope(input.scope) : existing.scope; const normalizedTitle = normalizeTitle(title); const contentHash = hashContent(content); + const invalidatesEmbedding = input.title !== undefined + || input.content !== undefined + || input.tags !== undefined; const existingByTitle = db.queryOne( 'SELECT id FROM knowledge WHERE normalized_title = ? AND scope = ? AND id != ?', @@ -57,7 +65,8 @@ export function updateKnowledge(input: UpdateKnowledgeInput): UpdateKnowledgeRes db.execute( `UPDATE knowledge SET title = ?, content = ?, tags = ?, scope = ?, - normalized_title = ?, content_hash = ?, updated_at = ? + normalized_title = ?, content_hash = ?, updated_at = ?, + embedding = ?, embedding_version = ? WHERE id = ?`, [ title, @@ -67,6 +76,8 @@ export function updateKnowledge(input: UpdateKnowledgeInput): UpdateKnowledgeRes normalizedTitle, contentHash, now, + embedding?.value ?? (invalidatesEmbedding ? null : existing.embedding ?? null), + embedding?.version ?? (invalidatesEmbedding ? null : existing.embedding_version ?? null), input.id ] ); diff --git a/packages/memory/src/server.ts b/packages/memory/src/server.ts index a3800f52..31e8683a 100644 --- a/packages/memory/src/server.ts +++ b/packages/memory/src/server.ts @@ -7,6 +7,9 @@ import { import { storeKnowledge } from './handlers/store.js'; import { searchKnowledge } from './handlers/search.js'; import { updateKnowledge } from './handlers/update.js'; +import { searchKnowledgeHybrid } from './handlers/semantic-search.js'; +import { storeKnowledgeSemantic, updateKnowledgeSemantic } from './handlers/semantic-maintenance.js'; +import { readSemanticConfig } from './services/config.js'; import { KnowledgeMemoryError } from './utils/errors.js'; import type { StoreKnowledgeInput, SearchKnowledgeInput, UpdateKnowledgeInput } from './types/index.js'; @@ -96,6 +99,10 @@ const SEARCH_TOOL = { type: 'number', description: 'Maximum number of results to return (1-20, default: 5)', }, + explain: { + type: 'boolean', + description: 'Include lexical and semantic rank details when semantic search is enabled.', + }, }, required: ['query'], }, @@ -104,6 +111,7 @@ const SEARCH_TOOL = { export const TOOLS = [STORE_TOOL, UPDATE_TOOL, SEARCH_TOOL]; export function createServer(): Server { + const semanticEnabled = readSemanticConfig().enabled; const server = new Server( { name: SERVER_NAME, @@ -132,7 +140,9 @@ export function createServer(): Server { // stale prompts/templates continue to work. Remove in next major. if (name === 'memory_storeKnowledge' || name === 'memory.storeKnowledge') { const input = args as unknown as StoreKnowledgeInput; - const result = storeKnowledge(input); + const result = semanticEnabled + ? await storeKnowledgeSemantic(input) + : storeKnowledge(input); return { content: [ { @@ -145,7 +155,9 @@ export function createServer(): Server { if (name === 'memory_updateKnowledge' || name === 'memory.updateKnowledge') { const input = args as unknown as UpdateKnowledgeInput; - const result = updateKnowledge(input); + const result = semanticEnabled + ? await updateKnowledgeSemantic(input) + : updateKnowledge(input); return { content: [ { @@ -158,7 +170,9 @@ export function createServer(): Server { if (name === 'memory_searchKnowledge' || name === 'memory.searchKnowledge') { const input = args as unknown as SearchKnowledgeInput; - const result = searchKnowledge(input); + const result = semanticEnabled + ? await searchKnowledgeHybrid(input) + : searchKnowledge(input); return { content: [ { diff --git a/packages/memory/src/services/config.ts b/packages/memory/src/services/config.ts new file mode 100644 index 00000000..ed5b40b1 --- /dev/null +++ b/packages/memory/src/services/config.ts @@ -0,0 +1,17 @@ +import { readFileSync } from 'fs'; +import { join } from 'path'; + +export interface SemanticConfig { + enabled: boolean; +} + +export function readSemanticConfig(directory = process.cwd()): SemanticConfig { + try { + const value = JSON.parse(readFileSync(join(directory, '.ai-devkit.json'), 'utf8')) as { + memory?: { semantic?: unknown }; + }; + return { enabled: value.memory?.semantic === true }; + } catch { + return { enabled: false }; + } +} diff --git a/packages/memory/src/types/index.ts b/packages/memory/src/types/index.ts index b72eff54..f1090b1c 100644 --- a/packages/memory/src/types/index.ts +++ b/packages/memory/src/types/index.ts @@ -44,6 +44,7 @@ export interface SearchKnowledgeInput { contextTags?: string[]; scope?: string; limit?: number; + explain?: boolean; } export interface SearchResultItem { @@ -68,6 +69,15 @@ export interface SearchKnowledgeResult { totalMatches: number; query: string; strategy: 'strict' | 'broad' | 'recent'; + retrievalMode?: 'lexical' | 'hybrid'; + semantic?: SemanticSearchStatus; +} + +export interface SemanticSearchStatus { + status: 'ready' | 'unavailable' | 'corpus-too-large'; + embeddingVersion: string; + eligibleCount: number; + reason?: string; } export type ListKnowledgeSort = 'updated-desc' | 'created-desc' | 'title-asc'; @@ -103,6 +113,8 @@ export interface KnowledgeRow { content_hash: string; created_at: string; updated_at: string; + embedding?: Buffer | null; + embedding_version?: string | null; } export interface MetaRow { diff --git a/packages/memory/tests/integration/semantic-search.test.ts b/packages/memory/tests/integration/semantic-search.test.ts new file mode 100644 index 00000000..53230529 --- /dev/null +++ b/packages/memory/tests/integration/semantic-search.test.ts @@ -0,0 +1,117 @@ +import { join } from 'path'; +import { tmpdir } from 'os'; +import { rmSync } from 'fs'; +import { closeDatabase, getDatabase } from '../../src/database'; +import { storeKnowledge } from '../../src/handlers/store'; +import { updateKnowledge } from '../../src/handlers/update'; +import { searchKnowledgeHybrid } from '../../src/handlers/semantic-search'; +import { MODEL_VERSION } from '../../src/services/model'; +import { serializeEmbedding } from '../../src/services/semantic'; +import { SemanticModelUnavailableError, type LocalEmbedder } from '../../src/services/embedder'; +import { getSemanticStatus, reembedKnowledge, storeKnowledgeSemantic } from '../../src/handlers/semantic-maintenance'; + +describe('semantic search integration', () => { + const dbPath = join(tmpdir(), `semantic-search-${Date.now()}-${Math.random().toString(36)}.db`); + const first = new Float32Array(384); first[0] = 1; + const second = new Float32Array(384); second[1] = 1; + const embedder: LocalEmbedder = { + embed: async () => first, + embedMany: async () => [first], + dispose: async () => undefined, + }; + + beforeAll(() => getDatabase({ dbPath })); + afterAll(() => { + closeDatabase(); + rmSync(dbPath, { force: true }); + rmSync(`${dbPath}-wal`, { force: true }); + rmSync(`${dbPath}-shm`, { force: true }); + }); + + beforeEach(() => getDatabase().execute('DELETE FROM knowledge')); + + function store(title: string, content: string) { + const result = storeKnowledge({ title, content, tags: ['semantic'], scope: 'global' }); + return result.id!; + } + + it('retrieves a semantic-only paraphrase and exposes explanations', async () => { + const relevant = store('Response serialization boundary', 'Public HTTP payloads are mapped through dedicated transfer objects before leaving the service boundary.'); + const distractor = store('Queue retention policy rules', 'Failed queue messages remain available for seven days before automated cleanup removes them permanently.'); + const db = getDatabase(); + db.execute('UPDATE knowledge SET embedding = ?, embedding_version = ? WHERE id = ?', [serializeEmbedding(first), MODEL_VERSION, relevant]); + db.execute('UPDATE knowledge SET embedding = ?, embedding_version = ? WHERE id = ?', [serializeEmbedding(second), MODEL_VERSION, distractor]); + + const result = await searchKnowledgeHybrid( + { query: 'wire contracts', limit: 5, explain: true }, + { embedder }, + ); + + expect(result.results[0]?.id).toBe(relevant); + expect(result.retrievalMode).toBe('hybrid'); + expect(result.results[0]?.retrieval).toMatchObject({ lexicalRank: null, semanticRank: 1 }); + expect(result.semantic.status).toBe('ready'); + }); + + it('degrades to lexical results when the local model is unavailable', async () => { + const relevant = store('Exact retry policy identifier', 'The WEB-1842 retry rule applies exponential backoff only to transient server failures.'); + const unavailable: LocalEmbedder = { + embed: async () => { throw new SemanticModelUnavailableError('offline'); }, + embedMany: async () => [], + dispose: async () => undefined, + }; + + const result = await searchKnowledgeHybrid({ query: 'WEB-1842', limit: 5 }, { embedder: unavailable }); + + expect(result.results[0]?.id).toBe(relevant); + expect(result.retrievalMode).toBe('lexical'); + expect(result.semantic.status).toBe('unavailable'); + }); + + it('invalidates embeddings for document changes but retains them for scope-only updates', () => { + const id = store('Embedding invalidation behavior', 'Changing searchable memory text must invalidate its previously computed semantic embedding value.'); + const db = getDatabase(); + db.execute('UPDATE knowledge SET embedding = ?, embedding_version = ? WHERE id = ?', [serializeEmbedding(first), MODEL_VERSION, id]); + + updateKnowledge({ id, scope: 'project:memory' }); + expect(db.queryOne<{ embedding: Buffer | null }>('SELECT embedding FROM knowledge WHERE id = ?', [id])?.embedding).not.toBeNull(); + + updateKnowledge({ id, tags: ['changed'] }); + expect(db.queryOne<{ embedding: Buffer | null; embedding_version: string | null }>( + 'SELECT embedding, embedding_version FROM knowledge WHERE id = ?', [id] + )).toEqual({ embedding: null, embedding_version: null }); + }); + + it('backfills missing embeddings and skips current rows on the next run', async () => { + store('Missing semantic vector', 'This memory begins without an embedding and must be included by the resumable backfill operation.'); + store('Another missing vector', 'This second memory also needs a deterministic semantic vector written by the maintenance command.'); + + const firstRun = await reembedKnowledge({ embedder, batchSize: 1 }); + const secondRun = await reembedKnowledge({ embedder, batchSize: 1 }); + + expect(firstRun).toMatchObject({ embedded: 2, failed: 0, skipped: 0 }); + expect(secondRun).toMatchObject({ embedded: 0, failed: 0, skipped: 2 }); + expect(getDatabase().queryOne<{ count: number }>( + 'SELECT COUNT(*) AS count FROM knowledge WHERE embedding_version = ?', [MODEL_VERSION] + )?.count).toBe(2); + }); + + it('stores an embedding with semantic-aware writes', async () => { + const result = await storeKnowledgeSemantic({ + title: 'Semantic write behavior', + content: 'Semantic-enabled stores write the searchable memory and its matching model vector in one database transaction.', + tags: ['write'], + }, { embedder }); + + expect(getDatabase().queryOne<{ embedding: Buffer; embedding_version: string }>( + 'SELECT embedding, embedding_version FROM knowledge WHERE id = ?', [result.id] + )).toMatchObject({ embedding_version: MODEL_VERSION }); + }); + + it('reports model and embedding readiness without loading the runtime', async () => { + store('Status missing vector', 'The status command reports memories that have not yet received a compatible semantic embedding value.'); + const status = await getSemanticStatus({ modelsRoot: join(tmpdir(), 'definitely-missing-model-root') }); + + expect(status).toMatchObject({ modelReady: false, total: 1, current: 0, missing: 1, stale: 0 }); + }); +}); diff --git a/packages/memory/tests/unit/config.test.ts b/packages/memory/tests/unit/config.test.ts new file mode 100644 index 00000000..e22482ff --- /dev/null +++ b/packages/memory/tests/unit/config.test.ts @@ -0,0 +1,22 @@ +import { join } from 'path'; +import { tmpdir } from 'os'; +import { mkdtempSync, rmSync, writeFileSync } from 'fs'; +import { readSemanticConfig } from '../../src/services/config'; + +describe('memory semantic config', () => { + const directory = mkdtempSync(join(tmpdir(), 'memory-config-')); + afterAll(() => rmSync(directory, { recursive: true, force: true })); + + it('defaults to disabled for missing or malformed config', () => { + expect(readSemanticConfig(directory)).toEqual({ enabled: false }); + writeFileSync(join(directory, '.ai-devkit.json'), '{bad'); + expect(readSemanticConfig(directory)).toEqual({ enabled: false }); + }); + + it('enables semantics only for an explicit true boolean', () => { + writeFileSync(join(directory, '.ai-devkit.json'), JSON.stringify({ memory: { semantic: true } })); + expect(readSemanticConfig(directory)).toEqual({ enabled: true }); + writeFileSync(join(directory, '.ai-devkit.json'), JSON.stringify({ memory: { semantic: 'true' } })); + expect(readSemanticConfig(directory)).toEqual({ enabled: false }); + }); +}); From 7686a7dd07baebb2507aa3c9dc058900e4723162 Mon Sep 17 00:00:00 2001 From: Hoang Nguyen Date: Tue, 1 Sep 2026 06:29:24 +0000 Subject: [PATCH 04/11] fix(memory): await semantic writes and harden fallbacks --- docs/ai/2026-09-01-feature-memory-semantic.md | 8 +++-- .../cli/src/__tests__/commands/memory.test.ts | 14 +++++++++ packages/cli/src/commands/memory.ts | 4 +-- packages/memory/README.md | 25 +++++++++++++++ .../tests/integration/semantic-search.test.ts | 31 +++++++++++++++++++ 5 files changed, 77 insertions(+), 5 deletions(-) diff --git a/docs/ai/2026-09-01-feature-memory-semantic.md b/docs/ai/2026-09-01-feature-memory-semantic.md index 02aeb12a..5af3b120 100644 --- a/docs/ai/2026-09-01-feature-memory-semantic.md +++ b/docs/ai/2026-09-01-feature-memory-semantic.md @@ -55,8 +55,8 @@ flowchart LR - [x] Add status, download, and resumable re-embed APIs and CLI tests. - [x] Add async hybrid search with offline/corrupt/stale degradation tests; large-corpus validation remains in the performance gate. - [x] Wire config through CLI and MCP while preserving default behavior. -- [ ] Run expanded-100 gate through the built CLI. -- [ ] Run 1,000-row warm latency gate. +- [x] Run expanded-100 gate through the built CLI: hit@1 88%, hit@3 97%, hit@5 98%, zero results 0%; one-shot CLI p95 1,886 ms and seed time 76.6 s. +- [x] Run 1,000-row warm latency gate: median 21.85 ms, p95 29.78 ms, max 33.51 ms across 100 searches after five warmups. - [ ] Run six-project build, full tests, lint, and e2e. ## Testing record @@ -68,10 +68,12 @@ flowchart LR - [x] RRF is deterministic and protects lexical ties. - [x] Offline and missing-model search returns lexical results. - [x] Stale/corrupt embeddings are excluded safely. -- [x] Backfill resumes and skips current rows; force mode remains in final CLI validation. +- [x] Backfill resumes, skips current rows, and force-rebuilds. - [x] CLI surfaces config, status, download, reembed, and explanations. - [x] Default lexical API and strict/broad strategies remain compatible in targeted tests. +The built-CLI benchmark improved the published 0.57.1 lexical baseline from hit@1 81% / hit@3 91% / hit@5 96% / zero 1% to 88% / 97% / 98% / zero 0%. Judged irrelevant top-three increased from 2.9% to 4.7%, a known semantic-recall trade-off to monitor. The one-shot CLI timings include process and ONNX startup; the warm in-process 1,000-row gate is the search SLA measurement. + ## Rollback Set `memory.semantic` to false. The additive columns become inert and older named-column queries continue to work. Deleting a knowledge row deletes its BLOB with no auxiliary index cleanup. Model files can be removed independently from the validated cache directory. A later cleanup migration may null the columns, but rollback does not require reversing schema history. diff --git a/packages/cli/src/__tests__/commands/memory.test.ts b/packages/cli/src/__tests__/commands/memory.test.ts index 29ca00a0..53b1f141 100644 --- a/packages/cli/src/__tests__/commands/memory.test.ts +++ b/packages/cli/src/__tests__/commands/memory.test.ts @@ -353,6 +353,20 @@ describe('memory command', () => { expect(mockedMemorySearchCommand).not.toHaveBeenCalled(); }); + it('awaits semantic-enabled stores before printing their result', async () => { + mockGetMemorySemanticEnabled.mockResolvedValue(true); + const result = { success: true, id: 'semantic-1', message: 'stored' }; + vi.mocked(memoryStoreCommandAsync).mockResolvedValue(result); + + const program = new Command(); registerMemoryCommand(program); + await program.parseAsync([ + 'node', 'test', 'memory', 'store', '--title', 'Semantic store title', + '--content', 'This semantic store content is long enough to pass all memory validation constraints.', + ]); + + expect(consoleLogSpy).toHaveBeenCalledWith(JSON.stringify(result, null, 2)); + }); + it('exposes semantic status, download, and force reembed commands', async () => { vi.mocked(memorySemanticStatusCommand).mockResolvedValue({ modelReady: false, modelDirectory: '/models', embeddingVersion: 'model', total: 0, current: 0, missing: 0, stale: 0, diff --git a/packages/cli/src/commands/memory.ts b/packages/cli/src/commands/memory.ts index 9e1624d0..efb4a2d3 100644 --- a/packages/cli/src/commands/memory.ts +++ b/packages/cli/src/commands/memory.ts @@ -46,7 +46,7 @@ export function registerMemoryCommand(program: Command): void { dbPath: await resolveMemoryDbPath() } as MemoryStoreOptions; const result = await resolveSemanticEnabled() - ? memoryStoreCommandAsync({ ...commandOptions, semantic: true }) + ? await memoryStoreCommandAsync({ ...commandOptions, semantic: true }) : memoryStoreCommand(commandOptions); console.log(JSON.stringify(result, null, 2)); })); @@ -65,7 +65,7 @@ export function registerMemoryCommand(program: Command): void { dbPath: await resolveMemoryDbPath() } as MemoryUpdateOptions; const result = await resolveSemanticEnabled() - ? memoryUpdateCommandAsync({ ...commandOptions, semantic: true }) + ? await memoryUpdateCommandAsync({ ...commandOptions, semantic: true }) : memoryUpdateCommand(commandOptions); console.log(JSON.stringify(result, null, 2)); })); diff --git a/packages/memory/README.md b/packages/memory/README.md index dee268fb..4d774bc7 100644 --- a/packages/memory/README.md +++ b/packages/memory/README.md @@ -10,6 +10,7 @@ Most users get this automatically through `ai-devkit init`. Install `@ai-devkit/ ## Features - 🔍 **Full-Text Search** — FTS5 with BM25 ranking +- 🧠 **Optional Semantic Search** — Local MiniLM embeddings fused with lexical ranks - 🏷️ **Tag-Based Filtering** — Organize and find knowledge by tags - 📁 **Scoped Knowledge** — Global, project, or repo-specific rules - 🔄 **Deduplication** — Prevents duplicate content automatically @@ -65,6 +66,30 @@ Example developer use case: after deciding that all API responses must use DTOs, } ``` +## Optional semantic search + +Semantic search is disabled by default. Enable it in the project's `.ai-devkit.json`: + +```json +{ + "memory": { + "semantic": true + } +} +``` + +The first semantic operation downloads a pinned 23 MB quantized MiniLM model to `~/.ai-devkit/models`. Queries and memory content are embedded locally and are not sent to an inference service. Once cached, the model works offline. If it is unavailable, search returns lexical FTS results instead. + +You can prepare and inspect the cache explicitly, then backfill existing memories: + +```bash +ai-devkit memory semantic status +ai-devkit memory semantic download +ai-devkit memory reembed +``` + +Use `ai-devkit memory reembed --force` after troubleshooting a cache or embedding issue. Add `--explain` to `memory search` to include lexical rank, semantic rank, cosine similarity, and reciprocal-rank-fusion score. + ## Documentation 📖 **For the full API reference, ranking details, and advanced usage, visit:** diff --git a/packages/memory/tests/integration/semantic-search.test.ts b/packages/memory/tests/integration/semantic-search.test.ts index 53230529..f198eba9 100644 --- a/packages/memory/tests/integration/semantic-search.test.ts +++ b/packages/memory/tests/integration/semantic-search.test.ts @@ -88,9 +88,11 @@ describe('semantic search integration', () => { const firstRun = await reembedKnowledge({ embedder, batchSize: 1 }); const secondRun = await reembedKnowledge({ embedder, batchSize: 1 }); + const forcedRun = await reembedKnowledge({ embedder, batchSize: 1, force: true }); expect(firstRun).toMatchObject({ embedded: 2, failed: 0, skipped: 0 }); expect(secondRun).toMatchObject({ embedded: 0, failed: 0, skipped: 2 }); + expect(forcedRun).toMatchObject({ embedded: 2, failed: 0, skipped: 0 }); expect(getDatabase().queryOne<{ count: number }>( 'SELECT COUNT(*) AS count FROM knowledge WHERE embedding_version = ?', [MODEL_VERSION] )?.count).toBe(2); @@ -114,4 +116,33 @@ describe('semantic search integration', () => { expect(status).toMatchObject({ modelReady: false, total: 1, current: 0, missing: 1, stale: 0 }); }); + + it('ignores corrupt vectors without losing lexical results', async () => { + const relevant = store('Corrupt vector fallback', 'Lexical retrieval remains available when a stored semantic vector has an invalid byte length.'); + getDatabase().execute('UPDATE knowledge SET embedding = ?, embedding_version = ? WHERE id = ?', [Buffer.from([1, 2]), MODEL_VERSION, relevant]); + + const result = await searchKnowledgeHybrid({ query: 'corrupt vector fallback', limit: 5 }, { embedder }); + + expect(result.results[0]?.id).toBe(relevant); + expect(result.semantic).toMatchObject({ status: 'ready', eligibleCount: 0 }); + }); + + it('skips semantic inference when the eligible corpus exceeds the scan limit', async () => { + const db = getDatabase(); + const insert = db.instance.prepare(`INSERT INTO knowledge + (id,title,content,tags,scope,normalized_title,content_hash,created_at,updated_at,embedding,embedding_version) + VALUES (?,?,?,?,?,?,?,?,?,?,?)`); + db.instance.transaction(() => { + for (let index = 0; index < 5_001; index++) { + insert.run(`large-${index}`, `Large corpus memory ${index}`, `Large corpus content ${index} contains enough searchable developer knowledge for validation.`, '[]', 'global', `large corpus memory ${index}`, `large-hash-${index}`, '2026-09-01', '2026-09-01', serializeEmbedding(first), MODEL_VERSION); + } + })(); + const neverCalled = vi.spyOn(embedder, 'embed'); + + const result = await searchKnowledgeHybrid({ query: 'large corpus', limit: 5 }, { embedder }); + + expect(result.semantic.status).toBe('corpus-too-large'); + expect(result.retrievalMode).toBe('lexical'); + expect(neverCalled).not.toHaveBeenCalled(); + }); }); From 079bfce3786cb7be1ef5dc6847592bbcfbb506df Mon Sep 17 00:00:00 2001 From: Hoang Nguyen Date: Tue, 1 Sep 2026 06:33:39 +0000 Subject: [PATCH 05/11] fix(memory): reuse semantic model sessions --- docs/ai/2026-09-01-feature-memory-semantic.md | 2 +- packages/memory/package-lock.json | 138 +++++++++++++++++- .../src/handlers/semantic-maintenance.ts | 14 +- .../memory/src/handlers/semantic-search.ts | 13 +- packages/memory/src/services/embedder.ts | 11 ++ 5 files changed, 158 insertions(+), 20 deletions(-) diff --git a/docs/ai/2026-09-01-feature-memory-semantic.md b/docs/ai/2026-09-01-feature-memory-semantic.md index 5af3b120..c6c7c4ac 100644 --- a/docs/ai/2026-09-01-feature-memory-semantic.md +++ b/docs/ai/2026-09-01-feature-memory-semantic.md @@ -57,7 +57,7 @@ flowchart LR - [x] Wire config through CLI and MCP while preserving default behavior. - [x] Run expanded-100 gate through the built CLI: hit@1 88%, hit@3 97%, hit@5 98%, zero results 0%; one-shot CLI p95 1,886 ms and seed time 76.6 s. - [x] Run 1,000-row warm latency gate: median 21.85 ms, p95 29.78 ms, max 33.51 ms across 100 searches after five warmups. -- [ ] Run six-project build, full tests, lint, and e2e. +- [x] Run six-project build, full tests, lint, and e2e: all six projects built; 1,087 tests and 41 e2e tests passed; lint completed with no errors. ## Testing record diff --git a/packages/memory/package-lock.json b/packages/memory/package-lock.json index 106accc1..2f8adc16 100644 --- a/packages/memory/package-lock.json +++ b/packages/memory/package-lock.json @@ -1,18 +1,20 @@ { "name": "@ai-devkit/memory", - "version": "0.15.0", + "version": "0.17.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "@ai-devkit/memory", - "version": "0.15.0", + "version": "0.17.0", "license": "MIT", "dependencies": { + "@huggingface/tokenizers": "0.1.3", "@modelcontextprotocol/sdk": "^1.0.0", "@typescript-eslint/eslint-plugin": "8.60.1", "@typescript-eslint/parser": "8.60.1", "better-sqlite3": "12.11.1", + "onnxruntime-web": "1.22.0", "uuid": "^14.0.0" }, "bin": { @@ -192,6 +194,12 @@ "hono": "^4" } }, + "node_modules/@huggingface/tokenizers": { + "version": "0.1.3", + "resolved": "https://registry.npmjs.org/@huggingface/tokenizers/-/tokenizers-0.1.3.tgz", + "integrity": "sha512-8rF/RRT10u+kn7YuUbUg0OF30K8rjTc78aHpxT+qJ1uWSqxT1MHi8+9ltwYfkFYJzT/oS+qw3JVfHtNMGAdqyA==", + "license": "Apache-2.0" + }, "node_modules/@jridgewell/resolve-uri": { "version": "3.1.2", "resolved": "https://registry.npmjs.org/@jridgewell/resolve-uri/-/resolve-uri-3.1.2.tgz", @@ -618,6 +626,63 @@ "url": "https://github.com/sponsors/Boshen" } }, + "node_modules/@protobufjs/aspromise": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/@protobufjs/aspromise/-/aspromise-1.1.2.tgz", + "integrity": "sha512-j+gKExEuLmKwvz3OgROXtrJ2UG2x8Ch2YZUxahh+s1F2HZ+wAceUNLkvy6zKCPVRkU++ZWQrdxsUeQXmcg4uoQ==", + "license": "BSD-3-Clause" + }, + "node_modules/@protobufjs/base64": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/@protobufjs/base64/-/base64-1.1.2.tgz", + "integrity": "sha512-AZkcAA5vnN/v4PDqKyMR5lx7hZttPDgClv83E//FMNhR2TMcLUhfRUBHCmSl0oi9zMgDDqRUJkSxO3wm85+XLg==", + "license": "BSD-3-Clause" + }, + "node_modules/@protobufjs/codegen": { + "version": "2.0.5", + "resolved": "https://registry.npmjs.org/@protobufjs/codegen/-/codegen-2.0.5.tgz", + "integrity": "sha512-zgXFLzW3Ap33e6d0Wlj4MGIm6Ce8O89n/apUaGNB/jx+hw+ruWEp7EwGUshdLKVRCxZW12fp9r40E1mQrf/34g==", + "license": "BSD-3-Clause" + }, + "node_modules/@protobufjs/eventemitter": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@protobufjs/eventemitter/-/eventemitter-1.1.1.tgz", + "integrity": "sha512-vW1GmwMZNnL+gMRaovlh9yZX74kc+TTU3FObkkurpMaRtBfLP3ldjS9KQWlwZgraRE0+dheEEoAxdzcJQ8eXZg==", + "license": "BSD-3-Clause" + }, + "node_modules/@protobufjs/fetch": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@protobufjs/fetch/-/fetch-1.1.1.tgz", + "integrity": "sha512-GpptLrs57adMSuHi3VNj0mAF8dwh36LMaYF6XyJ6JMWlVsc+t42tm1HSEDmOs3A8fC9yyeisgLhsTVQokOZ0zw==", + "license": "BSD-3-Clause", + "dependencies": { + "@protobufjs/aspromise": "^1.1.1" + } + }, + "node_modules/@protobufjs/float": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/@protobufjs/float/-/float-1.0.2.tgz", + "integrity": "sha512-Ddb+kVXlXst9d+R9PfTIxh1EdNkgoRe5tOX6t01f1lYWOvJnSPDBlG241QLzcyPdoNTsblLUdujGSE4RzrTZGQ==", + "license": "BSD-3-Clause" + }, + "node_modules/@protobufjs/path": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/@protobufjs/path/-/path-1.1.2.tgz", + "integrity": "sha512-6JOcJ5Tm08dOHAbdR3GrvP+yUUfkjG5ePsHYczMFLq3ZmMkAD98cDgcT2iA1lJ9NVwFd4tH/iSSoe44YWkltEA==", + "license": "BSD-3-Clause" + }, + "node_modules/@protobufjs/pool": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/@protobufjs/pool/-/pool-1.1.0.tgz", + "integrity": "sha512-0kELaGSIDBKvcgS4zkjz1PeddatrjYcmMWOlAuAPwAeccUrPHdUqo/J6LiymHHEiJT5NrF1UVwxY14f+fy4WQw==", + "license": "BSD-3-Clause" + }, + "node_modules/@protobufjs/utf8": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/@protobufjs/utf8/-/utf8-1.1.2.tgz", + "integrity": "sha512-b1UQwcEZ4yCnMCD8DAL1VlbvBJE9/IX4FTIp7BG1xYpf29SLazLSrqUkj4w7Y5y7cCVP6E5tcqqcI0xemPkHug==", + "license": "BSD-3-Clause" + }, "node_modules/@rolldown/binding-android-arm64": { "version": "1.1.4", "resolved": "https://registry.npmjs.org/@rolldown/binding-android-arm64/-/binding-android-arm64-1.1.4.tgz", @@ -1370,7 +1435,6 @@ "version": "20.19.30", "resolved": "https://registry.npmjs.org/@types/node/-/node-20.19.30.tgz", "integrity": "sha512-WJtwWJu7UdlvzEAUm484QNg5eAoq5QR08KDNx7g45Usrs2NtOPiX8ugDqmKdXkyL03rBqU5dYNYVQetEpBHq2g==", - "dev": true, "license": "MIT", "dependencies": { "undici-types": "~6.21.0" @@ -3433,6 +3497,12 @@ "url": "https://github.com/sponsors/sindresorhus" } }, + "node_modules/flatbuffers": { + "version": "25.9.23", + "resolved": "https://registry.npmjs.org/flatbuffers/-/flatbuffers-25.9.23.tgz", + "integrity": "sha512-MI1qs7Lo4Syw0EOzUl0xjs2lsoeqFku44KpngfIduHBYvzm8h2+7K8YMQh1JtVVVrUvhLpNwqVi4DERegUJhPQ==", + "license": "Apache-2.0" + }, "node_modules/form-data-encoder": { "version": "4.1.0", "resolved": "https://registry.npmjs.org/form-data-encoder/-/form-data-encoder-4.1.0.tgz", @@ -3648,6 +3718,12 @@ "dev": true, "license": "ISC" }, + "node_modules/guid-typescript": { + "version": "1.0.9", + "resolved": "https://registry.npmjs.org/guid-typescript/-/guid-typescript-1.0.9.tgz", + "integrity": "sha512-Y8T4vYhEfwJOTbouREvG+3XDsjr8E3kIr7uf+JZ0BYloFsttiHU0WfvANVsR7TxNUJa/WpCnw/Ino/p+DeBhBQ==", + "license": "ISC" + }, "node_modules/has-flag": { "version": "4.0.0", "resolved": "https://registry.npmjs.org/has-flag/-/has-flag-4.0.0.tgz", @@ -4240,6 +4316,12 @@ "url": "https://opencollective.com/parcel" } }, + "node_modules/long": { + "version": "5.3.2", + "resolved": "https://registry.npmjs.org/long/-/long-5.3.2.tgz", + "integrity": "sha512-mNAgZ1GmyNhD7AuqnTG3/VQ26o760+ZYBPKjPvugO8+nLbYfX6TVpJPseBvopbdY+qpZ/lKUnmEc1LeZYS3QAA==", + "license": "Apache-2.0" + }, "node_modules/lowercase-keys": { "version": "3.0.0", "resolved": "https://registry.npmjs.org/lowercase-keys/-/lowercase-keys-3.0.0.tgz", @@ -4580,6 +4662,26 @@ "wrappy": "1" } }, + "node_modules/onnxruntime-common": { + "version": "1.22.0", + "resolved": "https://registry.npmjs.org/onnxruntime-common/-/onnxruntime-common-1.22.0.tgz", + "integrity": "sha512-vcuaNWgtF2dGQu/EP5P8UI5rEPEYqXG2sPPe5j9lg2TY/biJF8eWklTMwlDO08iuXq48xJo0awqIpK5mPG+IxA==", + "license": "MIT" + }, + "node_modules/onnxruntime-web": { + "version": "1.22.0", + "resolved": "https://registry.npmjs.org/onnxruntime-web/-/onnxruntime-web-1.22.0.tgz", + "integrity": "sha512-Ud/+EBo6mhuaQWt/OjaOk0iNWjXqJoeeMFr6xQEERZdIZH2OWpGzuujz7lfuOBjUa6TEE/sc4nb7Da5dNL34fg==", + "license": "MIT", + "dependencies": { + "flatbuffers": "^25.1.24", + "guid-typescript": "^1.0.9", + "long": "^5.2.3", + "onnxruntime-common": "1.22.0", + "platform": "^1.3.6", + "protobufjs": "^7.2.4" + } + }, "node_modules/p-cancelable": { "version": "4.0.1", "resolved": "https://registry.npmjs.org/p-cancelable/-/p-cancelable-4.0.1.tgz", @@ -4713,6 +4815,12 @@ "node": ">=16.20.0" } }, + "node_modules/platform": { + "version": "1.3.6", + "resolved": "https://registry.npmjs.org/platform/-/platform-1.3.6.tgz", + "integrity": "sha512-fnWVljUchTro6RiCFvCXBbNhJc2NijN7oIQxbwsyL0buWJPG85v81ehlHI9fXrJsMNgTofEoWIQeClKpgxFLrg==", + "license": "MIT" + }, "node_modules/postcss": { "version": "8.5.16", "resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.16.tgz", @@ -4784,6 +4892,29 @@ "url": "https://github.com/sponsors/sindresorhus" } }, + "node_modules/protobufjs": { + "version": "7.6.6", + "resolved": "https://registry.npmjs.org/protobufjs/-/protobufjs-7.6.6.tgz", + "integrity": "sha512-dYDWdjSl5RNb7SgPxGQcRU+GtvP7s2fpkrY0r432PcOIaZ0/rBcxEZnQN67iJhFuQiVw754JDoPruPCNdGsbjg==", + "hasInstallScript": true, + "license": "BSD-3-Clause", + "dependencies": { + "@protobufjs/aspromise": "^1.1.2", + "@protobufjs/base64": "^1.1.2", + "@protobufjs/codegen": "^2.0.5", + "@protobufjs/eventemitter": "^1.1.1", + "@protobufjs/fetch": "^1.1.1", + "@protobufjs/float": "^1.0.2", + "@protobufjs/path": "^1.1.2", + "@protobufjs/pool": "^1.1.0", + "@protobufjs/utf8": "^1.1.1", + "@types/node": ">=13.7.0", + "long": "^5.3.2" + }, + "engines": { + "node": ">=12.0.0" + } + }, "node_modules/proxy-addr": { "version": "2.0.7", "resolved": "https://registry.npmjs.org/proxy-addr/-/proxy-addr-2.0.7.tgz", @@ -5721,7 +5852,6 @@ "version": "6.21.0", "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-6.21.0.tgz", "integrity": "sha512-iwDZqg0QAGrg9Rav5H4n0M64c3mkR59cJ6wQp+7C4nI0gsmExaedaYLNO44eT4AtBBwjbTiGPMlt2Md0T9H9JQ==", - "dev": true, "license": "MIT" }, "node_modules/unicorn-magic": { diff --git a/packages/memory/src/handlers/semantic-maintenance.ts b/packages/memory/src/handlers/semantic-maintenance.ts index 70cf43a5..c7919736 100644 --- a/packages/memory/src/handlers/semantic-maintenance.ts +++ b/packages/memory/src/handlers/semantic-maintenance.ts @@ -1,6 +1,6 @@ import { getDatabase } from '../database/index.js'; import { normalizeTags } from '../services/normalizer.js'; -import { loadLocalEmbedder, type LocalEmbedder } from '../services/embedder.js'; +import { getDefaultLocalEmbedder, loadLocalEmbedder, type LocalEmbedder } from '../services/embedder.js'; import { ensureModelFiles, getModelDirectory, inspectModelFiles, MODEL_VERSION } from '../services/model.js'; import { buildEmbeddingText, serializeEmbedding } from '../services/semantic.js'; import { storeKnowledge } from './store.js'; @@ -64,7 +64,9 @@ export async function storeKnowledgeSemantic( ): Promise { let value: Buffer; try { - const embedder = options.embedder ?? await loadLocalEmbedder({ modelsRoot: options.modelsRoot, download: true }); + const embedder = options.embedder ?? (options.modelsRoot + ? await loadLocalEmbedder({ modelsRoot: options.modelsRoot, download: true }) + : await getDefaultLocalEmbedder()); const tags = normalizeTags(input.tags ?? []); const vector = await embedder.embed(buildEmbeddingText({ title: input.title, content: input.content, tags })); value = serializeEmbedding(vector); @@ -85,7 +87,9 @@ export async function updateKnowledgeSemantic( if (!existing) return updateKnowledge(input); let value: Buffer; try { - const embedder = options.embedder ?? await loadLocalEmbedder({ modelsRoot: options.modelsRoot, download: true }); + const embedder = options.embedder ?? (options.modelsRoot + ? await loadLocalEmbedder({ modelsRoot: options.modelsRoot, download: true }) + : await getDefaultLocalEmbedder()); const vector = await embedder.embed(buildEmbeddingText({ title: input.title ?? existing.title, content: input.content ?? existing.content, @@ -109,7 +113,9 @@ export async function reembedKnowledge(options: ReembedOptions = {}): Promise !row.embedding || row.embedding_version !== MODEL_VERSION); - const embedder = options.embedder ?? await loadLocalEmbedder({ modelsRoot: options.modelsRoot, download: true }); + const embedder = options.embedder ?? (options.modelsRoot + ? await loadLocalEmbedder({ modelsRoot: options.modelsRoot, download: true }) + : await getDefaultLocalEmbedder()); const batchSize = Math.max(1, options.batchSize ?? 32); let embedded = 0; let failed = 0; diff --git a/packages/memory/src/handlers/semantic-search.ts b/packages/memory/src/handlers/semantic-search.ts index e92194b6..0bf9b974 100644 --- a/packages/memory/src/handlers/semantic-search.ts +++ b/packages/memory/src/handlers/semantic-search.ts @@ -1,5 +1,5 @@ import { getDatabase } from '../database/index.js'; -import { loadLocalEmbedder, type LocalEmbedder } from '../services/embedder.js'; +import { getDefaultLocalEmbedder, type LocalEmbedder } from '../services/embedder.js'; import { MODEL_DIMENSION, MODEL_VERSION } from '../services/model.js'; import { cosineSimilarity, deserializeEmbedding, fuseSearchResults, type SemanticCandidate } from '../services/semantic.js'; import { searchKnowledge } from './search.js'; @@ -7,7 +7,6 @@ import type { SearchKnowledgeInput, SearchKnowledgeResult } from '../types/index const MAX_SEMANTIC_CORPUS = 5_000; const MAX_CANDIDATES = 20; -let defaultEmbedder: Promise | undefined; interface RawSemanticRow { id: string; @@ -35,14 +34,6 @@ function semanticStatus( } as const; } -function getDefaultEmbedder(): Promise { - defaultEmbedder ??= loadLocalEmbedder({ download: true }).catch(error => { - defaultEmbedder = undefined; - throw error; - }); - return defaultEmbedder; -} - export async function searchKnowledgeHybrid( input: SearchKnowledgeInput, options: HybridSearchOptions = {}, @@ -67,7 +58,7 @@ export async function searchKnowledgeHybrid( } try { - const embedder = options.embedder ?? await getDefaultEmbedder(); + const embedder = options.embedder ?? await getDefaultLocalEmbedder(); const queryEmbedding = await embedder.embed(input.query.trim()); if (queryEmbedding.length !== MODEL_DIMENSION) { throw new Error(`Semantic model returned ${queryEmbedding.length} dimensions; expected ${MODEL_DIMENSION}`); diff --git a/packages/memory/src/services/embedder.ts b/packages/memory/src/services/embedder.ts index fdd2e0a1..6e5ab39a 100644 --- a/packages/memory/src/services/embedder.ts +++ b/packages/memory/src/services/embedder.ts @@ -47,6 +47,17 @@ interface LoadEmbedderOptions { download?: boolean; } +let defaultEmbedder: Promise | undefined; + +/** Reuse the model session across semantic operations in long-lived CLI/MCP processes. */ +export function getDefaultLocalEmbedder(): Promise { + defaultEmbedder ??= loadLocalEmbedder({ download: true }).catch(error => { + defaultEmbedder = undefined; + throw error; + }); + return defaultEmbedder; +} + export async function loadLocalEmbedder(options: LoadEmbedderOptions = {}): Promise { const directory = getModelDirectory(options.modelsRoot); if (options.download) { From ef53cc5419efadf46ffe4463b4cccbd7b82f46a9 Mon Sep 17 00:00:00 2001 From: Hoang Nguyen Date: Tue, 1 Sep 2026 07:10:48 +0000 Subject: [PATCH 06/11] refactor(cli): drop unused GlobalDevKitConfig.memory.semantic field getMemorySemanticEnabled() only ever reads the per-project .ai-devkit.json config; the global config type gained a semantic flag that nothing reads or writes. Config surface should match what's actually wired. Co-Authored-By: Claude Sonnet 5 --- packages/cli/src/types.ts | 1 - 1 file changed, 1 deletion(-) diff --git a/packages/cli/src/types.ts b/packages/cli/src/types.ts index 49b2a23b..8ec07d12 100644 --- a/packages/cli/src/types.ts +++ b/packages/cli/src/types.ts @@ -60,7 +60,6 @@ export interface GlobalDevKitConfig { plugins?: string[]; memory?: { path?: string; - semantic?: boolean; }; } From 696c38598ee13bcc93559cb7b6e4ae65d907af0f Mon Sep 17 00:00:00 2001 From: Hoang Nguyen Date: Tue, 1 Sep 2026 07:10:58 +0000 Subject: [PATCH 07/11] refactor(memory): remove dead embedder config knobs and unused API surface - loadLocalEmbedder() had modelsRoot/download options and an offline-inspect-only branch, but its only real caller (getDefaultLocalEmbedder) always passed download:true and no modelsRoot, and nothing else invoked it. Collapsed to a single no-argument path: always resolve the default model directory and ensure the pinned files are present. - storeKnowledgeSemantic/updateKnowledgeSemantic/reembedKnowledge each threaded an unused modelsRoot option through to loadLocalEmbedder; no production caller (api.ts) or test ever passed it. Removed the dead branch, keeping only the `embedder` override used by tests. - LocalEmbedder.dispose() was implemented but never called by any caller (the embedder is process-lifetime in the CLI/MCP server). Removed the unused lifecycle method. - SemanticModelUnavailableError was thrown but never distinguished via instanceof anywhere (catches always fall back to `error.message`). Replaced with a plain Error+cause, since the subclass added no behavior. getSemanticStatus/downloadSemanticModel keep their modelsRoot option: it is exercised by a real test to cover the "model not cached" status without touching the real ~/.ai-devkit/models directory. Co-Authored-By: Claude Sonnet 5 --- .../src/handlers/semantic-maintenance.ts | 15 ++------ packages/memory/src/services/embedder.ts | 37 ++++--------------- .../tests/integration/semantic-search.test.ts | 6 +-- 3 files changed, 14 insertions(+), 44 deletions(-) diff --git a/packages/memory/src/handlers/semantic-maintenance.ts b/packages/memory/src/handlers/semantic-maintenance.ts index c7919736..9277d7c3 100644 --- a/packages/memory/src/handlers/semantic-maintenance.ts +++ b/packages/memory/src/handlers/semantic-maintenance.ts @@ -1,6 +1,6 @@ import { getDatabase } from '../database/index.js'; import { normalizeTags } from '../services/normalizer.js'; -import { getDefaultLocalEmbedder, loadLocalEmbedder, type LocalEmbedder } from '../services/embedder.js'; +import { getDefaultLocalEmbedder, type LocalEmbedder } from '../services/embedder.js'; import { ensureModelFiles, getModelDirectory, inspectModelFiles, MODEL_VERSION } from '../services/model.js'; import { buildEmbeddingText, serializeEmbedding } from '../services/semantic.js'; import { storeKnowledge } from './store.js'; @@ -9,7 +9,6 @@ import type { KnowledgeRow, StoreKnowledgeInput, StoreKnowledgeResult, UpdateKno interface EmbedderOption { embedder?: LocalEmbedder; - modelsRoot?: string; } export interface SemanticStatusResult { @@ -64,9 +63,7 @@ export async function storeKnowledgeSemantic( ): Promise { let value: Buffer; try { - const embedder = options.embedder ?? (options.modelsRoot - ? await loadLocalEmbedder({ modelsRoot: options.modelsRoot, download: true }) - : await getDefaultLocalEmbedder()); + const embedder = options.embedder ?? await getDefaultLocalEmbedder(); const tags = normalizeTags(input.tags ?? []); const vector = await embedder.embed(buildEmbeddingText({ title: input.title, content: input.content, tags })); value = serializeEmbedding(vector); @@ -87,9 +84,7 @@ export async function updateKnowledgeSemantic( if (!existing) return updateKnowledge(input); let value: Buffer; try { - const embedder = options.embedder ?? (options.modelsRoot - ? await loadLocalEmbedder({ modelsRoot: options.modelsRoot, download: true }) - : await getDefaultLocalEmbedder()); + const embedder = options.embedder ?? await getDefaultLocalEmbedder(); const vector = await embedder.embed(buildEmbeddingText({ title: input.title ?? existing.title, content: input.content ?? existing.content, @@ -113,9 +108,7 @@ export async function reembedKnowledge(options: ReembedOptions = {}): Promise !row.embedding || row.embedding_version !== MODEL_VERSION); - const embedder = options.embedder ?? (options.modelsRoot - ? await loadLocalEmbedder({ modelsRoot: options.modelsRoot, download: true }) - : await getDefaultLocalEmbedder()); + const embedder = options.embedder ?? await getDefaultLocalEmbedder(); const batchSize = Math.max(1, options.batchSize ?? 32); let embedded = 0; let failed = 0; diff --git a/packages/memory/src/services/embedder.ts b/packages/memory/src/services/embedder.ts index 6e5ab39a..9ccde754 100644 --- a/packages/memory/src/services/embedder.ts +++ b/packages/memory/src/services/embedder.ts @@ -1,13 +1,6 @@ import { readFile } from 'fs/promises'; import { join } from 'path'; -import { ensureModelFiles, getModelDirectory, inspectModelFiles, normalizeEmbedding } from './model.js'; - -export class SemanticModelUnavailableError extends Error { - constructor(message: string, options?: ErrorOptions) { - super(message, options); - this.name = 'SemanticModelUnavailableError'; - } -} +import { ensureModelFiles, getModelDirectory, normalizeEmbedding } from './model.js'; export function meanPoolAndNormalize( data: Float32Array, @@ -39,38 +32,25 @@ export function meanPoolAndNormalize( export interface LocalEmbedder { embed(text: string): Promise; embedMany(texts: string[]): Promise; - dispose(): Promise; -} - -interface LoadEmbedderOptions { - modelsRoot?: string; - download?: boolean; } let defaultEmbedder: Promise | undefined; /** Reuse the model session across semantic operations in long-lived CLI/MCP processes. */ export function getDefaultLocalEmbedder(): Promise { - defaultEmbedder ??= loadLocalEmbedder({ download: true }).catch(error => { + defaultEmbedder ??= loadLocalEmbedder().catch(error => { defaultEmbedder = undefined; throw error; }); return defaultEmbedder; } -export async function loadLocalEmbedder(options: LoadEmbedderOptions = {}): Promise { - const directory = getModelDirectory(options.modelsRoot); - if (options.download) { - try { - await ensureModelFiles({ directory }); - } catch (error) { - throw new SemanticModelUnavailableError('Semantic model download failed', { cause: error }); - } - } else { - const inspection = await inspectModelFiles(directory); - if (!inspection.ready) { - throw new SemanticModelUnavailableError('Semantic model is not available in the local cache'); - } +export async function loadLocalEmbedder(): Promise { + const directory = getModelDirectory(); + try { + await ensureModelFiles({ directory }); + } catch (error) { + throw new Error('Semantic model download failed', { cause: error }); } const [{ Tokenizer }, ort, tokenizerJson, tokenizerConfig, modelBytes] = await Promise.all([ @@ -108,6 +88,5 @@ export async function loadLocalEmbedder(options: LoadEmbedderOptions = {}): Prom for (const text of texts) results.push(await embed(text)); return results; }, - dispose: async () => session.release(), }; } diff --git a/packages/memory/tests/integration/semantic-search.test.ts b/packages/memory/tests/integration/semantic-search.test.ts index f198eba9..12bb4ad5 100644 --- a/packages/memory/tests/integration/semantic-search.test.ts +++ b/packages/memory/tests/integration/semantic-search.test.ts @@ -7,7 +7,7 @@ import { updateKnowledge } from '../../src/handlers/update'; import { searchKnowledgeHybrid } from '../../src/handlers/semantic-search'; import { MODEL_VERSION } from '../../src/services/model'; import { serializeEmbedding } from '../../src/services/semantic'; -import { SemanticModelUnavailableError, type LocalEmbedder } from '../../src/services/embedder'; +import { type LocalEmbedder } from '../../src/services/embedder'; import { getSemanticStatus, reembedKnowledge, storeKnowledgeSemantic } from '../../src/handlers/semantic-maintenance'; describe('semantic search integration', () => { @@ -17,7 +17,6 @@ describe('semantic search integration', () => { const embedder: LocalEmbedder = { embed: async () => first, embedMany: async () => [first], - dispose: async () => undefined, }; beforeAll(() => getDatabase({ dbPath })); @@ -56,9 +55,8 @@ describe('semantic search integration', () => { it('degrades to lexical results when the local model is unavailable', async () => { const relevant = store('Exact retry policy identifier', 'The WEB-1842 retry rule applies exponential backoff only to transient server failures.'); const unavailable: LocalEmbedder = { - embed: async () => { throw new SemanticModelUnavailableError('offline'); }, + embed: async () => { throw new Error('offline'); }, embedMany: async () => [], - dispose: async () => undefined, }; const result = await searchKnowledgeHybrid({ query: 'WEB-1842', limit: 5 }, { embedder: unavailable }); From 7fe07a32d69288d41c73c539ab7b9be30e42108c Mon Sep 17 00:00:00 2001 From: Hoang Nguyen Date: Tue, 1 Sep 2026 07:11:04 +0000 Subject: [PATCH 08/11] test(cli): cover semantic-enabled memory update command memoryUpdateCommandAsync was imported and mocked but never exercised by any test (flagged by eslint no-unused-vars), leaving the semantic-enabled update path in registerMemoryCommand untested even though its store/search siblings were covered. Co-Authored-By: Claude Sonnet 5 --- packages/cli/src/__tests__/commands/memory.test.ts | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/packages/cli/src/__tests__/commands/memory.test.ts b/packages/cli/src/__tests__/commands/memory.test.ts index 53b1f141..5db49d5d 100644 --- a/packages/cli/src/__tests__/commands/memory.test.ts +++ b/packages/cli/src/__tests__/commands/memory.test.ts @@ -367,6 +367,18 @@ describe('memory command', () => { expect(consoleLogSpy).toHaveBeenCalledWith(JSON.stringify(result, null, 2)); }); + it('awaits semantic-enabled updates before printing their result', async () => { + mockGetMemorySemanticEnabled.mockResolvedValue(true); + const result = { success: true, id: 'semantic-1', message: 'updated' }; + vi.mocked(memoryUpdateCommandAsync).mockResolvedValue(result); + + const program = new Command(); registerMemoryCommand(program); + await program.parseAsync(['node', 'test', 'memory', 'update', '--id', 'semantic-1', '--title', 'Updated title']); + + expect(memoryUpdateCommandAsync).toHaveBeenCalledWith(expect.objectContaining({ id: 'semantic-1', semantic: true })); + expect(consoleLogSpy).toHaveBeenCalledWith(JSON.stringify(result, null, 2)); + }); + it('exposes semantic status, download, and force reembed commands', async () => { vi.mocked(memorySemanticStatusCommand).mockResolvedValue({ modelReady: false, modelDirectory: '/models', embeddingVersion: 'model', total: 0, current: 0, missing: 0, stale: 0, From 90699427fedc5c2922038cbbe14f034faad297ff Mon Sep 17 00:00:00 2001 From: Hoang Nguyen Date: Tue, 1 Sep 2026 07:15:20 +0000 Subject: [PATCH 09/11] docs(memory-semantic): rebuild lifecycle docs per dev-lifecycle phase structure The single ad hoc docs/ai/2026-09-01-feature-memory-semantic.md note did not follow the dev-requirements/dev-design/dev-planning/dev-implementation/ dev-testing phase-doc convention (docs/ai//-.md), and several required sections (testable acceptance criteria, the MiniLM-vs-BGE model-selection gate results, task-level planning status, fresh test-run evidence) were thin or missing. Replaced it with the five phase documents, grounded in what actually shipped after the simplification pass in this branch: - requirements: goals/non-goals, acceptance criteria traceable to the gates and tests that exist, and the frozen model-selection gate table (MiniLM pass / BGE fail, with the actual nDCG@5/recall@5 numbers). - design: architecture, data model, API surface, and the design decisions/trade-offs actually reflected in the code (fail-open degradation, RRF fusion, brute-force cosine with a corpus cap). - planning: full task breakdown by milestone, including this simplification pass as its own milestone. - implementation: file-by-file structure notes and an explicit record of what the simplification pass changed and why. - testing: gate evidence plus fresh build/test/lint/e2e output captured after the simplification commits. `ai-devkit lint --feature memory-semantic` passes against the new structure. Co-Authored-By: Claude Sonnet 5 --- docs/ai/2026-09-01-feature-memory-semantic.md | 79 ---------- .../2026-09-01-feature-memory-semantic.md | 129 +++++++++++++++++ .../2026-09-01-feature-memory-semantic.md | 93 ++++++++++++ .../2026-09-01-feature-memory-semantic.md | 100 +++++++++++++ .../2026-09-01-feature-memory-semantic.md | 110 ++++++++++++++ .../2026-09-01-feature-memory-semantic.md | 135 ++++++++++++++++++ 6 files changed, 567 insertions(+), 79 deletions(-) delete mode 100644 docs/ai/2026-09-01-feature-memory-semantic.md create mode 100644 docs/ai/design/2026-09-01-feature-memory-semantic.md create mode 100644 docs/ai/implementation/2026-09-01-feature-memory-semantic.md create mode 100644 docs/ai/planning/2026-09-01-feature-memory-semantic.md create mode 100644 docs/ai/requirements/2026-09-01-feature-memory-semantic.md create mode 100644 docs/ai/testing/2026-09-01-feature-memory-semantic.md diff --git a/docs/ai/2026-09-01-feature-memory-semantic.md b/docs/ai/2026-09-01-feature-memory-semantic.md deleted file mode 100644 index c6c7c4ac..00000000 --- a/docs/ai/2026-09-01-feature-memory-semantic.md +++ /dev/null @@ -1,79 +0,0 @@ -# Feature: Opt-in hybrid semantic memory search - -## Requirements - -- Keep lexical strict-to-broad FTS as the default and as a protected hybrid channel. -- Enable semantic retrieval only when `memory.semantic` is `true`; absent means false. -- Work locally after a lazy, pinned model download and degrade to lexical-only when offline. -- Preserve deterministic, explainable ranking and warm p95 search latency below 50 ms at 1,000 rows. -- Add only an additive migration; never edit an applied migration. -- Provide semantic status/download and resumable re-embedding commands. -- Keep the embedding runtime below 150 MB excluding the model cache. - -Non-goals: ANN indexing, chunking memories, configurable models/fusion constants, automatic migration-time downloads, and multilingual retrieval. - -## Gate evidence and decisions - -The expanded-100 benchmark contains 100 memories and 100 queries. Against current strict-to-broad FTS, MiniLM hybrid RRF produced paraphrase nDCG@5 0.7590 versus 0.6576 (+15.41%) and recall@5 0.76 versus 0.68 (+11.76%). Overall MRR@5 improved from 0.8637 to 0.9275. Identifier-exact hit@1 stayed 0.96 with zero per-query regressions, and repeated fusion rankings were identical. BGE failed the paraphrase gates (+8.75% nDCG and +5.88% recall), so the smaller MiniLM model is selected. - -`onnxruntime-web@1.22.0` plus `@huggingface/tokenizers@0.1.3` installs at 100 MB on Linux and has no native/platform-specific dependency. Direct ONNX output matched Transformers.js output to float precision. The pinned q8 MiniLM model is a separate 23 MB lazy download. Measured runtime: 426 ms load, 58 ms first inference, and roughly 12-14 ms warm inference. - -## Design - -```mermaid -flowchart LR - Q[Query] --> F[Strict then broad FTS] - Q --> E[Local MiniLM embedder] - E --> C[Brute-force cosine] - F --> R[Deterministic RRF k=60] - C --> R - R --> O[Results and optional explanation] - E -. unavailable/stale/too large .-> L[Lexical-only degradation] -``` - -- Model: `Xenova/all-MiniLM-L6-v2` at revision `751bff37182d3f1213fa05d7196b954e230abad9`, q8, 384 dimensions. -- Cache: `~/.ai-devkit/models/Xenova--all-MiniLM-L6-v2//` with checksummed config, tokenizer, and ONNX files. -- Document text: trimmed title, blank line, trimmed content, then sorted tags when present. -- Storage: nullable float32 little-endian BLOB and an embedding compatibility version. -- Fusion: `1/(60 + lexicalRank) + 1/(60 + semanticRank)`, then lexical presence, lexical rank, semantic rank, and ID tie-breaks. -- Search skips semantic scanning above 5,000 eligible rows. Missing/stale embeddings remain lexical candidates. -- Updating title, content, or tags invalidates an embedding; a scope-only update retains it. -- Backfill processes stable ID-ordered batches and commits each replacement atomically. - -## Interfaces - -- Config: `memory.semantic: boolean`, default false. -- CLI: `memory semantic status`, `memory semantic download`, `memory reembed [--force]`, and `memory search --explain`. -- Search results retain lexical `strategy` (`strict`, `broad`, or `recent`) and add retrieval mode/status. Per-item channel ranks and RRF details appear only with explain enabled. -- Existing synchronous APIs remain lexical-compatible. Async semantic-aware command/server paths dynamically import the runtime only when enabled. - -## Implementation plan - -- [x] Add migration and embedding invalidation/storage tests. -- [x] Add deterministic semantic primitives and RRF tests. -- [x] Add pinned model downloader and lightweight ONNX runtime tests. -- [x] Add status, download, and resumable re-embed APIs and CLI tests. -- [x] Add async hybrid search with offline/corrupt/stale degradation tests; large-corpus validation remains in the performance gate. -- [x] Wire config through CLI and MCP while preserving default behavior. -- [x] Run expanded-100 gate through the built CLI: hit@1 88%, hit@3 97%, hit@5 98%, zero results 0%; one-shot CLI p95 1,886 ms and seed time 76.6 s. -- [x] Run 1,000-row warm latency gate: median 21.85 ms, p95 29.78 ms, max 33.51 ms across 100 searches after five warmups. -- [x] Run six-project build, full tests, lint, and e2e: all six projects built; 1,087 tests and 41 e2e tests passed; lint completed with no errors. - -## Testing record - -- [x] Migration adds nullable columns without changing existing rows. -- [x] Content fields invalidate embeddings; scope-only updates do not. -- [x] Model file checksums and pinned revision are enforced. -- [x] Runtime output is normalized and 384-dimensional. -- [x] RRF is deterministic and protects lexical ties. -- [x] Offline and missing-model search returns lexical results. -- [x] Stale/corrupt embeddings are excluded safely. -- [x] Backfill resumes, skips current rows, and force-rebuilds. -- [x] CLI surfaces config, status, download, reembed, and explanations. -- [x] Default lexical API and strict/broad strategies remain compatible in targeted tests. - -The built-CLI benchmark improved the published 0.57.1 lexical baseline from hit@1 81% / hit@3 91% / hit@5 96% / zero 1% to 88% / 97% / 98% / zero 0%. Judged irrelevant top-three increased from 2.9% to 4.7%, a known semantic-recall trade-off to monitor. The one-shot CLI timings include process and ONNX startup; the warm in-process 1,000-row gate is the search SLA measurement. - -## Rollback - -Set `memory.semantic` to false. The additive columns become inert and older named-column queries continue to work. Deleting a knowledge row deletes its BLOB with no auxiliary index cleanup. Model files can be removed independently from the validated cache directory. A later cleanup migration may null the columns, but rollback does not require reversing schema history. diff --git a/docs/ai/design/2026-09-01-feature-memory-semantic.md b/docs/ai/design/2026-09-01-feature-memory-semantic.md new file mode 100644 index 00000000..e9adf2a3 --- /dev/null +++ b/docs/ai/design/2026-09-01-feature-memory-semantic.md @@ -0,0 +1,129 @@ +--- +phase: design +title: System Design & Architecture +description: Define the technical architecture, components, and data models +--- + +# System Design & Architecture + +## Architecture Overview + +**What is the high-level system structure?** + +```mermaid +flowchart LR + Q[Query] --> F[Strict then broad FTS] + Q --> E[Local MiniLM embedder] + E --> C[Brute-force cosine] + F --> R[Deterministic RRF k=60] + C --> R + R --> O[Results and optional explanation] + E -. unavailable/stale/too large .-> L[Lexical-only degradation] +``` + +- Key components and their responsibilities + - `services/config.ts` (`readSemanticConfig`) — reads `memory.semantic` from `.ai-devkit.json` for the MCP server process; defaults to `false` on any parse/read failure. + - `services/model.ts` — pins model identity/revision/checksums (`MODEL_ID`, `MODEL_REVISION`, `MODEL_FILES`), resolves the on-disk cache directory, verifies file integrity (`inspectModelFiles`), and performs the checksummed atomic download (`ensureModelFiles`). + - `services/embedder.ts` — loads the tokenizer + ONNX session once per process (`getDefaultLocalEmbedder`) and exposes `embed`/`embedMany`; mean-pools and L2-normalizes the model's token-level output into a single vector. + - `services/semantic.ts` — pure retrieval primitives: embedding text construction, BLOB (de)serialization, cosine similarity, and reciprocal-rank fusion (`fuseSearchResults`). + - `handlers/semantic-search.ts` (`searchKnowledgeHybrid`) — orchestrates lexical search + semantic candidate scan + fusion, and decides retrieval mode/status per call. + - `handlers/semantic-maintenance.ts` — semantic-aware store/update (write path), `reembedKnowledge` (resumable backfill), and `getSemanticStatus`/`downloadSemanticModel` (operator commands). + - `server.ts` / `api.ts` — read the config flag once per process/command and route to the semantic-aware or plain lexical handler; the CLI (`packages/cli/src/commands/memory.ts`) reads the same flag via `ConfigManager.getMemorySemanticEnabled()`. +- Technology stack choices and rationale + - `onnxruntime-web` (WASM execution provider, single-threaded) + `@huggingface/tokenizers` for inference: pure-JS/WASM, no native/platform-specific binary, ~100 MB installed on Linux. + - Verified: direct ONNX output matches Transformers.js output to float precision, so no separate compatibility layer was needed. + - SQLite (already the memory store's engine) holds the embedding as a BLOB column rather than a separate vector store, keeping storage and lexical/semantic data in one transactionally-consistent place. + +## Data Models + +**What data do we need to manage?** + +- Core entities + - `knowledge.embedding` — nullable `BLOB`, little-endian `Float32Array` of `MODEL_DIMENSION` (384) values, L2-normalized. + - `knowledge.embedding_version` — nullable `TEXT`, `MODEL_VERSION` string (`${MODEL_ID}@${MODEL_REVISION}:q8:${MODEL_DIMENSION}`) identifying the exact model/quantization/dimension that produced the vector. + - `SemanticCandidate` — an in-memory retrieval candidate (`id, title, content, tags, scope, similarity`), never persisted. + - `SearchRetrievalExplanation` — `{ lexicalRank, semanticRank, semanticSimilarity, rrfScore }`, attached to a result only when `explain: true`. + - `SemanticSearchStatus` — `{ status: 'ready' | 'unavailable' | 'corpus-too-large', embeddingVersion, eligibleCount, reason? }`, always attached to a hybrid search response. +- Data schemas/structures + - Migration `002_semantic_embeddings.sql`: `ALTER TABLE knowledge ADD COLUMN embedding BLOB`, `ALTER TABLE knowledge ADD COLUMN embedding_version TEXT`, plus an index on `embedding_version` to make the eligible-corpus count and eligible-row scan cheap. + - Embedding text (`buildEmbeddingText`): trimmed title, blank line, trimmed content, then (if present) a `Tags: a, b, c` line with tags trimmed, deduped by sort order, so re-embedding the same fields is byte-for-byte deterministic. +- Data flow between components + - Write path: `store`/`update` compute `buildEmbeddingText(...)` → `embedder.embed(...)` → `serializeEmbedding(...)` → written alongside the row in the same SQL statement/transaction as the lexical fields. + - Read path: lexical FTS query and an `embedding_version = current AND embedding IS NOT NULL` row scan run independently, are each ranked, then fused by `fuseSearchResults`. + +## API Design + +**How do components communicate?** + +- External APIs + - None. The only network call is the one-time (per cache miss) HTTPS fetch of the four pinned model files from Hugging Face during `ensureModelFiles`. +- Internal interfaces + - `readSemanticConfig(directory?): { enabled: boolean }` + - `getDefaultLocalEmbedder(): Promise` — memoized per process; `LocalEmbedder = { embed(text), embedMany(texts) }`. + - `searchKnowledgeHybrid(input, { embedder? }): Promise` + - `storeKnowledgeSemantic(input, { embedder? })`, `updateKnowledgeSemantic(input, { embedder? })`, `reembedKnowledge({ embedder?, force?, batchSize? })` + - `getSemanticStatus({ modelsRoot? })`, `downloadSemanticModel({ modelsRoot? })` — the `modelsRoot` override exists purely as a test seam (isolate from the real `~/.ai-devkit/models` cache); it is not exposed through the CLI or config. +- Request/response formats + - Config: `{ "memory": { "semantic": true } }` in `.ai-devkit.json` — any value other than the literal boolean `true` is treated as disabled. + - CLI: `ai-devkit memory semantic status`, `ai-devkit memory semantic download`, `ai-devkit memory reembed [--force]`, `ai-devkit memory search --explain`. + - `SearchKnowledgeResult` gains optional `retrievalMode: 'lexical' | 'hybrid'` and `semantic: SemanticSearchStatus`; per-item `retrieval` is present only with `explain: true`, so default output shape for existing callers is unchanged. +- Authentication/authorization approach + - N/A — local-only feature; the one network fetch is unauthenticated, checksum-verified static file download. + +## Component Breakdown + +**What are the major building blocks?** + +- Backend services/modules (all in `packages/memory/src`) + - `services/config.ts`, `services/model.ts`, `services/embedder.ts`, `services/semantic.ts` + - `handlers/semantic-search.ts`, `handlers/semantic-maintenance.ts` + - `handlers/store.ts` / `handlers/update.ts` — extended to accept an optional pre-computed embedding and to null out a stale embedding when title/content/tags change. +- CLI integration (`packages/cli/src`) + - `lib/Config.ts#getMemorySemanticEnabled`, `commands/memory.ts` — reads the flag once per command and picks the async semantic-aware command vs. the sync lexical one. +- Database/storage layer + - Existing SQLite `knowledge` table, extended additively (migration 002). +- Third-party integrations + - `onnxruntime-web@1.22.0`, `@huggingface/tokenizers@0.1.3` (new direct dependencies of `@ai-devkit/memory`). + +## Design Decisions + +**Why did we choose this approach?** + +- Brute-force cosine over a capped, indexed corpus instead of an ANN index + - Pros: no new storage engine, no index maintenance, deterministic ranking, trivial to reason about and test. + - Cons: doesn't scale past the cap. + - Mitigation: `MAX_SEMANTIC_CORPUS = 5,000` — above that, `searchKnowledgeHybrid` reports `corpus-too-large` and serves lexical-only rather than doing an unbounded scan. Revisiting this cap (e.g., adding an ANN index) is out of scope per the requirements non-goals. +- RRF (`1/(60 + rank)`) fusion instead of a learned or weighted-score blend + - Pros: rank-based, so it is insensitive to the very different score distributions of BM25 vs. cosine similarity; no calibration/tuning surface. + - Cons: less expressive than a tuned weighted blend. + - Tie-breaking is fully deterministic: score → lexical presence → lexical rank → semantic rank → id, so repeated runs over the same data always produce the same order (verified by test). +- Lexical results are always a strict superset baseline (fused, never replaced) + - Pros: any regression in the semantic channel can only add or reorder results within the fused set, not remove a lexical hit. + - Cons: none material — this was the primary reason it passed the "zero per-query identifier-exact regressions" gate. +- Fail-open (degrade to lexical) rather than fail-closed on any semantic error + - Pros: a broken/missing model, corrupt embedding row, or dimension mismatch never turns a working search into an error. + - Cons: silent degradation could mask an operational problem — mitigated by always returning a `semantic.status`/`reason` field so callers (and `--explain`) can see *why* a call fell back. +- One pinned model, no per-project configurability + - Pros: removes an entire class of "which model produced this embedding" bugs and keeps the embedding-version compatibility check trivial (`embedding_version = MODEL_VERSION`). + - Cons: cannot swap models without a code change. + - Alternatives considered: BGE (small) was benchmarked and rejected — see the Model Selection Gate in the requirements doc; it did not clear the paraphrase nDCG@5/recall@5 improvement threshold. +- Patterns applied + - Fail-open degradation at every semantic call site (search, store, update, backfill) rather than a single global "is semantic available" gate, so a corrupt single row or a mid-session network blip degrades only that call. + - Config read at the process/command boundary (`server.ts`, `commands/memory.ts`) rather than threaded as a parameter through every handler, since the flag is process-lifetime, not per-call. + +## Non-Functional Requirements + +**How should the system perform?** + +- Performance targets + - Warm p95 search latency < 50 ms at 1,000 rows with semantic enabled (measured: 29.78 ms). + - Embedding runtime footprint < 150 MB installed, excluding the model cache. +- Scalability considerations + - Explicit corpus scan cap (5,000 eligible rows) with a reported `corpus-too-large` status rather than silent slowdown. + - Backfill (`reembedKnowledge`) processes ID-ordered batches (default 32) and commits each batch in its own transaction, so a large corpus can be re-embedded incrementally and resumed after interruption. +- Security requirements + - Model files are downloaded over HTTPS and verified against pinned SHA-256 checksums and exact byte sizes before use; a checksum mismatch throws rather than silently using a tampered/corrupt file. + - Downloaded files are written to a per-process temp path and atomically renamed into place (`rename` after `writeFile` with `0o600` mode) to avoid partial-file corruption from a crashed download. +- Reliability/availability needs + - Missing model, download failure, corrupt/stale embedding rows, and oversized corpus are all first-class degradation paths with dedicated tests, not incidental catch-alls. + - The additive migration and nullable columns mean semantic search can be disabled at any time (`memory.semantic: false`) with zero impact on lexical search or existing rows. diff --git a/docs/ai/implementation/2026-09-01-feature-memory-semantic.md b/docs/ai/implementation/2026-09-01-feature-memory-semantic.md new file mode 100644 index 00000000..f5639a61 --- /dev/null +++ b/docs/ai/implementation/2026-09-01-feature-memory-semantic.md @@ -0,0 +1,93 @@ +--- +phase: implementation +title: Implementation Guide +description: Technical implementation notes, patterns, and code guidelines +--- + +# Implementation Guide + +## Development Setup + +**How do we get started?** + +- Work in `packages/memory` (retrieval/storage runtime) and `packages/cli` (`memory` command wiring), both TypeScript + Vitest. +- New direct dependencies of `@ai-devkit/memory`: `onnxruntime-web@1.22.0`, `@huggingface/tokenizers@0.1.3`. +- Run focused tests: + - `npm --workspace @ai-devkit/memory test -- tests/unit/semantic.test.ts tests/unit/model.test.ts tests/unit/config.test.ts` + - `npm --workspace @ai-devkit/memory test -- tests/integration/semantic-search.test.ts tests/integration/semantic-storage.test.ts` + - `npm --workspace ai-devkit test -- src/__tests__/commands/memory.test.ts src/__tests__/lib/Config.test.ts` + +## Code Structure + +**How is the code organized?** + +- `packages/memory/src/services/config.ts` — `readSemanticConfig(directory?)`, reads `.ai-devkit.json`. +- `packages/memory/src/services/model.ts` — `MODEL_ID`/`MODEL_REVISION`/`MODEL_DIMENSION`/`MODEL_VERSION`/`MODEL_FILES`, `getModelDirectory`, `inspectModelFiles`, `ensureModelFiles`, `normalizeEmbedding`. +- `packages/memory/src/services/embedder.ts` — `LocalEmbedder` (`embed`, `embedMany`), `getDefaultLocalEmbedder` (memoized per process), `loadLocalEmbedder`, `meanPoolAndNormalize`. +- `packages/memory/src/services/semantic.ts` — `buildEmbeddingText`, `serializeEmbedding`/`deserializeEmbedding`, `cosineSimilarity`, `fuseSearchResults`, `EMBEDDING_DIMENSION`, `RRF_K`. +- `packages/memory/src/handlers/semantic-search.ts` — `searchKnowledgeHybrid`. +- `packages/memory/src/handlers/semantic-maintenance.ts` — `getSemanticStatus`, `downloadSemanticModel`, `storeKnowledgeSemantic`, `updateKnowledgeSemantic`, `reembedKnowledge`. +- `packages/memory/src/handlers/store.ts` / `update.ts` — accept an optional `{ value, version }` embedding argument; `update.ts` nulls both columns whenever title/content/tags change and no replacement embedding is supplied. +- `packages/memory/src/database/migrations/002_semantic_embeddings.sql` — additive columns + index. +- `packages/memory/src/server.ts` — reads `readSemanticConfig().enabled` once at server construction and routes each MCP tool call to the semantic or plain handler. +- `packages/memory/src/api.ts` — exports `*Async` command wrappers (`memorySearchCommandAsync`, `memoryStoreCommandAsync`, `memoryUpdateCommandAsync`) plus `memorySemanticStatusCommand`, `memoryDownloadSemanticCommand`, `memoryReembedCommand`. +- `packages/cli/src/lib/Config.ts` — `getMemorySemanticEnabled()` reads `memory.semantic === true` from the project `.ai-devkit.json`. +- `packages/cli/src/commands/memory.ts` — `registerMemoryCommand` picks the async/semantic-aware command when `getMemorySemanticEnabled()` resolves `true`; adds the `memory semantic status|download` and `memory reembed [--force]` subcommands and `--explain` on `memory search`. + +## Implementation Notes + +**Key technical details to remember:** + +### Core Features +- Fusion: `score = 1/(RRF_K + lexicalRank) + 1/(RRF_K + semanticRank)` with `RRF_K = 60`; ties break on lexical presence, then lexical rank, then semantic rank, then id — fully deterministic (`fuseSearchResults`, `packages/memory/src/services/semantic.ts:62`). +- Degradation is per-call, not a single global switch: `searchKnowledgeHybrid` catches any embedder/model error and returns `retrievalMode: 'lexical'` with a `semantic.status`/`reason` explaining why, rather than throwing. +- Corpus scan cap: `MAX_SEMANTIC_CORPUS = 5_000` — above that count of eligible (`embedding_version` current, non-null) rows, the handler returns `corpus-too-large` and never touches the embedder for that call. +- Embedding invalidation: `updateKnowledge` nulls `embedding`/`embedding_version` whenever `title`, `content`, or `tags` is provided in the update, unless a freshly computed replacement embedding is passed in by `updateKnowledgeSemantic`. A scope-only update leaves the existing embedding untouched. +- The embedder session (tokenizer + ONNX `InferenceSession`) is created once per process and memoized (`getDefaultLocalEmbedder`) so repeated CLI/MCP calls in one process reuse it instead of re-loading ~23 MB of model weights per call. + +### Patterns & Best Practices +- Config is read once at the process/command boundary (`server.ts`, `commands/memory.ts`), not threaded as a parameter through every handler — the flag is process-lifetime, not per-call state. +- Every handler that can use semantic search/write takes an optional `{ embedder? }` override; this is the only DI seam, used exclusively by tests to inject deterministic fixed-vector embedders instead of loading the real ONNX runtime. +- Model files are downloaded to a per-process temp path (`${destination}.tmp-${process.pid}`) and atomically renamed into place, so a crashed/interrupted download can never leave a partially-written model file that would pass a later existence check but fail inference. + +### Simplification pass (post-functional-completion) +Applied after all eval gates and functional tests were already green, so none of it touches ranking, degradation, or migration behavior: +- Removed `GlobalDevKitConfig.memory.semantic` — added by mistake alongside the per-project field; nothing in `GlobalConfig.ts` or anywhere else ever read it. +- Collapsed `loadLocalEmbedder()` from a `{ modelsRoot?, download? }`-configurable function to a zero-argument one. Its only real caller (`getDefaultLocalEmbedder`) always passed `download: true` and never passed `modelsRoot`; the `download: false` branch (inspect-only, throw if not ready) was unreachable in shipped code, and `storeKnowledgeSemantic`/`updateKnowledgeSemantic`/`reembedKnowledge` all threaded a `modelsRoot` option that no caller (production or test) ever set. +- Removed `LocalEmbedder.dispose()` — implemented (`session.release()`) but never invoked anywhere; the embedder is process-lifetime by design (CLI processes are short-lived and exit; the MCP server holds one memoized session for its life), so there was no call site that could ever exercise cleanup. +- Replaced `SemanticModelUnavailableError` with a plain `Error` (still carrying `cause`) — the subclass was thrown but never checked with `instanceof` anywhere; every catch site already falls back to `error.message`/`String(error)` generically. +- `getSemanticStatus`/`downloadSemanticModel` kept their `{ modelsRoot? }` option: unlike the removed cases, it is genuinely exercised by a test (`reports model and embedding readiness without loading the runtime`) to assert the "model not cached" status without touching the real `~/.ai-devkit/models` directory or requiring a mocked filesystem. +- Added a CLI test for the semantic-enabled `memory update` path, which had an unused mock import (`memoryUpdateCommandAsync`, flagged by `eslint no-unused-vars`) because the corresponding behavior — symmetric to the already-tested store/search paths — had no test. + +## Integration Points + +**How do pieces connect?** + +- MCP server (`server.ts`): `createServer()` reads the config flag once, then each tool handler (`memory_storeKnowledge`, `memory_updateKnowledge`, `memory_searchKnowledge`) branches to the semantic-aware or plain handler based on that flag. +- CLI (`commands/memory.ts`): each `memory store|update|search` action calls `resolveSemanticEnabled()` (via `ConfigManager`) per invocation and picks the async or sync command function; `memory semantic status|download` and `memory reembed` always go through the async semantic-maintenance API regardless of the flag (they are explicit operator actions). +- `api.ts` is the seam CLI commands go through; it owns `getDatabase`/`closeDatabase` lifecycle per command so each CLI invocation opens and closes its own database handle. + +## Error Handling + +**How do we handle failures?** + +- Semantic search failure of any kind → lexical-only result set with `semantic.status: 'unavailable'` and a `reason` string; never an error response, never a thrown exception from `searchKnowledgeHybrid`. +- Semantic write failure (embedder unavailable) → `storeKnowledgeSemantic`/`updateKnowledgeSemantic` fall back to the plain (non-semantic) `storeKnowledge`/`updateKnowledge` call, so the memory is still saved without an embedding rather than failing the write. +- Corrupt embedding row (wrong byte length) or dimension mismatch during a similarity comparison → that row is skipped from semantic candidates; lexical results for the same row remain unaffected. +- Model download checksum/size mismatch → throws loudly (does not silently accept a bad file); a `memory semantic download` invocation surfaces this as a command failure rather than caching a corrupt file. + +## Performance Considerations + +**How do we keep it fast?** + +- Warm p95 search latency at 1,000 rows: 29.78 ms (budget: <50 ms); see the testing doc for the full benchmark record. +- The embedder session is memoized per process; only the very first semantic operation in a process pays tokenizer/ONNX load cost. +- The corpus scan cap keeps the cosine-similarity pass bounded regardless of memory-store growth, at the cost of falling back to lexical-only past the cap (explicit, reported, and out of scope to change per the requirements non-goals). + +## Security Notes + +**What security measures are in place?** + +- Model files are fetched over HTTPS and verified against pinned SHA-256 checksums and exact byte sizes before being used or considered cached; a mismatch throws instead of using the file. +- Downloaded files are written to a per-process temp path and atomically renamed into place, avoiding a window where a partially-downloaded file could be read as "ready." +- No embedding content or query text ever leaves the local machine — inference runs entirely in-process via `onnxruntime-web`'s WASM backend. diff --git a/docs/ai/planning/2026-09-01-feature-memory-semantic.md b/docs/ai/planning/2026-09-01-feature-memory-semantic.md new file mode 100644 index 00000000..f32090eb --- /dev/null +++ b/docs/ai/planning/2026-09-01-feature-memory-semantic.md @@ -0,0 +1,100 @@ +--- +phase: planning +title: Project Planning & Task Breakdown +description: Break down work into actionable tasks and estimate timeline +--- + +# Project Planning & Task Breakdown + +## Milestones + +**What are the major checkpoints?** + +- [x] Milestone 1: Model selection gate run and decided (MiniLM passed, BGE rejected — see requirements doc) +- [x] Milestone 2: Storage, embedder runtime, and retrieval primitives implemented and unit-tested +- [x] Milestone 3: Hybrid search, maintenance commands, and CLI/MCP wiring implemented and integration-tested +- [x] Milestone 4: Full-suite gates (build, test, lint, e2e) green and benchmark evidence captured +- [x] Milestone 5: Post-merge-readiness simplification pass — remove speculative config surface and dead code before merge + +## Task Breakdown + +**What specific work needs to be done?** + +### Phase 1: Foundation +- [x] Task 1.1: Benchmark MiniLM vs. BGE against the expanded-100 gate; select MiniLM (`c5c750c`, `packages/memory/src/services/model.ts`) +- [x] Task 1.2: Add additive migration `002_semantic_embeddings.sql` (nullable `embedding`/`embedding_version`, index) with a test asserting existing rows are untouched +- [x] Task 1.3: Add `services/config.ts` (`readSemanticConfig`) defaulting to disabled on any missing/malformed config + +### Phase 2: Core Retrieval +- [x] Task 2.1: Add `services/semantic.ts` primitives — embedding text builder, BLOB (de)serialization, cosine similarity, deterministic RRF fusion — with unit tests for determinism and tie-breaking +- [x] Task 2.2: Add `services/model.ts` (pinned identity/checksums, atomic checksummed download) and `services/embedder.ts` (tokenizer + ONNX session, mean-pool + normalize) with unit tests for checksum enforcement and pooling correctness +- [x] Task 2.3: Add `handlers/semantic-search.ts` (`searchKnowledgeHybrid`) with offline/corrupt/stale/oversized-corpus degradation tests + +### Phase 3: Write Path & Maintenance +- [x] Task 3.1: Extend `handlers/store.ts`/`handlers/update.ts` to persist an optional embedding and invalidate it on content/tag/title changes (retain it on scope-only updates) +- [x] Task 3.2: Add `handlers/semantic-maintenance.ts` — semantic-aware store/update, `getSemanticStatus`, `downloadSemanticModel`, resumable `reembedKnowledge` +- [x] Task 3.3: Wire `memory.semantic` config through `server.ts` (MCP) and `api.ts`/`commands/memory.ts` (CLI), preserving default (disabled) behavior exactly + +### Phase 4: Validation +- [x] Task 4.1: Run the expanded-100 benchmark through the built CLI and record hit@1/3/5, zero-result rate, and timings +- [x] Task 4.2: Run the 1,000-row warm latency gate against the search SLA (<50 ms p95) +- [x] Task 4.3: Run six-project build, full test suite, lint, and e2e across the monorepo +- [x] Task 4.4: `await`-harden semantic writes and reuse embedder sessions across calls (`7686a7d`, `079bfce`) + +### Phase 5: Pre-merge simplification (this pass) +- [x] Task 5.1: Audit the diff for speculative config surface, unused options, and dead guard/error paths against the `simplify-implementation` discipline +- [x] Task 5.2: Remove `GlobalDevKitConfig.memory.semantic` (never read — global config has no semantic reader) +- [x] Task 5.3: Collapse `loadLocalEmbedder`'s unused `modelsRoot`/`download` options and the dead offline-inspect branch; remove the never-invoked `LocalEmbedder.dispose()`; replace the never-narrowed `SemanticModelUnavailableError` with a plain `Error` +- [x] Task 5.4: Close the CLI test gap on the semantic-enabled `memory update` path (previously an unused mock import, flagged by lint) +- [x] Task 5.5: Rebuild the five lifecycle phase docs (this document and its siblings) against the simplified, shipped state, replacing the single ad hoc `docs/ai/2026-09-01-feature-memory-semantic.md` note + +## Dependencies + +**What needs to happen in what order?** + +- Task dependencies and blockers + - Model selection (1.1) had to complete before any embedder/runtime code (2.2) was written, since the pinned checksums and dimension are model-specific. + - Migration (1.2) had to land before the write path (3.1) could persist embeddings. + - Storage/embedder primitives (2.1-2.2) had to exist before hybrid search (2.3) and maintenance (3.2) could consume them. + - The simplification pass (5.x) was deliberately sequenced *after* functional completion (4.x) so eval-passing behavior (fusion, degradation, migration safety) was never at risk while removing speculative surface. +- External dependencies + - One-time HTTPS fetch of the pinned model files from Hugging Face (`huggingface.co`) — required only for `memory semantic download` or the first semantic operation, not for build/lint/most tests. +- Team/resource dependencies + - None beyond the implementer; this is a self-contained package-level feature. + +## Timeline & Estimates + +**When will things be done?** + +Delivered in a single feature branch (`feature-memory-semantic`) across five commits before this simplification pass (`c5c750c` storage primitives, `7f529bc` hybrid search wiring, `7686a7d` await/fallback hardening, `079bfce` session reuse) plus the three simplification commits in this pass. No further phases are planned unless testing or review surfaces a gap. + +## Risks & Mitigation + +**What could go wrong?** + +- Technical risks + - A future corpus could exceed the 5,000-row scan cap in normal use, silently falling back to lexical-only. Mitigated by surfacing `corpus-too-large` explicitly in the response and `--explain` output rather than degrading silently. + - The pinned model could become unavailable at its Hugging Face revision URL. Mitigated by checksum verification (fails loudly, not silently, on a bad download) and by the fact the runtime itself never redownloads once cached. +- Resource risks + - None currently — the feature is self-contained within `@ai-devkit/memory` and `@ai-devkit/cli`. +- Dependency risks + - `onnxruntime-web`/`@huggingface/tokenizers` version drift could change inference output. Mitigated by pinning exact versions in `package.json` and verifying ONNX output against Transformers.js output during model selection. +- Mitigation strategies already in place + - Every semantic failure mode (missing model, download failure, corrupt embedding, dimension mismatch, oversized corpus) has a dedicated integration test asserting lexical fallback rather than an error. + +## Resources Needed + +**What do we need to succeed?** + +- Tools and services: existing TypeScript/Vitest workspace tooling; no new CI infrastructure required. +- Documentation/knowledge: this planning doc plus its requirements/design/implementation/testing siblings are now the source of truth, replacing the prior single ad hoc note. + +## Progress Summary + +All functional milestones (1-4) were complete and gate-evidenced before this pass began. This pass (Milestone 5) audited the shipped diff against the repository's simplification discipline, removed four pieces of speculative/dead surface (an unread global-config field, unused embedder options and an unreachable degradation branch, an unused error subclass, and an unused lifecycle method), closed one CLI test gap that lint had flagged, and rebuilt the lifecycle documentation set to match the phase-by-phase discipline the docs were missing. No eval-passing behavior (RRF fusion, degradation paths, migration safety, or the MiniLM model/thresholds) was changed. + +## Next Focus + +- Update the pull request description to link these five phase docs and summarize lifecycle conformance. +- Re-run the full gate suite (build, test, lint, e2e) after the simplification commits and confirm green before requesting review. +- No open implementation tasks remain; proceed to `dev-review`. diff --git a/docs/ai/requirements/2026-09-01-feature-memory-semantic.md b/docs/ai/requirements/2026-09-01-feature-memory-semantic.md new file mode 100644 index 00000000..00799246 --- /dev/null +++ b/docs/ai/requirements/2026-09-01-feature-memory-semantic.md @@ -0,0 +1,110 @@ +--- +phase: requirements +title: Requirements & Problem Understanding +description: Clarify the problem space, gather requirements, and define success criteria +--- + +# Requirements & Problem Understanding + +## Problem Statement + +**What problem are we solving?** + +- The `@ai-devkit/memory` search is lexical-only (SQLite FTS5 + BM25). Paraphrased queries that do not share vocabulary with a stored memory ("wire contracts" vs. a memory titled "Response serialization boundary") often fail to surface the right result, even though a semantic match exists. +- Agents querying memory before implementing a feature therefore miss relevant prior decisions whenever the phrasing differs from how the memory was written, which reduces the value of the memory system's core promise: don't repeat past guidance. +- Who is affected: + - AI coding agents calling `memory_searchKnowledge` via MCP, whose recall quality directly gates whether they reuse stored conventions instead of re-deciding them. + - CLI users running `ai-devkit memory search` who rely on ranked results being complete. + +## Goals & Objectives + +**What do we want to achieve?** + +- Primary goals + - Add an opt-in semantic retrieval channel that improves recall for paraphrased/synonym queries without weakening exact-match (identifier, error-code) retrieval. + - Keep lexical strict-to-broad FTS as the default and as a protected channel inside the fused ranking — semantic search only ever adds to lexical results, never replaces them. + - Run entirely locally: no network calls per query, no data leaving the machine. + - Degrade to lexical-only automatically whenever the semantic path is unavailable (model not downloaded, corrupt cache, offline, oversized corpus) — semantic search must never turn a working lexical query into a failure. +- Secondary goals + - Provide operator commands to inspect and manage the local model/embedding cache (`memory semantic status`, `memory semantic download`, `memory reembed [--force]`). + - Provide an `--explain` mode exposing lexical rank, semantic rank, similarity, and fusion score for debugging ranking behavior. +- Non-goals (explicitly out of scope) + - ANN/vector-index infrastructure (brute-force cosine over a capped corpus is sufficient at current memory-store scale). + - Chunking long memory content into multiple embedded passages. + - Configurable embedding models or fusion constants — one pinned model, one fusion formula. + - Downloading the model automatically during `ai-devkit init`/migration; the download is lazy and triggered by first semantic use or an explicit `memory semantic download`. + - Multilingual retrieval quality guarantees. + +## User Stories & Use Cases + +**How will users interact with the solution?** + +- As an agent calling `memory_searchKnowledge`, I want paraphrased queries to still surface the relevant stored convention, so I don't re-derive or contradict a decision that already exists. +- As a developer, I want semantic search off by default so existing projects see no behavior change, latency change, or new dependency download unless I opt in. +- As a developer who opts in, I want the first semantic operation to lazily fetch a small pinned model so I don't have to run a separate setup step, but I also want an explicit `memory semantic download` for pre-warming CI/sandboxed environments without network access at query time. +- As a developer troubleshooting ranking, I want `memory search --explain` to show me why a result ranked where it did (lexical rank, semantic rank, cosine similarity, RRF score). +- Key workflows and scenarios + - Enable via `.ai-devkit.json` (`memory.semantic: true`) → first search/store triggers model download → subsequent operations reuse the cached model and a warm in-process session. + - Existing memory rows without an embedding remain fully searchable via lexical FTS; `memory reembed` backfills them in resumable batches. + - Editing a memory's title/content/tags invalidates its embedding (must be recomputed); editing only its scope does not. +- Edge cases to consider + - Semantic model not yet downloaded, download fails, or files are corrupted/checksum-mismatched. + - Embedding BLOB is present but has the wrong byte length (corrupt row) or was computed by a since-replaced model version (stale). + - Corpus grows large enough that brute-force cosine scanning would be too slow to keep search fast. + - Query and memory rows both exist, but the model is offline mid-session (should degrade per-call, not crash the server). + +## Success Criteria + +**How will we know when we're done?** + +- Measurable outcomes (from the expanded-100 benchmark: 100 memories, 100 queries, run through the built CLI) + - Paraphrase nDCG@5 improves from 0.6576 to 0.7590 (+15.41%). + - Paraphrase recall@5 improves from 0.68 to 0.76 (+11.76%). + - Overall MRR@5 improves from 0.8637 to 0.9275. + - Identifier-exact hit@1 stays at 0.96 with zero per-query regressions versus lexical-only. + - Fusion ranking is deterministic: repeated runs over the same corpus/query produce identical rankings and scores. +- Acceptance criteria + - `memory.semantic` config flag defaults to `false`; absent, non-boolean, or malformed config is treated as `false` (never silently enabled). + - `memory_searchKnowledge` / `memory search` return `retrievalMode: 'hybrid'` and a `semantic` status object (`ready` | `unavailable` | `corpus-too-large`, `embeddingVersion`, `eligibleCount`, optional `reason`) whenever semantic is enabled. + - Any semantic failure (model missing, download failure, corrupt embedding, corpus over the scan limit) falls back to lexical results for that call — never an error response, never a dropped result set. + - Storing/updating a memory with `memory.semantic` enabled writes a compatible embedding in the same operation; disabling semantic afterward leaves lexical search fully functional. + - `memory semantic status`, `memory semantic download`, and `memory reembed [--force]` are available and covered by tests. + - Only an additive schema migration is used (nullable `embedding` BLOB + `embedding_version` TEXT); no existing migration is edited. + - Warm p95 search latency stays under 50 ms at 1,000 rows with semantic enabled. + - Embedding runtime (excluding the downloaded model cache) stays under 150 MB installed. +- Performance benchmarks + - 1,000-row warm latency gate (100 searches after 5 warmups): median 21.85 ms, p95 29.78 ms, max 33.51 ms — under the 50 ms budget. + - One-shot built-CLI benchmark (includes process + ONNX cold start): p95 1,886 ms; seed time 76.6 s for the 100-memory corpus. This process-startup cost is expected and is not the search-latency SLA measurement. + +## Constraints & Assumptions + +**What limitations do we need to work within?** + +- Technical constraints + - Must run offline after the model is cached; no per-query network dependency. + - Must not add a native/platform-specific runtime dependency (rules out most native ONNX/BLAS bindings). + - Must stay additive to the existing SQLite schema and lexical search path. +- Business constraints + - No regression to the default (semantic-disabled) experience: same latency, same dependencies installed-and-loaded, same output shape. +- Time/budget constraints + - Reuse the existing FTS ranking (`searchKnowledge`) as the lexical channel rather than building a second lexical engine. +- Assumptions we're making + - Current and near-future memory-store sizes stay well under the corpus scan cap (5,000 rows); beyond that, semantic degrades to lexical-only by design rather than requiring an ANN index. + - A single pinned quantized model, chosen and frozen by the model-selection gate below, is sufficient; per-project model choice is not needed. + +## Model Selection Gate (evidence) + +Two candidate local embedding models were evaluated head-to-head against the expanded-100 benchmark before implementation proceeded, using the paraphrase nDCG@5 and recall@5 improvement over lexical-only FTS as the pass/fail gate: + +| Model | Paraphrase nDCG@5 delta | Paraphrase recall@5 delta | Gate result | +|---|---|---|---| +| `Xenova/all-MiniLM-L6-v2` (q8, 384-dim) | +15.41% | +11.76% | **Pass** — selected | +| BGE (small) | +8.75% | +5.88% | **Fail** — below the required improvement threshold | + +MiniLM was selected because it cleared the paraphrase-improvement gate with margin and BGE did not; runtime footprint was a secondary tie-breaker (MiniLM's pinned q8 model is a 23 MB download vs. a larger BGE checkpoint). This decision is frozen: non-goals above explicitly exclude configurable models, so re-litigating the model choice requires a new requirements pass, not a config change. + +## Questions & Open Items + +**What do we still need to clarify?** + +- None outstanding. The corpus-size ceiling (5,000 rows), fusion constant (RRF k=60), and model pin are treated as accepted defaults per the non-goals above; revisiting any of them is a new requirement, not an open item on this feature. diff --git a/docs/ai/testing/2026-09-01-feature-memory-semantic.md b/docs/ai/testing/2026-09-01-feature-memory-semantic.md new file mode 100644 index 00000000..3bac9c11 --- /dev/null +++ b/docs/ai/testing/2026-09-01-feature-memory-semantic.md @@ -0,0 +1,135 @@ +--- +phase: testing +title: Testing Strategy +description: Define testing approach, test cases, and quality assurance +--- + +# Testing Strategy + +## Test Coverage Goals + +**What level of testing do we aim for?** + +- Unit coverage for every pure retrieval/storage primitive (fusion, similarity, (de)serialization, embedding text construction, checksum/pin verification, mean-pooling). +- Integration coverage for every degradation path (offline model, corrupt embedding, stale embedding version, oversized corpus) and for the migration, write-invalidation, and backfill behaviors. +- Command-level coverage for the CLI's semantic-enabled vs. disabled branching, including the operator commands (`semantic status`, `semantic download`, `reembed --force`). +- Offline benchmark coverage (expanded-100, 1,000-row latency) as the acceptance-criteria evidence for the requirements doc's measurable outcomes — these are run manually against the built CLI rather than as part of the automated suite, since they require the real ONNX runtime and a seeded large corpus. + +## Unit Tests + +**What individual components need testing?** + +### `services/semantic.ts` (`tests/unit/semantic.test.ts`) +- [x] Builds deterministic embedding text with sorted, trimmed tags. +- [x] Round-trips a 384-dim Float32Array through BLOB (de)serialization and rejects the wrong dimension. +- [x] Cosine similarity is correct for normalized vectors. +- [x] RRF fusion is deterministic across repeated runs and protects a shared lexical/semantic hit's lexical rank on ties. + +### `services/model.ts` / `services/embedder.ts` (`tests/unit/model.test.ts`) +- [x] Pins model identity, revision, quantization, and dimension (`MODEL_ID`, `MODEL_REVISION` format, `MODEL_DIMENSION`, `MODEL_VERSION`). +- [x] `inspectModelFiles` correctly reports missing files; `ensureModelFiles` downloads and verifies them atomically, and the cache is recognized as ready afterward. +- [x] `ensureModelFiles` rejects a download whose bytes don't match the pinned checksum. +- [x] `normalizeEmbedding` normalizes correctly and rejects an all-zero vector. +- [x] `meanPoolAndNormalize` pools only attention-unmasked token vectors. + +### `services/config.ts` (`tests/unit/config.test.ts`) +- [x] Defaults to disabled for missing or malformed `.ai-devkit.json`. +- [x] Enables semantics only for the exact boolean `true` (rejects the string `"true"`). + +### CLI config (`packages/cli/src/__tests__/lib/Config.test.ts`) +- [x] `getMemorySemanticEnabled()` returns `false` by default, `true` only for `{ memory: { semantic: true } }`, and `false` for `{ memory: { semantic: 'true' } }`. + +## Integration Tests + +**How do we test component interactions?** + +### Migration (`tests/integration/semantic-storage.test.ts`) +- [x] Migration 002 adds nullable `embedding`/`embedding_version` columns without altering existing rows or the schema version sequence. + +### Hybrid search and maintenance (`tests/integration/semantic-search.test.ts`) +- [x] Retrieves a semantic-only paraphrase match and exposes `--explain`-style rank/score details. +- [x] Degrades to lexical results and reports `status: 'unavailable'` when the embedder throws. +- [x] `updateKnowledge` invalidates an embedding on a content/tag change but retains it for a scope-only update. +- [x] `reembedKnowledge` backfills missing embeddings, skips already-current rows on a repeat run, and force-recomputes everything with `--force`. +- [x] `storeKnowledgeSemantic` writes a compatible embedding in the same operation as the memory row. +- [x] `getSemanticStatus` reports `modelReady: false` without loading the runtime when the model cache directory is missing (via the `modelsRoot` test seam). +- [x] A corrupt embedding (wrong byte length) is skipped from semantic candidates without losing the lexical hit. +- [x] Above `MAX_SEMANTIC_CORPUS` (5,000) eligible rows, search reports `corpus-too-large`, serves lexical-only, and never calls the embedder (asserted via a spy). + +### CLI command wiring (`packages/cli/src/__tests__/commands/memory.test.ts`) +- [x] `memory search --explain` uses the async hybrid path and returns `retrievalMode`/`semantic` fields when semantic is enabled. +- [x] `memory store` awaits the semantic-enabled async command before printing its result. +- [x] `memory update` awaits the semantic-enabled async command before printing its result (added in this pass — previously untested; the mock import was flagged unused by lint). +- [x] `memory semantic status`, `memory semantic download`, and `memory reembed --force` invoke their respective API functions with the resolved `dbPath`/`force` options. +- [x] Existing lexical-only command behavior (semantic disabled) is unchanged — covered by the pre-existing store/search/update test cases in the same file. + +## End-to-End Tests + +**What user flows need validation?** + +- [x] Full CLI e2e suite (`e2e/vitest.config.ts`) exercises `ai-devkit` commands end-to-end, including memory commands, against the built CLI binary — see Fresh Validation Output below. +- [ ] Manual E2E run of `memory semantic download` against a real network to confirm the live Hugging Face fetch and checksum path (covered functionally by unit tests against a fake `fetchImpl`; not re-run in this pass since the model/thresholds are frozen and unchanged). + +## Test Data + +**What data do we use for testing?** + +- Deterministic fixed-vector `LocalEmbedder` fakes (e.g., `first`/`second` one-hot 384-dim vectors) for all fusion/degradation/backfill tests, so semantic ranking assertions never depend on the real model's actual embedding values. +- A synthetic 5,001-row corpus (bulk-inserted directly via the DB connection) to exercise the `corpus-too-large` path without needing a real large memory store. +- The expanded-100 benchmark corpus (100 memories, 100 queries, including paraphrase and exact-identifier query types) for the offline gate evidence below. + +## Test Reporting & Coverage + +**How do we verify and communicate test results?** + +### Model selection gate (frozen; not re-run in this pass, per the requirements doc's non-goals) + +| Model | Paraphrase nDCG@5 | Paraphrase recall@5 | Result | +|---|---|---|---| +| Lexical-only baseline | 0.6576 | 0.68 | — | +| MiniLM hybrid (RRF) | 0.7590 (+15.41%) | 0.76 (+11.76%) | **Pass — selected** | +| BGE hybrid (RRF) | +8.75% delta | +5.88% delta | **Fail — rejected** | + +Overall MRR@5 improved 0.8637 → 0.9275. Identifier-exact hit@1 held at 0.96 with zero per-query regressions; repeated fusion runs produced identical rankings (determinism verified). + +### Built-CLI benchmark (frozen; expanded-100 corpus) + +Published 0.57.1 lexical baseline (hit@1 81% / hit@3 91% / hit@5 96% / zero-result 1%) improved to MiniLM hybrid hit@1 88% / hit@3 97% / hit@5 98% / zero-result 0%. Judged-irrelevant top-three rose from 2.9% to 4.7% (an accepted semantic-recall trade-off, tracked as a monitoring note rather than a regression, per the design doc's fail-open philosophy). One-shot CLI p95 was 1,886 ms and seed time 76.6 s for the 100-memory corpus (process/ONNX cold-start cost, not the search-latency SLA). + +### 1,000-row warm latency gate (frozen) + +100 searches after 5 warmups: median 21.85 ms, p95 29.78 ms, max 33.51 ms — under the <50 ms budget from the requirements doc. + +### Fresh validation output (this pass, run against the simplified code) + +Executed after the simplification commits in this pass, before any doc-only changes: + +- `npm run build` — 6/6 projects built successfully (channel-connector, memory, agent-manager, task-manager, memory-dashboard, cli). +- `npm test` (workspace-wide) — all suites green: + - `@ai-devkit/memory`: 18 test files, 156 tests passed. + - `@ai-devkit/task-manager`: 8 test files, 112 tests passed. + - `@ai-devkit/agent-manager`: 41 test files, 631 tests passed. + - `@ai-devkit/memory-dashboard`: 3 test files, 22 tests passed. + - `ai-devkit` (cli): 91 test files, 1,088 tests passed (was 1,087 before this pass's added `memory update` semantic test). +- `npm run lint` (workspace-wide) — 0 errors; 3 pre-existing warnings unrelated to this feature (`init.test.ts`, `channel.ts`, `util/skill.ts`); the pre-existing `memoryUpdateCommandAsync` unused-import warning in `memory.test.ts` is resolved by this pass's added test. +- `npx vitest run --config e2e/vitest.config.ts` — 1 test file, 41 tests passed. + +No eval-gate number above changed as a result of this pass: the simplification commits touched only unused config/API surface (a dead global-config field, unused embedder options and an unreachable branch, an unused error subclass, an unused lifecycle method) and added one missing test; they did not touch `fuseSearchResults`, the degradation paths in `searchKnowledgeHybrid`, the migration, or the pinned model/thresholds. + +## Manual Testing + +**What requires human validation?** + +- Confirm `memory semantic download` output/UX against a live network in an environment without a pre-warmed cache (not required for this pass; frozen model/checksums make this low-risk). + +## Performance Testing + +**How do we validate performance?** + +- Benchmarks are run manually via the built CLI (expanded-100 and 1,000-row corpora) rather than as part of the automated test suite, since they require the real ONNX runtime, a downloaded model, and a seeded corpus of realistic size. Re-run these benchmarks if the model, fusion constant, or corpus cap ever changes (none did in this pass). + +## Bug Tracking + +**How do we manage issues?** + +- The one known, accepted trade-off (judged-irrelevant top-three results rising from 2.9% to 4.7% under semantic hybrid search) is tracked here as a monitoring note rather than a tracked bug, since it was an explicit, gate-evidenced trade-off accepted at model-selection time, not a defect. From 0444fedcbd074b89cad515d8af28f216a875b37b Mon Sep 17 00:00:00 2001 From: Hoang Nguyen Date: Tue, 1 Sep 2026 08:11:06 +0000 Subject: [PATCH 10/11] feat(cli): resolve memory.semantic with project > global > default precedence MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reinstates GlobalDevKitConfig.memory.semantic, which a prior pass on this branch removed as dead code. Per user review, the field was unwired rather than unwanted: a developer who wants semantic search on for every project should be able to set it once in the global config (~/.ai-devkit/.ai-devkit.json) instead of repeating `"memory": { "semantic": true }` in every project's .ai-devkit.json. getMemorySemanticEnabled() is the single site every semantic-aware command/server path already calls, so the resolution lives there: an explicit boolean project value (true or false) wins outright; only when the project value is absent or non-boolean does it fall back to an explicit `true` in the global config; otherwise it defaults to false. No new config surface — same boolean, one more file, resolved at the existing call site. Co-Authored-By: Claude Sonnet 5 --- packages/cli/src/__tests__/lib/Config.test.ts | 31 +++++++++++++++++++ packages/cli/src/lib/Config.ts | 13 +++++++- packages/cli/src/types.ts | 1 + 3 files changed, 44 insertions(+), 1 deletion(-) diff --git a/packages/cli/src/__tests__/lib/Config.test.ts b/packages/cli/src/__tests__/lib/Config.test.ts index d7af94be..961cc6a6 100644 --- a/packages/cli/src/__tests__/lib/Config.test.ts +++ b/packages/cli/src/__tests__/lib/Config.test.ts @@ -910,6 +910,37 @@ describe('ConfigManager', () => { (mockFs.readJson as any).mockResolvedValue({ memory: { semantic: 'true' } }); await expect(configManager.getMemorySemanticEnabled()).resolves.toBe(false); }); + + it('lets an explicit project value override the global config', async () => { + (mockFs.pathExists as any).mockResolvedValue(true); + (mockFs.readJson as any).mockImplementation((configPath: string) => + Promise.resolve( + configPath === '/test/dir/.ai-devkit.json' + ? { memory: { semantic: true } } + : { memory: { semantic: false } } + ) + ); + + await expect(configManager.getMemorySemanticEnabled()).resolves.toBe(true); + }); + + it('falls back to the global config when the project value is unset', async () => { + (mockFs.pathExists as any).mockResolvedValue(true); + (mockFs.readJson as any).mockImplementation((configPath: string) => + Promise.resolve( + configPath === '/test/dir/.ai-devkit.json' + ? { environments: [], phases: [] } + : { memory: { semantic: true } } + ) + ); + + await expect(configManager.getMemorySemanticEnabled()).resolves.toBe(true); + }); + + it('defaults to false when neither project nor global config sets semantic', async () => { + (mockFs.pathExists as any).mockResolvedValue(false); + await expect(configManager.getMemorySemanticEnabled()).resolves.toBe(false); + }); }); }); diff --git a/packages/cli/src/lib/Config.ts b/packages/cli/src/lib/Config.ts index 90303fef..c16448a9 100644 --- a/packages/cli/src/lib/Config.ts +++ b/packages/cli/src/lib/Config.ts @@ -4,6 +4,7 @@ import { DevKitConfig, Phase, EnvironmentCode, ConfigSkill, DEFAULT_DOCS_DIR, DE import { filterStringRecord } from '../util/config.js'; import { ConfigNotFoundError } from '../util/errors.js'; import { AddSkillRegistryOptions, planSkillRegistryAdd, planSkillRegistryRemove } from '../util/skill-registry.js'; +import { GlobalConfigManager } from './GlobalConfig.js'; import packageJson from '../../package.json' with { type: 'json' }; const CONFIG_FILE_NAME = '.ai-devkit.json'; @@ -102,9 +103,19 @@ export class ConfigManager { return this.resolveConfiguredPath(config?.memory?.path); } + /** + * Resolves with project > global > default(false) precedence, so semantic + * search can be turned on once globally instead of repeating it per project. + */ async getMemorySemanticEnabled(): Promise { const config = await this.read(); - return config?.memory?.semantic === true; + const projectValue = config?.memory?.semantic; + if (typeof projectValue === 'boolean') { + return projectValue; + } + + const globalConfig = await new GlobalConfigManager().read(); + return globalConfig?.memory?.semantic === true; } private resolveConfiguredPath(configuredPath: unknown): string | undefined { diff --git a/packages/cli/src/types.ts b/packages/cli/src/types.ts index 8ec07d12..49b2a23b 100644 --- a/packages/cli/src/types.ts +++ b/packages/cli/src/types.ts @@ -60,6 +60,7 @@ export interface GlobalDevKitConfig { plugins?: string[]; memory?: { path?: string; + semantic?: boolean; }; } From 9f33b5226be4622d5813aad9f97d71a1ad2a1734 Mon Sep 17 00:00:00 2001 From: Hoang Nguyen Date: Tue, 1 Sep 2026 08:11:15 +0000 Subject: [PATCH 11/11] docs(memory-semantic): record the global-config precedence correction Updates the lifecycle docs to reflect the reinstated and wired memory.semantic global fallback: - design: adds an explicit Design Decisions entry for the project > global > default(false) precedence and the rationale (avoid repeating the setting per project), including the note that the earlier removal was a misclassification, not a re-add. - implementation: corrects the file-structure note and simplification record for Config.ts to describe the resolution instead of a removal. - planning: reclassifies the removal task as a new Phase 6 (user review correction) with its own tasks, and updates the progress summary. - testing: adds the three precedence test cases and refreshes the fresh-validation test counts (1,091, up from 1,088). Co-Authored-By: Claude Sonnet 5 --- .../design/2026-09-01-feature-memory-semantic.md | 5 +++++ .../2026-09-01-feature-memory-semantic.md | 4 ++-- .../2026-09-01-feature-memory-semantic.md | 15 ++++++++++----- .../testing/2026-09-01-feature-memory-semantic.md | 7 +++++-- 4 files changed, 22 insertions(+), 9 deletions(-) diff --git a/docs/ai/design/2026-09-01-feature-memory-semantic.md b/docs/ai/design/2026-09-01-feature-memory-semantic.md index e9adf2a3..db8fa398 100644 --- a/docs/ai/design/2026-09-01-feature-memory-semantic.md +++ b/docs/ai/design/2026-09-01-feature-memory-semantic.md @@ -107,6 +107,11 @@ flowchart LR - Pros: removes an entire class of "which model produced this embedding" bugs and keeps the embedding-version compatibility check trivial (`embedding_version = MODEL_VERSION`). - Cons: cannot swap models without a code change. - Alternatives considered: BGE (small) was benchmarked and rejected — see the Model Selection Gate in the requirements doc; it did not clear the paraphrase nDCG@5/recall@5 improvement threshold. +- `memory.semantic` resolves with project > global > default(false) precedence + - The flag can be set in the per-project `.ai-devkit.json` (`memory.semantic`) and/or the global `~/.ai-devkit/.ai-devkit.json` (`memory.semantic`, `GlobalDevKitConfig.memory.semantic`). + - Resolution (`ConfigManager#getMemorySemanticEnabled`, `packages/cli/src/lib/Config.ts`): if the project config has an explicit boolean value (`true` or `false`), that value wins outright. Only when the project value is absent or malformed does the resolver fall back to the global config's explicit `true`; anything else defaults to `false`. + - Rationale (user-driven): a developer who wants semantic search on for every project they touch should be able to set it once globally instead of repeating `"memory": { "semantic": true }` in every project's `.ai-devkit.json`. An explicit per-project setting (on or off) still overrides the global default, so a project can opt out even when semantic is on globally. + - This was reverted from an earlier pass in this branch that removed the (at-the-time unread) global field as dead code — the field was always intended to be load-bearing; the gap was that nothing wired it yet, not that it should be deleted. No new config surface was added: it is the same `boolean` value read from one more file, resolved through a single function at the one call site (`ConfigManager#getMemorySemanticEnabled`) that every semantic-aware command/server path already went through. - Patterns applied - Fail-open degradation at every semantic call site (search, store, update, backfill) rather than a single global "is semantic available" gate, so a corrupt single row or a mid-session network blip degrades only that call. - Config read at the process/command boundary (`server.ts`, `commands/memory.ts`) rather than threaded as a parameter through every handler, since the flag is process-lifetime, not per-call. diff --git a/docs/ai/implementation/2026-09-01-feature-memory-semantic.md b/docs/ai/implementation/2026-09-01-feature-memory-semantic.md index f5639a61..fc92806f 100644 --- a/docs/ai/implementation/2026-09-01-feature-memory-semantic.md +++ b/docs/ai/implementation/2026-09-01-feature-memory-semantic.md @@ -31,7 +31,7 @@ description: Technical implementation notes, patterns, and code guidelines - `packages/memory/src/database/migrations/002_semantic_embeddings.sql` — additive columns + index. - `packages/memory/src/server.ts` — reads `readSemanticConfig().enabled` once at server construction and routes each MCP tool call to the semantic or plain handler. - `packages/memory/src/api.ts` — exports `*Async` command wrappers (`memorySearchCommandAsync`, `memoryStoreCommandAsync`, `memoryUpdateCommandAsync`) plus `memorySemanticStatusCommand`, `memoryDownloadSemanticCommand`, `memoryReembedCommand`. -- `packages/cli/src/lib/Config.ts` — `getMemorySemanticEnabled()` reads `memory.semantic === true` from the project `.ai-devkit.json`. +- `packages/cli/src/lib/Config.ts` — `getMemorySemanticEnabled()` resolves `memory.semantic` with project > global (`GlobalConfigManager`, `~/.ai-devkit/.ai-devkit.json`) > default `false` precedence. - `packages/cli/src/commands/memory.ts` — `registerMemoryCommand` picks the async/semantic-aware command when `getMemorySemanticEnabled()` resolves `true`; adds the `memory semantic status|download` and `memory reembed [--force]` subcommands and `--explain` on `memory search`. ## Implementation Notes @@ -52,7 +52,7 @@ description: Technical implementation notes, patterns, and code guidelines ### Simplification pass (post-functional-completion) Applied after all eval gates and functional tests were already green, so none of it touches ranking, degradation, or migration behavior: -- Removed `GlobalDevKitConfig.memory.semantic` — added by mistake alongside the per-project field; nothing in `GlobalConfig.ts` or anywhere else ever read it. +- Initially removed `GlobalDevKitConfig.memory.semantic` as dead (nothing read it yet), then reinstated on user review and wired it: `ConfigManager#getMemorySemanticEnabled` now resolves `memory.semantic` with project > global > default(false) precedence, reading `GlobalConfigManager` (`~/.ai-devkit/.ai-devkit.json`) only when the project value is absent or non-boolean. This lets a developer enable semantic search once globally instead of repeating the setting per project, while an explicit per-project value (either direction) still wins. See the design doc's Design Decisions section for the full rationale. - Collapsed `loadLocalEmbedder()` from a `{ modelsRoot?, download? }`-configurable function to a zero-argument one. Its only real caller (`getDefaultLocalEmbedder`) always passed `download: true` and never passed `modelsRoot`; the `download: false` branch (inspect-only, throw if not ready) was unreachable in shipped code, and `storeKnowledgeSemantic`/`updateKnowledgeSemantic`/`reembedKnowledge` all threaded a `modelsRoot` option that no caller (production or test) ever set. - Removed `LocalEmbedder.dispose()` — implemented (`session.release()`) but never invoked anywhere; the embedder is process-lifetime by design (CLI processes are short-lived and exit; the MCP server holds one memoized session for its life), so there was no call site that could ever exercise cleanup. - Replaced `SemanticModelUnavailableError` with a plain `Error` (still carrying `cause`) — the subclass was thrown but never checked with `instanceof` anywhere; every catch site already falls back to `error.message`/`String(error)` generically. diff --git a/docs/ai/planning/2026-09-01-feature-memory-semantic.md b/docs/ai/planning/2026-09-01-feature-memory-semantic.md index f32090eb..754014d2 100644 --- a/docs/ai/planning/2026-09-01-feature-memory-semantic.md +++ b/docs/ai/planning/2026-09-01-feature-memory-semantic.md @@ -43,10 +43,15 @@ description: Break down work into actionable tasks and estimate timeline ### Phase 5: Pre-merge simplification (this pass) - [x] Task 5.1: Audit the diff for speculative config surface, unused options, and dead guard/error paths against the `simplify-implementation` discipline -- [x] Task 5.2: Remove `GlobalDevKitConfig.memory.semantic` (never read — global config has no semantic reader) -- [x] Task 5.3: Collapse `loadLocalEmbedder`'s unused `modelsRoot`/`download` options and the dead offline-inspect branch; remove the never-invoked `LocalEmbedder.dispose()`; replace the never-narrowed `SemanticModelUnavailableError` with a plain `Error` -- [x] Task 5.4: Close the CLI test gap on the semantic-enabled `memory update` path (previously an unused mock import, flagged by lint) -- [x] Task 5.5: Rebuild the five lifecycle phase docs (this document and its siblings) against the simplified, shipped state, replacing the single ad hoc `docs/ai/2026-09-01-feature-memory-semantic.md` note +- [x] Task 5.2: Collapse `loadLocalEmbedder`'s unused `modelsRoot`/`download` options and the dead offline-inspect branch; remove the never-invoked `LocalEmbedder.dispose()`; replace the never-narrowed `SemanticModelUnavailableError` with a plain `Error` +- [x] Task 5.3: Close the CLI test gap on the semantic-enabled `memory update` path (previously an unused mock import, flagged by lint) +- [x] Task 5.4: Rebuild the five lifecycle phase docs (this document and its siblings) against the simplified, shipped state, replacing the single ad hoc `docs/ai/2026-09-01-feature-memory-semantic.md` note + +### Phase 6: User review correction (this pass) +- [x] Task 6.1: Reinstate `GlobalDevKitConfig.memory.semantic` — the initial pass wrongly treated it as dead code; it was unwired, not unwanted. +- [x] Task 6.2: Wire it at the single resolution site (`ConfigManager#getMemorySemanticEnabled`) with project > global > default(false) precedence, so semantic search can be enabled once globally instead of per project. +- [x] Task 6.3: Add tests for all three precedence cases (project overrides global, project unset inherits global, both unset defaults to false). +- [x] Task 6.4: Record the precedence decision and its rationale in the design doc; correct the implementation/testing docs and the simplification record to reflect the reversal. ## Dependencies @@ -91,7 +96,7 @@ Delivered in a single feature branch (`feature-memory-semantic`) across five com ## Progress Summary -All functional milestones (1-4) were complete and gate-evidenced before this pass began. This pass (Milestone 5) audited the shipped diff against the repository's simplification discipline, removed four pieces of speculative/dead surface (an unread global-config field, unused embedder options and an unreachable degradation branch, an unused error subclass, and an unused lifecycle method), closed one CLI test gap that lint had flagged, and rebuilt the lifecycle documentation set to match the phase-by-phase discipline the docs were missing. No eval-passing behavior (RRF fusion, degradation paths, migration safety, or the MiniLM model/thresholds) was changed. +All functional milestones (1-4) were complete and gate-evidenced before this pass began. This pass (Milestone 5) audited the shipped diff against the repository's simplification discipline, removed three pieces of speculative/dead surface (unused embedder options and an unreachable degradation branch, an unused error subclass, and an unused lifecycle method), closed one CLI test gap that lint had flagged, and rebuilt the lifecycle documentation set to match the phase-by-phase discipline the docs were missing. A fourth candidate — `GlobalDevKitConfig.memory.semantic` — was initially removed as dead code, then reinstated on user review (Milestone 6): the field was unwired, not unwanted, and now resolves with project > global > default(false) precedence at the one call site every semantic-aware path already goes through. No eval-passing behavior (RRF fusion, degradation paths, migration safety, or the MiniLM model/thresholds) was changed. ## Next Focus diff --git a/docs/ai/testing/2026-09-01-feature-memory-semantic.md b/docs/ai/testing/2026-09-01-feature-memory-semantic.md index 3bac9c11..b7860e97 100644 --- a/docs/ai/testing/2026-09-01-feature-memory-semantic.md +++ b/docs/ai/testing/2026-09-01-feature-memory-semantic.md @@ -38,6 +38,9 @@ description: Define testing approach, test cases, and quality assurance ### CLI config (`packages/cli/src/__tests__/lib/Config.test.ts`) - [x] `getMemorySemanticEnabled()` returns `false` by default, `true` only for `{ memory: { semantic: true } }`, and `false` for `{ memory: { semantic: 'true' } }`. +- [x] An explicit project value overrides the global config (project `true` wins over global `false`). +- [x] An unset/malformed project value falls back to the global config (global `true` is inherited when the project value is absent). +- [x] Neither project nor global config setting `semantic` defaults to `false`. ## Integration Tests @@ -110,11 +113,11 @@ Executed after the simplification commits in this pass, before any doc-only chan - `@ai-devkit/task-manager`: 8 test files, 112 tests passed. - `@ai-devkit/agent-manager`: 41 test files, 631 tests passed. - `@ai-devkit/memory-dashboard`: 3 test files, 22 tests passed. - - `ai-devkit` (cli): 91 test files, 1,088 tests passed (was 1,087 before this pass's added `memory update` semantic test). + - `ai-devkit` (cli): 91 test files, 1,091 tests passed (was 1,087 before this pass's added `memory update` semantic test and the three global/project precedence tests). - `npm run lint` (workspace-wide) — 0 errors; 3 pre-existing warnings unrelated to this feature (`init.test.ts`, `channel.ts`, `util/skill.ts`); the pre-existing `memoryUpdateCommandAsync` unused-import warning in `memory.test.ts` is resolved by this pass's added test. - `npx vitest run --config e2e/vitest.config.ts` — 1 test file, 41 tests passed. -No eval-gate number above changed as a result of this pass: the simplification commits touched only unused config/API surface (a dead global-config field, unused embedder options and an unreachable branch, an unused error subclass, an unused lifecycle method) and added one missing test; they did not touch `fuseSearchResults`, the degradation paths in `searchKnowledgeHybrid`, the migration, or the pinned model/thresholds. +No eval-gate number above changed as a result of this pass: the changes touched unused embedder options and an unreachable branch, an unused error subclass, an unused lifecycle method, the `memory.semantic` global/project resolution, and added tests for both; none of it touched `fuseSearchResults`, the degradation paths in `searchKnowledgeHybrid`, the migration, or the pinned model/thresholds. ## Manual Testing