[REVERTED — see #24] v1.0.0: architecture2 rearchitecture + OSS release prep#23
Merged
[REVERTED — see #24] v1.0.0: architecture2 rearchitecture + OSS release prep#23
Conversation
chore: split local side-by-side compose cleanup
…coverage (#8) • add createCoreRuntime, createApp, and structured startup checks under src/app/ • remove setEntityRepository and thread entityRepo explicitly through the search pipeline • slim server.ts to boot from the runtime container while preserving existing exports • add focused runtime-container and deployment/config validation coverage • keep endpoint behavior and route behavior unchanged
• return a canonical search-scope contract from the API instead of relying on implicit behavior • keep scope semantics explicit at the route/service boundary • add focused tests to lock the contract and preserve current behavior
…#10) • return canonical search scope metadata from the API • make retrieval observability explicit while keeping it optional when trace payloads are absent • route config mutation through the runtime adapter instead of hidden direct wiring • preserve current search behavior while tightening the API contract and coverage
• finish the Phase 3 config-seam cleanup needed before honest Phase 4 work • thread runtime config through the search, ingest, route, and lineage paths that previously relied on singleton reads • make API scope/observability behavior explicit without changing the intended fast-path behavior • add focused tests and guardrails so the narrowed seam stays honest during later Phase 4 changes
Extract shared test helpers and deduplicate search pipeline test setup: - Extract bindEphemeral + BootedApp into src/__tests__/test-helpers.ts, used by memory-route-config-seam.test.ts and composed-boot-parity.test.ts - Extract twoLowSimilarityResults + createVectorRepo helpers in search-pipeline-runtime-config.test.ts to eliminate repeated 4-test setup boilerplate - Extract resolveSearchPreamble in routes/memories.ts to deduplicate the parse-scope-limit pattern between /search and /search/fast Remaining 234 lines (0.7%) are structural patterns that can't be meaningfully extracted: camelCase/snake_case config snapshots, idiomatic Express route handler preambles, port interface shapes, and test mock config objects. fallow exits 0 (no threshold configured).
Splits memory-ingest.ts (475 → 215 lines) into three focused modules with explicit seams, completing Phase 4 of the core rearchitecture. Extraction (Steps 4A/4B): - ingest-post-write.ts: backdate, link generation, and composite generation behind runPostWriteProcessors(). Composites are caller-gated via compositesEnabled — only performIngest passes true. - ingest-fact-pipeline.ts: unifies ingestFact, quickIngestFact, and workspaceIngestFact behind processFactThroughPipeline() with FactPipelineOptions. Workspace AUDN asymmetry (no lineage, no deferred AUDN) preserved verbatim. Honesty fixes: - Removed dishonest resolveClaims option that was documented but never consulted in the full AUDN path. - Corrected overstated config-threading claims in runtime-container.ts JSDoc — now explicit about where threading is honest (search/ingest) vs where the singleton still leaks (embedding, LLM, extraction). - Ratcheted singleton gate from 34 → 33. Behavioral fences (codex review): - 6 workspace AUDN tests covering ADD/UPDATE/SUPERSEDE/NOOP routing plus negative assertions that the workspace path bypasses canonical storage, full AUDN resolution, and entropy gating. - 6 post-write tests covering the compositesEnabled boundary, backdate ordering, and composite threshold gating. 933/933 tests pass. tsc clean. fallow clean. No behavior change. Public API (memory-service.ts) untouched.
…aps (#15) Addresses the codex architecture review's two High findings with bounded, honest progress — unifying the search/expand boundary behind MemoryScope and documenting the schema scoping gaps as explicitly deferred. Scope contract (Finding 1 — partially addressed): - Expands MemoryScope workspace variant with required agentId for visibility enforcement, adds ScopedSearchOptions to collapse divergent search parameter lists into one options bag. - Introduces scopedSearch() and scopedExpand() on MemoryService that dispatch on scope.kind — user scope routes to performSearch/expandMemories, workspace scope routes to performWorkspaceSearch/expandMemoriesInWorkspace. - Search routes (/search, /search/fast) and /expand now call the scoped methods instead of branching inline. Old paired methods marked @deprecated. - Honestly scoped: list/get/delete routes stay branched (no agent_id in their request contracts). Ingest stays branched (needs write-context fields MemoryScope doesn't carry). The deeper repo/pipeline split is Phase 5. Schema honesty (Finding 2 — deferred, not resolved): - docs/design/scope-access-contract.md documents the full scope model: what MemoryScope is, where it flows, which tables support workspace scoping and which don't, why claims and foresight are intentionally user-scoped (AUDN contradiction resolution is cross-workspace), and the Phase 5 prerequisites for goal/version expansion. - SCOPE_TODO comments added to all 6 unscoped tables in schema.sql (memory_atomic_facts, memory_foresight, memory_claims, memory_claim_versions, entities, lessons). 938/938 tests pass (6 new dispatch tests). tsc clean. Fallow clean. No behavior change to any existing path.
Replaces the 40+ method MemoryRepository facade with 8 focused store interfaces (MemoryStore, EpisodeStore, SearchStore, SemanticLinkStore, RepresentationStore, ClaimStore, EntityStore, LessonStore), routes workspace ingest through canonical lineage for the first time, and closes a live workspace visibility bypass on the public query routes. Executed in 10 planned steps with codex review at every gate. Access control (breaking): - Workspace GET /memories/list, GET /memories/:id, and DELETE /memories/:id now require agent_id when workspace_id is present — missing returns 400, not a visibility-unsafe fallback. Legacy *InWorkspace facade methods deleted. - scopedExpand, scopedGet, scopedDelete, scopedList all enforce agent-level visibility via the same SQL clause search uses (buildVisibilityClauseForSearch). agent_only memories hidden from non-owning agents; restricted requires an explicit grant. - Malformed agent_id validated as UUID → 400. Invisible workspace delete → 404, not 500. Store layer: - ClaimStore, EntityStore, LessonStore narrowed via Pick<> to the exact method surfaces their consumers call (no full-repo escape hatch). - 5 Pg*Store classes wrap the existing split repository functions. - CoreStores bundle wired into runtime-container alongside existing repos; MemoryServiceDeps now carries only stores.* (deprecated deps.repo/deps.claims/deps.entities/deps.lessons fields removed). - pool: pg.Pool isolated behind CoreStores.pool — all previous deps.repo.getPool() callers (search, link generation, deferred-audn, reconciliation) now source from deps.stores.pool. - All 10 downstream search helpers (iterative-retrieval, agentic-retrieval, 4 query-expansion variants, keyword-expansion, deferred-audn, consolidation-service, subject-aware-ranking) migrated to SearchStore/MemoryStore/EntityStore params. All 8 as any bridges deleted — the search pipeline is now fully compiler-enforced. Schema: - memory_atomic_facts and memory_foresight gain workspace_id + agent_id columns via idempotent ALTER TABLE ... ADD COLUMN IF NOT EXISTS, plus partial indexes on workspace_id. - StoreAtomicFactInput/StoreForesightInput and the AtomicFactRow/ForesightRow types carry the new fields. MemoryRepository marked @deprecated. Workspace canonical lineage (behavioral change): - processWorkspaceFact now routes through storeCanonicalFact and resolveAndExecuteAudn instead of the inline storeWorkspaceMemory path. Workspace memories gain: CMO creation, claim/version lineage, atomic fact decomposition, foresight projections, entity resolution, and linking. - AudnFactContext carries optional workspace; storeProjection, supersedeCanonicalFact, and updateCanonicalFact thread it through — fixing the scope-loss that would have occurred on delete-and-reinsert replace paths. - The two AUDN side branches (CLARIFY clarification memory, opinion-confidence-collapse) also thread workspace scope. - storeWorkspaceMemory deleted. Intentional Phase 5 debt, pinned by test: Claims, claim versions, entities, and entity relations remain user-scoped. Cross-workspace ingests for the same user share entity/claim state. A behavioral regression test runs storeCanonicalFact twice with two workspace contexts against a stateful fake EntityStore and asserts the second call reuses entities from the first, plus that resolveEntity is called with no workspace/agent fields — the exact Phase 6 flip point. Fences added: - 3 visibility regression tests (expand-visibility.test.ts) - 4 route-level 400/404 tests - 3 new 400-fence tests for missing agent_id - 2 runtime-container store-construction tests - 3 scoped dispatch tests - 2 AUDN workspace-scope fences (CLARIFY + opinion-collapse) with strict call-count assertions - 3 behavioral workspace-pipeline tests - 3 cross-workspace coupling fences (stateful fake, Phase 6 flip point) 953/953 tests pass. npx tsc --noEmit clean. fallow --no-cache clean. Config singleton gate unchanged. Public API shape: only scoped methods remain on MemoryService.
…ers (#17) Phases 1A–1B built the composition root (createCoreRuntime, createApp) but kept it internal. Phase 6 promotes the seams to the public surface, documents them, and pins behavior with a contract test so HTTP and in-process ingest cannot silently diverge. Public API src/index.ts exports createCoreRuntime + types, createApp, checkEmbeddingDimensions, bindEphemeral. Existing exports preserved — current research paths keep working. Seam promotion src/__tests__/test-helpers.ts moved to src/app/bind-ephemeral.ts so research harnesses can import the canonical HTTP-boot helper without reaching into a test dir. Two importers updated. Contract test (src/app/__tests__/research-consumption-seams.test.ts) - In-process seam: runtime.services.memory.ingest/search end-to-end - HTTP seam: POST /memories/ingest + /memories/search via bindEphemeral - Parity, both directions: in-process write → HTTP search, and HTTP write → in-process search. Two-directional parity catches regressions where HTTP input parsing or response shaping drifts from the in-process contract — one direction alone would miss that. Docs - docs/consuming-core.md — HTTP / in-process / docker seams, copy-pastable examples using the actual package name (@atomicmemory/atomicmemory-engine), stability boundary for deep-path imports, what belongs in research. - docs/README.md + CLAUDE.md — pointers to the new doc. Out of scope (per plan) - No research-repo migration - No narrowing of deep-path exports in package.json (Risk 4 — avoid breaking research in the same step that gives it a new seam) - No config split or retrieval decomposition Verification - 957/957 tests pass - tsc --noEmit clean - fallow --no-cache 0 above threshold
…18) Phase 7 Steps 3a–3c + doc followup. Completes the Phase 1A-descoped config split, deprecates PUT /memories/config for production, freezes provider/model selection as startup-only. No parity regression — gated by the v1→v2 parity audit (companion: atomicmemory-research#7). Changes 3a — partition runtime config. SUPPORTED_RUNTIME_CONFIG_FIELDS (39 stable) + INTERNAL_POLICY_CONFIG_FIELDS (66 experimental) in src/config.ts, with Pick<> slices as documentation types. New fence test rejects stray/duplicate keys — any new RuntimeConfig field must be tagged. Re-exported from root. 3b — deprecate PUT /memories/config for production. New runtimeConfigMutationEnabled flag, parsed from CORE_RUNTIME_CONFIG_MUTATION_ENABLED (default false). Production returns 410 Gone; .env.test opts in for dev/CI. Flag read from a memoized startup snapshot through configRouteAdapter — no per-request env reads, no NODE_ENV branching. 3c — freeze provider/model. embedding_provider, embedding_model, llm_provider, llm_model now rejected with 400 + rejected array. Rationale: the provider caches are fixed at first use — v1 accepted these fields but mid-flight mutation never took effect. Bug fix, not a feature loss. Followup. docs/api-reference.md § rewritten to match the new contract. Success-response note fixed (it claimed provider/model is applied in-memory; they're now rejected earlier in the same handler). Out of scope Step 3d (multi-PR leaf-config threading), Item 4 (retrieval polish), Item 2 (main cutover — deferred). Test plan - npx tsc --noEmit clean - npm test 963/963 (+6 new: 4 partition, 1 410, 1 400) - fallow --no-cache 0 above threshold
…n audit) (#21) Five leaves, two patterns Explicit parameter threading (callers already had deps.config in scope): - consensus-extraction.ts, write-security.ts, cost-telemetry.ts Module-local init (hot-path APIs with 15+ callers — ripple avoidance): - embedding.ts via initEmbedding(config), llm.ts via initLlm(config) - runtime-container.ts calls both inits inside createCoreRuntime Deep-path init requirement Dropping config from embedding.ts/llm.ts is a contract change for consumers who deep-import /services/embedding or /services/llm (still public per package.json#exports; narrowing is out of scope). Mitigations: - src/index.ts re-exports initEmbedding, initLlm, EmbeddingConfig, LLMConfig from the root. - docs/consuming-core.md adds a Deep-path init requirement section. Test-file cleanups write-security.test.ts, embedding-cache.test.ts, llm-providers.test.ts stopped mutating the config singleton; each builds its own config object / calls init*(testConfig) explicitly. Test plan - npx tsc --noEmit clean - npm test 963/963 pass - fallow --no-cache 0 above threshold, maintainability 91.0 Out of scope Remaining audit drops below 28 (infrastructure files); deep-path export narrowing; Item 4 (separate PR).
…-search.ts so it becomes pure orchestration. 374 → 248 lines (-34%)(#22) Helpers moved - buildInjection → retrieval-format.ts (alongside existing formatSimpleInjection / formatTieredInjection; new InjectionBuildResult interface) - applyFlatPackagingPolicy → composite-dedup.ts (thin wrapper around deduplicateCompositeMembersForFlatQuery already there) - recordSearchSideEffects → new retrieval-side-effects.ts (touchMemory + audit emission) - recordConsensusLessons → lesson-service.ts (static import replaces previous dynamic-import round-trip; signature narrowed to take LessonStore instead of full MemoryServiceDeps) - finalizePackagingTrace → packaging-observability.ts (bundles packaging/assembly summary + tiered-packaging event + trace setters; absorbs deduplicateCompositeMembersHard dependency) Stability performSearch, performFastSearch, performWorkspaceSearch keep their existing signatures. All plan regression canaries pass: smoke.test.ts, memory-search-runtime-config.test.ts, search-pipeline-runtime-config.test.ts, contradiction-safe.test.ts, temporal-correctness.test.ts. Test plan - npx tsc --noEmit clean - npm test 963/963 pass (no new tests — pure refactor) - fallow --no-cache 0 above threshold, maintainability 90.9 - memory-search.ts ≤ 250 lines (248, hits plan target)
…ish wiring
Preparing atomicmemory-core for public release. All internal-codename
references ('SuperMem') removed from src/; private-repo URLs unlinked
from public-facing docs; package flipped to public + renamed to match
repo and SDK naming; npm publish step added to release workflow; tarball
scoped to runtime code only (test files excluded).
- package.json: drop 'private: true', rename to @atomicmemory/atomicmemory-core
(from @atomicmemory/atomicmemory-engine — matches repo name and SDK's
@atomicmemory/atomicmemory-sdk pattern). Add 'files' field excluding
__tests__/, *.test.ts, and test-helpers; reduces tarball 250→119 files,
1.5MB→812KB unpacked.
- src/: 8 files SuperMem → AtomicMemory rename (src/index.ts docstring,
src/db/ppr.ts HippoRAG attribution, 6 test fixtures with 'SuperMem
engine' fact strings). No behavior change; all 963 tests pass.
- docs/README.md:14, README.md:23, SECURITY.md:22, CLAUDE.md:26,
docs/consuming-core.md:194: unlink private atomicmemory-research repo
URL; keep the 'research lives separately' boundary wording.
- src/db/schema.sql, src/app/__tests__/composed-boot-parity.test.ts,
src/app/runtime-container.ts: replace internal doc-path references
(atomicmemory-research/docs/...) with self-contained explanations.
These would have 404'd for public readers.
- CHANGELOG.md 'Initial extraction from atomicmemory-research prototype'
left as historical attribution (factually accurate, doesn't link out).
- .github/workflows/release.yml: add 'Publish to npm' step with
NODE_AUTH_TOKEN from NPM_TOKEN secret, set registry-url to
registry.npmjs.org. Triggers on tag push (v*).
Verification:
- npx tsc --noEmit: clean
- npm test: 963/963 pass
- fallow --no-cache: 0 above threshold (maintainability 91.0)
- npm publish --dry-run: @atomicmemory/atomicmemory-core@1.0.0
- Git history secret scan: 0 matches for sk-*, AKIA*, ghp_*, xox*, password= patterns
- Remaining 'atomicmemory-research' references in repo: only CHANGELOG
historical line
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Codex review surfaced three blockers for public v1.0.0: 1. Package not consumable as published. main/exports pointed at raw src/*.ts but src/index.ts re-exports *.js neighbors not present in the tarball. Add tsconfig.build.json (excludes tests), build script (tsc → dist), flip main/types/exports to dist/, ship dist/ in files. prepublishOnly runs build automatically so `npm publish` never ships an uncompiled tarball. Verified with bare import from clean dir: 17 exports resolve. 2. docs/consuming-core.md still referenced @atomicmemory/atomicmemory-engine in 5 code examples. Renamed to @atomicmemory/atomicmemory-core. 3. docker-compose + .env.example still exposed 'supermem' as DB name/user/ password. Renamed to 'atomicmemory' across compose files, env examples, and docker-smoke-test.sh project name. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mount `memories` and `agents` routers under `/v1` in `createApp`, leaving the unversioned `/health` infrastructure probe untouched. Updates observability labels, tests that hit real HTTP, docs, and the docker smoke script to match. Adds `versioned-mount.test.ts` to assert the prefix is wired for both route families and that bare paths 404. BREAKING CHANGE: clients must prefix requests with `/v1` (e.g. `POST /v1/memories/ingest`, `PUT /v1/agents/trust`). `/health` is unchanged.
The composed-boot-parity test exercises `PUT /v1/memories/config`, which returns 410 Gone unless the startup-validated flag is set. Local runs pass because `.env.test` (gitignored) has the flag set, but CI didn't — leaving this test broken on every run since it landed. Match local by setting the flag in the workflow env block.
ethanj
changed the title
Contributor
Author
|
The work is now in #24, retargeted at |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Cuts over
mainto the Phase 1A–7 rearchitecture (composition root, scope contract, observability contract, config split, store interfaces, consumption seams, leaf-module config threading) plus the OSS-release-prep on top./v1API prefixAll application endpoints are now mounted under
/v1(e.g.POST /v1/memories/ingest,PUT /v1/agents/trust). The unversioned/healthliveness probe is unchanged. Clients must prefix requests with/v1. Added pre-1.0 so the versioning namespace is in place before any external consumers depend on the HTTP surface.What's in this PR
Rearchitecture (Phases 1A–7):
createCoreRuntime(Phase 1A: introduce runtime container composition root + integration coverage #8)PUT /memories/configdeprecation (Phase 7 Steps 3a-3c: config split + PUT /memories/config deprecation #18)OSS release prep (this branch):
@atomicmemory/atomicmemory-engine→@atomicmemory/atomicmemory-core(npm package rename)"private": trueremoved frompackage.json;filesfield scopes tarballrelease.ymlpublishes to public npm on tag push (NPM_TOKEN secret)docs.atomicmemory.ai/v1API prefix on all routes (see breaking change above). Adds a mount-coverage test (src/app/__tests__/versioned-mount.test.ts) that asserts both route families resolve under/v1and that bare paths 404.Post-merge steps (manual)
v1.0.0onmain— triggers npm publish via release.ymlarchitecture2branchTest plan
npm publish --dry-runsucceeds locallyv1.0.0→ confirm GitHub Release + npm publish succeed🤖 Generated with Claude Code