v0.4.6 — fast reads, explicit projects, hardened imports, Phase 0 API

Release covering 0.4.2 through 0.4.6 (tags v0.4.2, v0.4.3, v0.4.4, v0.4.5, v0.4.6). Deployed to memora-all on 2026-09-23. Contract memora-api/v1 1.0.0 is frozen at tag memora-api-v1.0.0.

0.4.6

A plain JSON API (Phase 0 of the clmux memora daemon), the absorb type
boundary, and typed tags no longer taken as project evidence.

The API is OFF unless configured: its routes are registered ONLY when
MEMORA_API_TOKENS_FILE is set. memora-all does not set it in this release,
so there the API is not registered at all (every /api/v1/... path is 404).
The contract is frozen as contracts/memora-api/v1 1.0.0, tagged
memora-api-v1.0.0. No configuration change is needed for this release.

Plain JSON API /api/v1/<store>/{health,search,absorb} (Phase 0 of the clmux memora daemon)

• New HTTP routes next to MCP on the streamable-http/sse transports, outside the MCP session machinery: GET health (200 ok, or a 503 health document: down probe_error/store_missing/store_locked/integrity_fault, degraded unproven/stale), POST search (hybrid-v1 with follow="active", raw cosine, fused order, 1-based rank, preview, project and tags_any filters, unscored coverage count) and POST absorb (validated, then 501 writes_unsupported: no store is transactional yet). The versioned contract lives in contracts/memora-api/v1 (1.0.0: schemas, fixtures, manifest; scripts/memora_api_contract.py validate / live), and scripts/memora_api_smoke.py is a live gate.
• Off unless configured: the routes are registered only with MEMORA_API_TOKENS_FILE (sha256 token -> stores, re-read on change at most once a second, strict file and parent-directory checks); every request needs a token for the store. Request order: 401, 404, 403, 404, 413 (64 KiB body cap, counted at the ASGI receive boundary), 400, 429 (MEMORA_API_MAX_INFLIGHT, default 8), result. One error envelope for every non-2xx except health's 503.
• Reads never write, set up a schema or create files: the API search is strictly read-only (no embedding rebuild on a model mismatch: 503 store_degraded; no vector repair: unscored), and the API and the readiness probe (/health/db too) connect without schema setup. A local SQLite store opens read-only under a new per-store in-process reader-writer lock that every local writer connection's open and close takes (so no in-process writer can open or close during a read); a missing store, schema or usable WAL sidecars answers 503 instead of being created. Writers in another process are out of scope. The read-only corpus snapshot is single-flight and shares the corpus cache's budget.
• Thread contract: a local SQLite connection keeps stock sqlite3's check_same_thread behaviour exactly -- every public Connection and Cursor method, cursors returned by execute/executemany/executescript, blobs and iterdump, verified against native sqlite3 -- although the native connection underneath is opened with check_same_thread=False so a close by the garbage collector can happen under the store lock. GC closes never block; one that cannot take the lock at once is deferred and drained inside the next writer section.
• memora-server now pins uvicorn to http="h11", loop="asyncio", h11_max_incomplete_event_size=16384 (the stated body bound -- 64 KiB plus one transport read -- depends on it: the largest ASGI body message was measured at 262144 bytes in the image runtime itself, CPython 3.12.14 and uvicorn 0.53.0, with docker run --rm -i memora:latest python - < scripts/measure_asgi_body_messages.py (17 messages for a 4 MiB chunked body); additional data points, all 262144: uvicorn 0.53.0 and 0.42.0 on CPython 3.12.8, uvicorn 0.42.0 on CPython 3.13.1). A local SQLite store's directory is now created on its first writing connection rather than when the backend object is built.

Typed tags are not project evidence

• A typed tag (<project>/issues, /todos, /sections, /documents, /knowledge) says what a memory IS, not which project it belongs to: the old default put memora/issues and memora/todos on every issue and todo whatever its project, so many clmux issues carry memora/issues as their only memora tag. Project inference from tags now ignores typed tags; a memory's project comes from an explicit project, metadata.project, or a non-typed configured project tag (clmux/tui).
• A memory's own typed tags (a system kind matching its metadata.type, bare or under any project prefix, the old default included) stay exempt from the tag allowlist and are exported as system_tags. They are re-prefixed to the memory's project once one is resolved: an update that sets metadata.project or adds a non-typed project tag, or an import that declares a project, turns memora/issues into clmux/issues. With no resolved project they are left as they are. On import, the only legacy form accepted without a project is the old default memora/<kind>; any other prefix in system_tags is refused as a forged system tag.
• scripts/preview_backfill_47.py is a READ-ONLY preview of the #47 backfill with an explicit approval list (every row "approved": false); it creates nothing and may refuse: a local WAL store with sidecars (a writer process such as the server) is refused, with a non-zero exit. It supports local SQLite and D1 stores only: any other store URI (s3:// or unknown) is refused from the configured URI text, before any backend is built or memora is imported, because a cloud backend creates its cache directory and syncs its cache; with a registry present, MEMORA_STORAGE_URI is pinned to the selected store for the run so memora's import-time backend is never a cloud one. Contradictions (stored section vs the project the old typed-tag rule gave) get a proposed target only from agreeing non-typed evidence (metadata.project, non-typed tags, section); keyword-only rows only where the section came from content keywords; everything else is needs-human. A proposal sets metadata.project and re-prefixes the typed tags, with a <project>/<subsection> marker tag listed as an alternative.

Absorb: a supersession never crosses memory types

• Absorb could let a plain narrative fact supersede an open todo when the verifier judged the fact to "reiterate and confirm" it, so the task vanished from follow="active" lists (memora issue memory 1126: #1122 superseded open todo #1118). The per-leaf supersede gate now checks the type boundary first, before the similarity floor and without an LLM call: a leaf whose metadata.type (todo, issue, section, document_root, document_fragment, or none for a plain memory) differs from the new fact's (the absorb call's metadata.type) is downgraded to a related link. The decision's supersede_check and leaf_checks report gate: "type", type_mismatch: true, old_type and new_type.
• The same rule applies to the concurrent-sibling (fork heal) pair check: siblings of different types never collapse.
• For a same-type pair the supersede verifier now sees both sides' type.
• A leaf check is reused at the write boundary only while the leaf's fingerprint is unchanged, and the fingerprint now covers its type and metadata.project as well as text and tags: a metadata-only patch between classification and the write (e.g. to type: "todo") forces a re-gate. The fingerprint also covers the leaf's stored vector (every metadata change re-embeds a memory), and a reused verdict is still refused if the fresh similarity is below the supersede floor.

0.4.5

Project identity is explicit (issue #47), and the import hardening that fixing
it forced. Configuration change: set MEMORA_PROJECTS (below); without it
no project is inferred from tags, so only memories given an explicit project
get sections and project-prefixed tags. The MCP tool count is now 44 (new
admin tool memory_import_sweep, full profile only).

Project identity is explicit, never guessed (issue #47)

• Removed keyword-based project detection (_detect_project, _PROJECT_INDICATORS, _TAG_PROJECT_MAP, _KNOWN_PROJECT_PREFIXES). Generic words such as "embedding", "workspace", "daemon" or "sidebar" no longer put a memory into memora or clmux; that misfiled memories, prefixed their generic tags with the wrong project (clmux/architecture on pi facts, #1109-#1114) and fed a wrong supersession (#1082).
• A memory's project now comes, in order, from: an explicit project argument (new, optional, on memory_create, memory_create_issue, memory_create_todo, memory_absorb, the CLI absorb --project, and the storage functions); else the memory's metadata.project; else exactly one tag naming a project configured for the store. Otherwise it has no project. An explicit project is also recorded as metadata.project.
• MEMORA_PROJECTS (new, optional): the projects a store holds, as a JSON list (every store) or {store: [projects]} ("default" for a single-store deployment). Unset: no project is inferred from tags, and explicit projects are accepted as any valid name ([a-z0-9_-]{1,64}). Set: a project outside the store's list is rejected (invalid_input), whether it arrives as the project argument or as metadata.project (on create, import and update). The whole value, including stores this server does not open, is validated at startup, and the server refuses to start on a malformed one. Deployments that relied on memora/... and clmux/... tags implying a project should set it, e.g. MEMORA_PROJECTS='["memora","clmux"]': under it, memora-tagged content still resolves to memora, so the memora store keeps its conventions.
• Section/subsection assignment and generic-tag prefixing (architecture -> <project>/architecture) now act only on that resolved project; the memora/clmux section conventions are unchanged when the project is given, and now work for any project.
• LLM-suggested absorb tags are kept when the configured tag allowlist permits them (MEMORA_ALLOW_ANY_TAG permits any project-prefixed tag), instead of only memora/ and clmux/; with an explicit project, a suggestion naming a different configured project is dropped. The classify prompt no longer uses memora/research and clmux/architecture as examples.
• Typed tags follow the project, with no memora default: memory_create_issue, memory_create_todo, memory_create_section and memory_store_document (new project argument on the last two) tag <project>/issues, /todos, /sections, /documents, or bare issues, todos, sections, documents without a project (previously always memora/...). memory_create's type-based tag suggestions (<project>/todos, /issues, /knowledge) use the memory's own project, never another's. These typed tags are memora's own and are exempt from the tag allowlist, so the tools work under the default policy; user-supplied tags are still enforced (a caller cannot hand-apply pi/issues under a policy that does not allow it). A system tag is bound to the memory's metadata.type (issues only on an issue, and so on) and to its own project, and only internal paths can apply one: a memory_create_batch entry carrying system_tags is rejected (invalid_batch). An update may keep a memory's own typed tags; adding one it never had is enforced like any user tag.
• Generic tags are prefixed with the project (plan -> <project>/plan) only when the tag policy permits the prefixed form; otherwise they stay bare, so an explicit project never turns an allowed tag into a rejected one.
• Existing memories are not modified, and backfill_tags does not remediate them: it keeps an existing section and treats legacy clmux/.../memora/... tags as identity. scripts/report_project_detection.py is a read-only remediation preview: it lists memories whose stored section or project-prefixed tags came from the removed keyword heuristics and would change under a remediation (a separate item).
• A document's tag follows its resolved project: explicit, else metadata.project, else a configured project tag. The graph viewer's issue and todo filters, and the digest's todo and issue buckets, accept any <project>/issues, <project>/todos or bare tag, the legacy memora/... ones included, as well as metadata.type.

Import and export (hardening forced by #47)

A D1 replace is NOT atomic. D1 has no transactions: every statement
commits on its own. If a replace stops part-way, the store may hold part of
the old contents, part of the new, or neither; the result says exactly what
happened (below). Recovery: keep the export file and re-run the same
memory_import(..., strategy="replace") from it -- clearing is idempotent,
and the re-run replaces whatever the stopped one left. (If the stopped import's
process died, its lease still blocks the re-run with import_in_progress until
it expires, at most 30 minutes; memory_import also has a 60 s cooldown.) Local SQLite imports
are one transaction and are all-or-nothing.

• Export: records gain a system_tags field (the typed tags memora applied); import re-applies them through the same type-bound validation, so an export restores under the default tag policy.
• Prepare before delete: every entry is validated and embedded before anything is written. A replace with any failing entry aborts with nothing deleted (replaced: false); it used to clear the store first and then reject entries one by one.
• Local SQLite: the replace's DELETEs and every INSERT are one transaction; any write error rolls back to the unchanged store (replaced: false).
• D1 staged clear: a replace clears crossrefs, embeddings, FTS, then memories last, each stage retried. A stage that still fails stops with replaced: "partial" and clear_stage; every memory is still present until the last stage.
• D1 rows: written in order, each retried a bounded number of times. Every INSERT carries a per-import, per-row marker (metadata.import_attempt, with the row's own time), stripped once the row is complete by a compare-and-set followed by a read-back. Only a row carrying that marker is ever adopted after a lost response or removed after a failed write, so a pre-existing memory with the same text is never touched. A row found removed before completion is inserted again, and a row is never counted until it is verified complete. A failed row's cleanup deletes the memory row before its vector and then checks it is gone.
• Truthful partial results: the import stops at the first row that still fails and reports failed and written_ids (exactly the rows it added), replaced: "partial" for a replace, and a message. Also, when they apply: orphan_ids (a failed row the cleanup could not remove; present without a vector, hidden from reads, removed by the sweep), left_marked (a row left marked when the lease was lost; for the sweep), unconfirmed_ids (a row completed just before the lease was lost: a normal memory, reported apart), and post_write (below). A replace is never reported as done with errors.
• One import per store (D1): every strategy (replace, merge, append) first takes the store's single import lease (new table import_lease), before the sweep, the merge read and preparation. A second import on the same store fails fast with error: "import_in_progress" and writes nothing; it does not wait. The lease is valid 30 minutes and renewed at least every 30 s; renewal only extends an unexpired lease this import still owns, so an expired lease is never revived. Ownership is proven by a fresh read before each clear stage, each row's INSERT, its completion and its counting; an import that loses its lease stops at once and writes nothing further.
• Post-write steps under the lease: restoring a replace's embedding-integrity baseline and rebuilding cross-references (with the rebuild's lazy embedding backfill) run only after a clean row phase and only while the lease is proven, fenced before every write. post_write is "done", "skipped" (row phase incomplete) or "incomplete" (lease lost during the rebuild); otherwise post_write_note says to run memory_rebuild_crossrefs. The lease is released after these steps.
• Stale-marker sweep: rows left marked by a crash or a failed cleanup are finished by any later run. The sweep never touches rows of an import holding a live lease; otherwise it completes each marked row at least 10 minutes old that has its vector (strips the marker) and removes each one that has none. It runs at the start of every import, at server startup (every configured store, on a background thread), and on demand with the new admin tool memory_import_sweep (older_than_minutes, default 10).
• Pending rows are hidden from every read: until finished, a marked row is not a memory. List, keyword and semantic search, hydration, get, export, the graph viewer's /api/memories (and, through list/get, /api/graph and /api/memories/{id}), tags, tag validation, the hierarchy, related-metadata batches, duplicate pairs, link/boost/update, backfill_tags, merge-import dedupe and the R2 image migration all skip it; memory_stats counts it separately as import_pending. Neither the corpus repair pass, the embedding integrity audit nor a rebuild embeds it.
• import_attempt is a reserved metadata key: a create or update carrying it is rejected, and an import strips it.

0.4.4

Fast reads. Prompted by live timings from the Mac against memora-all:
memory_semantic_search 10-14 s even warm, memory_get 2 s, memory_list
1.3 s, with clmux meant to become memora's only client.

Read paths

Fake-D1 bench (scripts/measure_read_roundtrips.py: 964 rows, the real MCP
tool functions, one statement == one D1 request). "Warm" is a repeat call,
"cold" the first call after a write. Seconds are modeled at 0.2 s per
request, not measured live.

tool	warm requests	warm seconds (modeled)
memory_semantic_search	16 -> 3	3.3 -> 0.6
memory_hybrid_search	17 -> 4	3.5 -> 0.8
memory_get (current)	8 -> 1	1.6 -> 0.2
memory_get (stale id -> leaf)	20 -> 8	4.1 -> 1.8
memory_get follow=full_history	37 -> 9	7.5 -> 1.8
memory_list	4 -> 2	0.8 -> 0.4
memory_related (empty stored list, cold)	21 -> 1

The first search after a write still costs ~25 requests (integrity audit on
the new epoch + a cold corpus reload); not addressed in this release.

• Semantic and hybrid search score against the epoch-validated in-process corpus snapshot (metadata, tag and date filters applied before top-k, same tie-breaks) and hydrate only the ranked results. The snapshot's repair pass replaces the inline embedding backfill.
• One memories_meta read feeds both the integrity check and the corpus-cache freshness check.
• follow=active / latest: superseded, retired and malformed-link status for a whole page in one statement; latest walks only superseded items, through one shared bounded view.
• memory_get: one statement (row, crossrefs, retirement); walks a chain only when one exists.
• memory_related recomputes score against the snapshot.
• Results are identical to the previous read paths: tests/test_fast_reads.py compares every fast path against the old one on SQLite and fake D1. Any SQL failure, a neighbourhood past the view bounds, or crossref data the old walks read quirkily falls back to the old reads (one WARNING per process for SQL failures).

D1 transport

• One persistent HTTPS connection per worker thread (keep-alive across tool calls; reconnect after 25 s idle). A request is retried at most once, only for a SELECT, only on a reused socket, and only when no response byte arrived; writes are never re-sent. The old urllib path is kept whenever an HTTP(S) proxy variable is set. Not reflected in the bench above (it saves a TCP + TLS handshake per request live).

Query-embedding cache

• Search query embeddings are cached in-process (LRU 256), keyed by backend, model, endpoint and query; empty results and failures are never cached.

Bounded corpus cache

• The corpus snapshot is now also cached for databases that are only searched (before, only after an absorb). About 93 KB per row with 1024-dim vectors (measured; vectors dominate).
• Least-recently-used eviction of whole snapshots under a byte budget across all stores: MEMORA_CORPUS_CACHE_BUDGET_MB (new, optional, default 384; valid values are finite, > 0 and <= 1 TiB, anything else uses the default). A snapshot larger than the budget is served uncached. A store's entries cached under a previous embedding model are evicted on its next load. Evictions are logged at INFO; an evicted store just reloads cold.

Read profiles

• memory_semantic_search, memory_hybrid_search, memory_get, memory_list, memory_list_compact and memory_related return a profile field (per-phase seconds and D1 request counts), also logged at INFO (visible with MEMORA_LOG_LEVEL=INFO).

Behaviour changes

• memory_related: a memory whose stored crossref list is empty now gets that empty list back; it is recomputed only with refresh=True or memory_rebuild_crossrefs, the same staleness rule every non-empty list already had. Previously an empty list was recomputed (a full-store scan) on every call, which also let a list stored empty while the store had no neighbours heal itself on the next read; it now stays empty until refreshed. A memory whose crossrefs were never computed (no stored row) is still computed on read.
• Malformed tags JSON (e.g. from a bad import) is read as untagged everywhere instead of raising: every filter mode (tags_any, tags_all, tags_none, dates, none) and hybrid search treat the row as untagged, and search, get and list return it with "tags": [] plus "tags_invalid": true (the marker appears only on such rows). The embedding rebuild (including semantic search's auto-rebuild) and the corpus snapshot load also read it as untagged. A warning is logged once per memory. Previously any search that scanned such a row, and any get/list that returned it, failed with a JSON error.

0.4.3

Absorb: far fewer D1 round trips, and supersessions that must be verified
against the exact memory they hide. Prompted by live memory_absorb calls
failing the caller's 300 s timeout on update-heavy batches while the server
kept committing, and by #1082 (a parked design idea) being superseded by
#1109 (unrelated work that only shared "clmux agent delivery").

Absorb: D1 round trips

• Measured offline with scripts/measure_absorb_roundtrips.py (a 9-fact update-heavy absorb against a 964-row store through the FakeD1 double; modeled 0.2 s per D1 request, 0.1 s per embedding, 2 s per LLM call): 550 D1 requests / ~120 s -> 204 / ~49 s, with identical decisions (the script asserts this against a pre-change run). These are modeled seconds, not a live measurement.
• Phase 1 is batched across facts: one tombstone-hash lookup, one embedding batch, one hydration of every fact's candidates, one bounded retirement lookup (was per fact, and per candidate).
• Supersession graph reads use a bounded neighborhood view: one memories LEFT JOIN memories_crossrefs query per BFS level plus two retirement queries, instead of per-node crossref and existence reads (each walk re-read nodes several times). Loaded fresh per call, never reused across a graph write; falls back to per-row reads past 1000 nodes.
• add_link checks existence with SELECT 1 instead of fetching both full memories.
• Phase-3 storage embeddings go out as one batch on the dense backend.
• Writes, their order, and corpus-cache invalidation are unchanged. What batching writes would take is in plans/absorb-write-batching-notes.md (not in the repo; plans/ is git-ignored).

Absorb: per-phase profile

• Every call returns result["profile"]: exclusive wall time and DB request count per phase (corpus_load, phase1_prep, embeddings, classification, supersede_plan, supersede_verify, phase3_insert, phase3_link, supersede_resolve, supersede_link, fork_heal, final_checks, inflight, ...) plus counters (LLM calls, embedding requests, late/re-gated/sibling checks). Also logged at INFO.
• D1Connection.request_count counts HTTPS POSTs.

Absorb: supersede gate

• A classifier UPDATE is only a proposal. Absorb supersedes a memory only after gating every leaf it would actually supersede (the classifier's candidate is resolved to the current live leaves of its supersession chain first): the fact's similarity to that leaf must be at least 0.55, and a second, narrow LLM check (_verify_absorb_supersede_llm) shown both texts in full, their tags and the caller's context must answer an explicit yes to same project, same entity and full replacement. No LLM, an error or an unparseable answer never supersedes.
• Leaves that fail stay live: an intentional fork, reported in the decision (intentional_fork, not_superseded, leaf_checks). If no leaf passes, the new memory is linked RELATED to the closest leaf instead (or left unlinked if the check calls them unrelated).
• The write boundary re-resolves and re-reads every leaf: a check is reused only if the leaf's fingerprint (text + tags) is unchanged since it was made, so an update_memory edit in between is re-gated; a leaf that appeared in between is gated then.
• Fork heal no longer lets a concurrent absorb's new memory supersede this call's new memory on the strength of both having passed against the same old leaf: that exact pair must pass the gate, or both stay live.
• The classifier sees 800 characters per candidate (was 300) and a strict UPDATE definition.
• Every supersede and every downgraded UPDATE is logged at INFO with target, score, gate, both reasons, old text and new text.
• Calibration (scripts/measure_supersede_gate.py, 19 labelled pairs in tests/fixtures/supersede_gate_pairs.json, live bge-m3 + openai/gpt-4o-mini): precision 7/7, recall 7/7, 0 false supersedes; true updates scored 0.79-0.89, the #1082/#1109 analogue 0.47. A project-tag-prefix rule was tried and removed: it blocked a genuine update tagged clmux/ vs memora/, and the verifier (which sees the tags) rejected every cross-project pair on its own. 19 pairs is a small set.
• Prompt-injection framing: stored and caller text goes into nonce-delimited data blocks with marker runs defanged, and the prompt says the blocks contain no instructions. Limit: this stops stored text escaping its block, not semantic injection inside it; the tests prove the framing only. What bounds the damage is structural (explicit yes on all three fields, fail-closed parsing, the score floor, the audit log).

Operations

• New opt-in MEMORA_LOG_LEVEL (e.g. INFO): attaches a stderr handler to the memora loggers. Nothing configured logging before, so every memora INFO line (including the absorb profile and supersede audit above) was silently dropped. Unset keeps the old behaviour. The memora-all deploy sets it to INFO; note that the supersede audit lines put up to 500 characters of memory text into the container log.
• No other new env vars or config. MEMORA_LLM_MODEL is unchanged (openai/gpt-4o-mini). Each UPDATE now costs one extra LLM call per leaf it would supersede.

0.4.2

Absorb classification fix for the v0.4.1 gpt-4o-mini switch — a same-day
follow-up.

Absorb

• The classify response parser now recovers memory_id from a real int, or a string that is (optional whitespace +) exactly one of 482, #482, [#482] — whole-string only, nothing else in the value. openai/gpt-4o-mini was found — live, against the real model — to consistently echo the prompt's own [#482] match-display notation back as the value rather than the bare number the prompt asks for, which the old parser rejected outright: every classification on the v0.4.1 deploy came back LLM classify empty; preserving as related despite OpenRouter returning 200 on every call. The accepted forms are deliberately narrow: stripping every non-digit character out of an arbitrary string is unsafe, since e.g. "#482 and #483" would strip to 482483 and "1. [#482]" to 1482 — both digit-run concatenations that can coincide with a genuine candidate id in the same fact's match set and silently misroute the classification onto the wrong memory. Ambiguous text is dropped, not guessed at.
• json.loads now retries against the outermost {...} span if the first parse fails, recovering a JSON object a model prefixed with reasoning or commentary text despite being told not to.
• The classify prompt is more explicit that memory_id must be the bare number, not the bracketed form — a second line of defense, not a substitute for the parser fix, since this model didn't comply with the prior wording either way.
• The raw LLM response is now logged at debug level whenever the model answers but nothing survives validation, so this class of bug is diagnosable from logs without a live repro.