Repository navigation
v7.68.0
Pairs with engine-service-v0.1.143. The engine carries 14 new changesets (vectors-020 to vectors-026, fk-006-1, vectors-023-1 and staging-6-drop-landing-schema), the refusal of a chunk write that has no live manifest row (nexus-z0o2p.24, NX_OWNERLESS_WRITE_MODE: the managed service starts in log-only, and the local engine launch sets enforce), the engine's own hourly reaper of ownerless chunks and the live(c) read predicate (RDR-192), the removal of /v1/staging/* and the staging schema (nexus-z0o2p.27, irreversible by binary rollback, see Removed), the HNSW scan budget (nexus-wbfpw.47) and the census count fix (nexus-wbfpw.60). Two of its wire changes are not additive: a client older than this release has its chunk-before-owner writes refused once the engine enforces, and no 7.x client calls the removed staging routes. Local installs get the new engine with this release, and after nx upgrade every long-lived nx-mcp server and hook-spawned nx must be restarted (nx daemon restart-stale).
Added
-
nx catalog doctor --visible-outside-manifestflags a chunk that live(c) should hide and does not (nexus-wbfpw.26, RDR-192 Step 14). For eachknowledge__collection it takes the censussupersededbucket (manifest-less chunks whose owning note is live) and asks the normal read path, neverinclude_non_live, whether any is visible; each one that is names the collection, note and chash and fails the check. It is a regression canary: a healthy engine finds nothing. It reads at most 300 per call, probes each census page as it lands, probes at most 5,000 chunks and checks a 120-second budget between calls, and printschecked N of M. It probesgetonly, not search. A run cut short before it probed anything reads INCONCLUSIVE and exits 1; with the superseded bucket empty everywhere it reads "nothing to probe" (exit 0, no PASS word,vacuous: truein--json); it reads not applicable on a box with noknowledge__collection or an engine without the census route. -
store_putandnx store putsay which chunks a re-put removed (nexus-wbfpw.25, RDR-192 Step 13). When re-putting a note under the same title deletes the old chunk, the result gains a second line,Superseded: N chunk(s) removed: [<chash>, ...](at most three chashes spelled out, then(and M more)). A first put, an identical re-put, and a re-put whose dropped chunk another document still owns print exactly as before, because the line reports what the engine's sweep deleted, not what the manifest dropped. The engine half is an additive change to the sweep response: eachsweep_detailentry gainsswept_chashes(hex, ascending, at most 300) andswept_chashes_truncated;sweptstays the exact count. Against an engine that predates the field the line carries the count only. When the sweep errors instead (a gate or statement timeout), the note is stored and the old chunk stays, hidden from search, and the engine reaper collects it later if its collection passes the reaper's census and floor; the result says so:Superseded: the sweep did not finish; up to N replaced chunk(s) were not removed. .... A put whose acknowledgement was lost and resent prints neither line, because the resend's sweep removes nothing and the first attempt's answer is gone. -
nx doctorwarns when the engine has seen a chunk write with no owner (nexus-20onx, RDR-223 Phase 3 Step 2). The new "Ownerless writes" row readsownerless_writes_refused_totalandownerless_writes_would_refuse_totalfrom the engine's/v1/status. A count above zero means a client older than the Phase 2 release is writing, a process upgraded on disk still runs its old code, or a caller that is not the nexus client (curl, a script) writes that way; the counters are since-boot and name no cause. After upgrading conexus, restart every long-livednx-mcpserver (one per Claude Code session) and any hook-spawnednx. The row is green and reads "not applicable" when the engine cannot be reached or predates the refusal. The cloud cutover order (log-only soak, thenenforceby environment value and a same-tag redeploy) isdocs/operations/ownerless-write-cutover.md. -
nx t3 quarantine restorebrings chunks back out of aquarantine-collection with no manifest row required, and makes the ones whose own metadata names a live document visible again (nexus-2x9xa and nexus-wbfpw.49, RDR-192 Step 9 Day-2). The only restore until now,gc_restore_rereferenced, needs a manifest row in the collection naming the chunk, and a chunk the reaper quarantined wrongly has none, so recovering it meant hand SQL from thegc_auditchash list. Name the collection the chunks came from and one source:--chash(repeatable),--audit-id(the chash list of areaper_quarantinerow; agc_quarantine_orphansrow, which is whatnx index repoandnx t3 gcwrite, lists only a sample of what it moved and is refused with that reason), or--quarantined-since/--quarantined-before(the chunks in the sibling quarantined from that collection inside the window, which works for any quarantine). The chunks move in one engine statement under the exclusive sweep gate, with text, vector andcreated_atintact and the quarantine stamp and the reaper's own tags removed. A chunk the collection already holds is skipped and reportedpresent, never overwritten; a chash found nowhere reportsmissing;--dry-runclassifies and moves nothing. Moving bytes is not enough, because a chunk with no live owning manifest row is hidden from search and get, so--reattach, the default, also writes the manifest row when it can. Reach, stated plainly: it reaches chunks whose own metadata names their document (written before RDR-108, or bynx store put) and single-chunk legacy notes. A chunk the indexer cut from a file (docs__,code__,rdr__) carries no document key, readsno_live_ownereven when its file's document is live, and comes back as bytes that stay hidden: the remedy there isnx index repo --forceon the owning file, and the output says so instead of printing a re-put command. It writes nothing and reportssuperseded, with a reason, when the document is stamped complete (its manifest is authoritative, so a manifest-less chunk of it is stale, including a document emptied on purpose), is mid index run or its last index run failed (its manifest is partial), was cut from another content hash, already holds a chunk at that position, has the position past its registered chunk count or manifest rows under another collection, or when another stored chunk in the collection or its quarantine sibling names the same document and position (two versions of one position are both refused, even when they land on different pages or in different runs). The manifest is never changed to make room anddocuments.chunk_countis not touched; the output saysM of N attachedwhen a multi-chunk document ends with fewer rows than it registers. A chunk that is not attached stays hidden, and the verb says so with its exit status: exit 3 when every requested chunk is back but at least one stays hidden from search and get (a dry run exits 3 when a real run would), withhidden,ownersandpartial_ownersin--json. In aknowledge__collection the output printsnx store put - --collection C --title 'T'for a chunk with no live owner (nx t3 backfill-manifestdoes nothing for this class); asupersededchunk gets no recipe, because its document's current text is live. The reaper may take a hidden chunk again 30 days after the restore; the output prints that date.--no-reattachmoves bytes only and still reports what reattach would have done; running the verb again attaches chunks that arepresent. Each restore or attach writes onegc_auditrow,operation=quarantine_restore(nx catalog gc-audit list --operation quarantine_restore). New engine routePOST /v1/vectors/gc/quarantine-restore, functionsnexus.quarantine_restore_chunksandnexus.quarantine_reattach_plan(changesetvectors-025); the client verb needs an engine that carries it and exits 4 against one that does not. Exit status is 1 when a requested chunk was missing or had a different embedding width (a dry run too; wins over 3), 5 when the engine refused, and 6 when the engine was busy: a manifest writer held the collection's lock for more than 2 seconds, or the call lost a deadlock, the busy statement rolled back (each quarantine collection is restored in its own transaction, so the engine says when an earlier one had already committed:nothing_moved: false, with its audit ids), and the engine answered a typed retryable 503 (reason: quarantine_restore_busy,Retry-After: 5) rather than a 500. A failure after the first page, an unexpected one included, still prints the report and audit ids of the pages already committed. Seedocs/cli-reference.mdand the procedure indocs/operations/engine-reaper.md§ Getting a chunk back. -
Engine:
POST /v1/pipeline/reset_uploaded, anduploaded_chunksinGET /v1/pipeline/counts(nexus-z0o2p.11). A streaming PDF upload killed hard leaves the head of the document flagged uploaded, and the writer state that made sending only the tail safe died with the process.reset_uploadedputs every chunk of one run back to not-uploaded and zeroeschunks_uploadedin one transaction, and keeps the extracted pages, the chunks and their embeddings. It checksrun_epochlike every other write route, so a run another process took over cannot reset the new owner's flags; a ref naming no row, or another tenant's, resets nothing.uploaded_chunkscounts the embedded chunks flagged uploaded. The client refuses an engine that answers without the new field.
Changed
-
The engine's orphan GC waits 30 days after a chunk loses an owner, and for 30 days after this engine is first deployed nothing that already exists is old enough (nexus-wbfpw.15, .16, .17, RDR-192 Steps 7 and 8).
nexus.chunk_is_reapableis the one predicate for "garbage collection may take this chunk": no own-collection manifest row in any owner state, the later oflast_written_atand the chunk's orphaning time older than 30 days, and not in a quarantine collection. Database triggers on the manifest record the orphaning time in a new table,nexus.chunk_orphaned_at, whenever a manifest change drops a chunk's owner row (any owner, whether or not another remains; a chunk written within the last hour is not recorded), so the 30 days run from when the chunk became ownerless: a deleted file's chunks and the old tail of a multi-batch re-index stay for 30 days. The record is a separate table, so dropping 5000 chunks' manifest rows adds tens of milliseconds (one laptop-container measurement: 51 to 60 ms on a 35,000-chunk table, against 11 to 14 s when the record was a chunk-row update) and rewrites no chunk row or vector index.gc_quarantine_orphansand its bounded form use it, so the cleanup at the end ofnx index repomoves less than it did (often nothing right after a delete).vectors-020addedlast_written_atwithDEFAULT now(), so every existing chunk takes the migration time and none is reapable until 30 days after the deploy; the first cleanups after an upgrade, and any reaper census, read zero for that long. New read-only routePOST /v1/vectors/reapablelists what the predicate selects, with a keyset cursor (after_chash); no client verb uses it yet (nexus-wbfpw.18, .19). -
The engine refuses a chunk write whose chashes have no live manifest row (nexus-z0o2p.24, RDR-223 Phase 3).
POST /v1/vectors/upsert-chunksand/store-putanswer 422 (reason: ownerless_chunk_write, naming/v1/catalog/manifest/write_manyand/append) when any chash in the request has no live manifest row in the collection; writes of chunks a live document already owns (a collection re-embed, a metadata refresh) are unchanged. Not additive. A client older than this release gets 422 on its ownerless writes, so after the upgrade every long-livednx-mcpserver and every hook-spawnednxmust be restarted, not merely upgraded.NX_OWNERLESS_WRITE_MODEisenforceorlog-only; unset meanslog-only(the engine logs and counts such writes,ownerless_writes_would_refuse_totalonGET /v1/status, and refuses nothing), and the local engine launch setsenforceitself, so a local install enforces from the first boot of the upgraded engine, with no soak: restart everynx-mcpserver and hook-spawnednxright afternx upgrade(nx daemon restart-stale, thennx doctor'sProcess freshnessrow), or the first write from a process still running old code is refused. The log line names the client byUser-Agentand the newX-Nexus-Client-Versionheader that the shared client classes now send (the vector client, the T2 stores, the catalog client, the token store and the scratch store;absentmarks a client older than this release, or an engine request that goes through none of those classes, such as a hook call or a status, version or health probe, none of which writes a chunk); it is rate limited to one per route, tenant and collection per minute. The 422 text says to upgrade conexus and restart nx-mcp / Claude Code sessions.nx collection re-embedresends only the chunks that are still owned when a chunk loses its owner mid-run. -
The engine reaps ownerless chunks on its own, once an hour: it moves them to quarantine, and deletes what it moved 14 days later (nexus-2x9xa, RDR-192 Step 9). A pass visits every tenant that has a service token, plus the default tenant, and every collection of every prefix (
knowledge__,docs__,code__,rdr__). It first expires the tenant'squarantine-collections, but only the chunks the reaper itself moved: every moved chunk is taggedquarantined_by: engine-reaper, and the new functionnexus.reaper_expire_quarantinedeletes tagged chunks stamped more thanNX_REAPER_QUARANTINE_RETENTION_DAYS(14) days ago, with no fraction floor and at most 5,000 per origin per pass, never a chunk a manifest row of the origin still names (those are counted asexpiry_protected, which is not a refusal). Quarantine thatnx index repoornx t3 gcfilled carries no tag; the engine never touches it, and it keeps the client'sNX_GC_QUARANTINE_DAYSexpiry (so a repository nobody has indexed for months keeps its quarantine). The split is symmetric: the client's expiry (nexus.gc_expire_quarantine, the callnx index repomakes, and thegc/expire-quarantineroute with or withoutforce) now skips the chunks the reaper tagged and counts only its own rows toward its floor, so a clientNX_GC_QUARANTINE_DAYSbelow 14 cannot shorten what the reaper moved. Then it moves the chunksnexus.chunk_is_reapableselects (no owner row in any state, 30 days since the chunk last had an owner or was written) into the collection'squarantine-sibling, at most 300 per collection per pass. It is the safety net for the post-commit sweep's fail-open paths (a gate timeout, a statement timeout) and for a crashed multi-request run, which leave replaced chunks behind. A moved chunk is moved back when a laternx index repoor heal names it in a manifest row of a repository collection's origin; for aknowledge__collection nothing does (a re-put embeds a fresh chunk in the origin) and a manifest row that names the chunk only protects the quarantine copy from expiry.nx t3 quarantine restorerestores by chash,gc_auditid or window (see its entry). A pass writesgc_auditrows (engine-reaper, operationreaper_quarantine, the chashes, one per collection it moves from) and logsevent=reaper_passfor every tenant,candidates=0included. It refuses, moves nothing, logs aWARNwithreason=and a runningrefused_total, and writes onegc_auditrow (operationreaper_refused, with the reason, counts and up to five title/source_path samples, once per collection each time the reason changes) when: the tenant has no verifiedrdr192-manifest-backfillrecord; the collection is not registered or notlive; the collection's whole reapable set is at least 100 chunks and more than 0.25 of the collection (judged before the census, so a refused collection pays for none); the manifest-less census, re-run in the engine and bounded to 60 seconds, reads anylegacy-unmanifestedorunclassifiedchunk, reads a chunk count other than the one being judged, or times out; or another statement (the dry run, the move, one sibling's expiry) hits its 25 second bound (statement_timed_out_total; an expiry's row is filed under thequarantine-collection's name). A collection whose census, dry run, move or expiry times out three passes running rests for 2, 4, 8 passes and so on, at most 24 hours (census_backoff_total;census_timed_out_totalcounts the timeouts), and a run that spends its wall-clock budget resumes where it stopped, so a slow collection cannot starve the others. A busy sweep gate and a row-lock timeout skip the collection until the next pass, with their own counters (gate_busy_total,lock_timeout_total); they are not refusals. Settings (engine environment):NX_REAPER_ENABLED(true/false; any other value warns and leaves it on),NX_REAPER_INTERVAL_SECONDS(3600, at least 60),NX_REAPER_BATCH_SIZE(300, the most it accepts),NX_REAPER_FLOOR_FRACTION(0.25; the floor on the move only, the expiry has none; 1.0 turns it off),NX_REAPER_FLOOR_MIN_CHUNKS(100),NX_REAPER_WALL_CLOCK_BUDGET_SECONDS(600, a whole run),NX_REAPER_CENSUS_TIMEOUT_SECONDS(60),NX_REAPER_QUARANTINE_RETENTION_DAYS(14, the engine's retention for what it moved),NX_REAPER_FLOOR_EXEMPT_COLLECTIONS(comma-separatedtenant/collectionentries, or bare collection names that match in every tenant, exempt from the MOVE floor only, for draining a collection that is legitimately mostly garbage; logged at boot and on every pass that uses it, and the move's audit row recordsfloor_fraction1.0); a malformed value logsevent=reaper_setting_invalidand uses the default. There is no grace setting: it is the predicate's 30 days. The first pass runs 60 seconds after the engine starts. For 30 days after the engine carryingvectors-020is deployed nothing that already exists is old enough, so the reaper moves nothing in that time; one cliff follows at deploy plus 30 days, when every existing orphan ages at once, drains at 300 per collection per hour, and is refused by the floor in a collection that is mostly orphans. A tenant with chunks and no live service token (a cloud tenant idle for more than seven days) is not visited until it holds a token again. New changesetsvectors-024(functionsnexus.reaper_quarantine_chunksandnexus.reaper_expire_quarantine) andvectors-026(nexus.gc_expire_quarantineskips the reaper's rows); no row changes at deploy. The operator runbook isdocs/operations/engine-reaper.md. -
nx t3 gcquarantines what the engine's reapable predicate selects instead of deleting by chunk id, and--orphan-windowis gone (nexus-wbfpw.18, RDR-192 Step 8). Paired client change: it needs the next engine tag, which carriesPOST /v1/vectors/reapableand the reapable-awaregc_quarantine_orphans(nexus-wbfpw.16, .17); against an older engine the verb refuses and says so. The verb used to compute orphans in the client (catalog alive-set againstindexed_at) and delete them by id, which could hard-delete a chunk a client re-wrote after the snapshot. It now lists withPOST /v1/vectors/reapable(advisory, paged by keyset) and acts throughPOST /v1/vectors/gc/quarantine-orphans, whose own statement carries the predicate and takes the sweep gate, so a racing write wins. What you will see change: chunks are moved to thequarantine-*collection, not deleted, and the move frees no storage until the chunks expire. Each side expires only what it moved: the client expires the chunks the client moved once they are older thanNX_GC_QUARANTINE_DAYS(default 14), and the engine expires only the chunks its own reaper moved. The client expiry is run bynx index repofor a repo's code, docs and rdr collections and bynx t3 gcfor any collection it moves (after its own move, and on a run with nothing to move), so aknowledge__*quarantine, which no repo index sweeps, expires too; the verb printsClient expiry of <quarantine>: N expired, M refused. The operator restore verbnx t3 quarantine restoreships separately (nexus-wbfpw.49) and refuses this verb's audit rows (a sample, not every chash), so restore by window or--chash;quarantine-*collections now appear in collection listings; the clock is the later oflast_written_atand when the chunk last lost an owner (30 days, fixed in the engine), notindexed_at, so a chunk with noindexed_atis now a candidate once old enough;--orphan-windowis refused with a message naming its removal; the engine records onegc_auditrow per batch and the verb's ownt3_gcrow,t3_gc_chunks_deletedevent,Summary: deleted Noutput andProtected by pending tombstonesline are gone; for 30 days after the engine upgrade nothing that already exists is reapable, so the verb reports0 reapable chunk(s); nothing to doand exits 0, and there is no replacement for--orphan-windowas a way to reclaim space sooner. The move is the bounded form of the route (2,000 rows per call, looped until drained), asking for the engine's full 5,000-chash audit sample so eachgc_auditrow records its batch. New refusals, each also reported by a dry run: the manifest-less census must readlegacy-unmanifested = 0andunclassified = 0for the collection (read at the start of the run and again right before the move; a live legacy note with no manifest row reads reapable once old and the engine route takes no exclusion list), and a pass of at least 100 reapable chunks that is more thanNX_GC_FLOOR_FRACTION(default 0.25) of the collection's stored chunks is refused unlessNX_GC_FORCE=1(the engine's own reading: the 100 minimum counts the reapable set, the fraction is compared by division). That floor is the verb's own for now: the route carries none, the reaper's floor has no HTTP route, andindexer._prune_deleted_filescalls the same route with no floor (a floor on the route is nexus-wbfpw.52);NX_GC_FORCE=1 nx t3 gcis the way past a floor refusal. A--dry-runthat names a refusal now exits 1, so a script can gate the real run on it. It reuses the GC family's variable names, not the engine reaper'sNX_REAPER_FLOOR_FRACTION. A census response withoutscope_chunk_totalis refused, and a bounded drain that stops short (no progress, the batch cap, or a failed batch after earlier ones moved) exits 1 naming how many chunks were already moved; each batch commits on its own, so a re-run is safe. A failed batch's own outcome is reported as unknown (a timeout can follow a commit), and a drain that moves more than the listing named says so. The RUNFENCE breaker,--allow-incomplete-index-state, the unknown-collection refusal and--allow-empty-manifest-setare unchanged. Removed as dead code with their last caller:HttpVectorClient.delete_by_chunk_ids,T3Database.delete_by_chunk_ids,HttpCatalogClient.chashes_for_collection_with_tombstone_protectedandHttpCatalogClient.record_gc_audit. -
nx store list --reapable -c COLLECTIONlists the chunks the engine would reap (nexus-wbfpw.19, RDR-192 Step 10). Paired client change: needs the same next engine tag (POST /v1/vectors/reapable). Read-only, advisory, paged by keyset; prints chash,ownerless_since, days since that instant,last_written_at,created_at, title andcatalog_doc_idper chunk, or0 reapable chunks in <collection>when there are none.ownerless_sinceis the engine's grace clock, the later oflast_written_atand when the chunk last lost an owner, so the age answers "how long ownerless";POST /v1/vectors/reapablereturns it as a new additive field, and against an engine that omits it the age falls back to the write age. It is not derived fromcreated_at.--collectionis required, and a name no collection carries is refused likenx t3 gcrefuses it, not reported as0 reapable. -
nx store put,nx memory promote,nx catalog importand MCPstore_putwrite a note in one request and say the same thing about every outcome (nexus-z0o2p.16, .17, .18, RDR-223 Phase 2). A note's chunks and its catalog owner rows go to the engine together, so a chunk can no longer be stored without an owner, a client that dies mid-write leaves nothing ownerless, and a failed write leaves the note's previous version as it was. The four commands share one wording (failure_message) and one firing of the post-store hooks (fire_note_chains). What you will see change: a refusal the client makes before it sends anything (a stale embedding profile, a missing Voyage key, a retired collection name) now reads as its own remedy followed by "The note was not written and its chunks and manifest are unchanged; run the command again once that is fixed" (and, if the note was stored before, that its index state may read 'failed' until a write succeeds, because the write began and then failed the index-run fence), where it used to say "retry is safe" or "may already have succeeded". An engine refusal reads "could not store ...; retry is safe" and is the only outcome that mentions chunk metadata having been refreshed. A write the engine accepted but would not stamp complete is its own outcome ("refused to stamp the document complete ... stays 'indexing'"). MCPstore_putreports those three client-side refusals as not stored and removes the catalog row it minted; it reported them as "may already have succeeded" and kept the row. A landed but unstamped note no longer says it "could not confirm the catalog manifest landed". A note that could not be cataloged now names the cause.nx store putandnx memory promoterefuse empty input with "nothing to store" instead of a traceback, as MCPstore_putalready did. An outcome a command does not recognise is an error, never "Stored:". -
nx catalog importexits 1 when any note failed, is uncertain, or was written but not stamped complete (nexus-z0o2p.18). It exited 0 on a partial import. Unresolvable links alone still do not change the exit status. Imported chunks carrysource_agent="recovery-import"; they carried none. -
nx memory promoteprints, N chunksfor a note split to the embedding model's token window (nexus-z0o2p.17), asnx store putdoes. A promoted note used to be one chunk whatever its size, so a long entry could exceed the model's window. -
nx index repowrites no chunk of a file that has no catalog document (RDR-223, nexus-z0o2p.20). Those chunks went through the legacy ownerless upsert, where thelive(c)filter hid them from every read, and the engine will refuse them. The run now refuses such a file before chunking it, names it in the end-of-run summary with its cause (register_failed,catalog_hook_failed), records it innx index failuresunderIdentityLessFile(a deliberate worktree or temp-dir refusal is not recorded, sonx doctorstays green for it), and holds the--since-headbase back. The run exits non-zero in that case; a per-file register failure or a catalog outage used to exit 0. A chunk shared with a file that does have a document is now written only by that document's own write. Worktree and temp-dir refusals keep their own summary line and do not fail the run. The oversize-file fallbacks (code, prose, PDF) write nothing for such a file either (causeoversize_no_catalog_document), count it as dropped, hold the--since-headbase, record it innx index failures, and report it as not indexed rather thanskipped: index fresh.nx doctornames each recorded file with its cause and the remedy. A failed durable write prints a warning. A non-service T3 (the in-memory test topology) keeps its write. -
The pre-PG upgrade path says to run the 6.18.1 hop against a local engine, and gives the cloud as a second hop (nexus-z0o2p.27). The stranded-install banner, the retired-sqlite message, the
nx doctorremedy,docs/migration-runbook.md, Getting Started, the CLI reference, the plugin'supgradeskill andAGENTS.mdtell a pre-PG user to stop any local service a 7.x install left running and runnx upgradeatconexus==6.18.1against a local engine: note yourservice_urlandservice_token(the cloud steps need both again), clearNX_SERVICE_URL,NX_SERVICE_TOKEN,NX_SERVICE_HOSTandNX_SERVICE_PORTin the shell, deleteservice_url(andservice_token) undercredentials:in~/.config/nexus/config.yml, which is a file edit because there is nonx config unset, andexport NX_LOCAL=1for the hop. Both halves are needed:NX_LOCAL=1does not by itself override a configuredservice_urlat the pin, and the pin's provisioning refuses whenis_local_mode()is false (NX_LOCAL=0,install.mode: managed, or a ChromaDB Cloud key with no mode record), whichNX_LOCAL=1overrides. This is read from the 6.18.1 source, not run at the pin. At the cloud switch,unset NX_LOCALbeforenx config set service_url. Nevernx guided-upgrade --service-url, never a managed endpoint. Voyage-embedded data needs the local engine run withNX_VOYAGE_API_KEY, or it is refused or re-embedded to bge and is not expected to import into a Voyage cloud. To reach the managed cloud, make it a second hop with the current client. That hop has not been rehearsed end to end (nexus-xbqh9). Hand-carry any T2 memory you need BEFORE switching to the cloud (nx memoryhas no export verb, by hand it does not scale past tens of entries, and it is lossy:getprints content only, so tags, agent, TTL and timestamps are not carried; read by id, and pass the body toputon stdin). The hop usesnx store export --all/nx store importandnx catalog export/nx catalog import(catalog import last, after re-indexing from source), andnx stranded ackonce on the cloud. It does not carry T2 plans, taxonomy (topics, assignments, links, centroids),document_aspects, the aspect queue,aspect_promotion_log,frecencyandrelevance_log(so a note's TTL is lost and an expiring note becomes permanent), the telemetry stores, DEVONthink highlights, or curated catalog metadata (author, year, corpus,meta, collection supersession); owner tumblers are re-minted. A local bge-768 collection is sent to the cloud engine under a bge name and what it answers is unverified (nexus-xbqh9); re-index it from source instead, which for code, docs and rdr content is the cheaper path anyway.nx catalog export --helpanddocs/catalog.mdnow name the realnx store exportverb. -
nx store importwrites each chunk together with its owner row and stores the vectors in the file (nexus-z0o2p.19, RDR-223). The import used to store every page of chunks through/v1/vectors/upsert-chunks(the engine re-embedded the text) and write the owner manifests once at the end, so a client that died mid-file left chunks with no owner, and an imported document readindex_stateNULL. It now reads the file to hash it and to count each document's records (and once more to write), resolves each owner group to its document (one document however many groups resolve to it, with the summed count), then writes a document's first request as a manifest replace (a document of one request also carries its sweep and stamp in that request) and every later page as an append, in pages of up to 300 records whatever the collection's embed cap, each with its chunks and their exported vectors (the engine embeds nothing and checks the model and dimension; one that embeds anyway fails the import). Every payload chunk replaces a stored vector with the file's, including a chunk another live document shares, metadata is merged so keys another document's enrichment set survive, and the summary says how many stored vectors differed. A document is swept and stampedcompleteon its own last page, inside that request, so a crash leaves only the unfinished documentsindexing, and the same command run again finishes them. Keep-existing (nexus-wbfpw.40) is unchanged with one exception: a document that owns chunks keeps its chunk list, and the file's chunks for it are now left out of the import and counted in "Skipped" (before, they were stored hidden);unowned_count/unowned_documentskeep their names and now mean chunks left out. The exception: a document leftindexingorfailedwhose recorded hash equals this file's hash is this file's own dead run, and a rerun finishes it with the append form (nothing dropped); a document with another hash, or alreadycomplete, stays kept; a kept document another run leftindexingorfailedis reported as unfinished (with the delete-and-reimport remedy), not as one "with a different chunk list". The summary also counts documents whose replaced chunks the engine's fail-open sweep could not delete (sweep_skipped;nx t3 gcremoves them).--skip-existingnow means "send no text or vector for a chunk the target already holds" (the record still gets its owner).taxonomy__*andquarantine-*targets are refused before anything is written. Two records of one document at one position keep both chunks, the later one past the document's highest position. Needs an engine with the RDR-223 routes (append_manywithcomplete,begin-manywithsnapshot_manifest, supplied vectors); against an older one the import stops at its first request with "the engine is older than this client" and what to do (upgrade the local engine, or wait for the cloud deploy), marks the documents that request fenced, and has no fallback. A legacy two-segment target is written with the model its collection is registered with, not the export header's prefix-based guess (which a local install's engine refused). -
nx index pdfwrites each chunk together with its owner row, and a killed upload resumes without extracting the PDF again (nexus-z0o2p.11, .15, RDR-223). The streaming pipeline sends every 128-chunk batch with its manifest rows in one request, so a process killed part way leaves every chunk it wrote owned. The non-streaming branches (a small PDF, a repo PDF over the chunk cap) write the same way. A killed streaming upload re-sends from chunk 0 on the next run throughreset_uploaded, without re-extracting, and the engine skips re-embedding chunks it holds. The completion stamp comes after the title, author and extraction-method passes, and a failed stage no longer releases the writer's last request. A PDF with no catalog document is refused before extraction, and--dry-runwrites only to a throwaway in-memory store. The checkpoint writer and reader andChunkLandingUnverifiedError, which nothing used any more, are deleted. -
Every writer path stamps its document complete last, after its post-store hooks (nexus-z0o2p.34, RDR-223 Technical Design 6). A process killed in a hook (taxonomy assignment, aspect enqueue, catalog enrichment) used to leave a document that read complete, and the next run skipped it as fresh. It now leaves the run fence at
indexingand the rerun redoes the write and fires the hooks again. This covers the oversize code, prose and PDF fallbacks, the batched flush innx index repo, the note writers (MCPstore_put,nx store put,nx memory promote, the recovery import) and the.nxexpimport. A refused or failed stamp leaves the fence atindexing. -
Engine: every HNSW search sets a scan budget (nexus-wbfpw.47, RDR-192). With most of an index filtered out by the
live(c)predicate, pgvector's default scan limit stopped before it found ten live rows: measured recall at 10 was 0.85 to 0.89 at 98% dead chunks on shared 768- and 1024-dimension indexes, and 1.000 with the budget. Every search path (plain, both hybrid branches, the combined-query pair and the taxonomy centroid lookup) now setshnsw.max_scan_tuplesto 200,000 and a scan memory multiplier derived from a 16 MB per-search budget and the server'swork_mem(4 at the stock 4 MB, 1 at 384 MB).NX_HNSW_MAX_SCAN_TUPLESandNX_HNSW_SCAN_MEM_BUDGET_MBoverride them on the engine; both are validated at boot and logged ashnsw_scan_budget. A filtered search that cannot fill itsLIMITcan now scan up to the cap before it stops, and what that costs on a large local index is unmeasured.
Removed
-
POST /v1/vectors/upsert-reference-only(nexus-z0o2p.24, RDR-223 Phase 3). It wrote a chunk with no manifest row and had no client caller; the production edge log shows no request to it from 2026-07-03 to 2026-10-01. It answers 410 Gone. The repository method behind it stays for test fixtures. -
nx t3 reidentifyandscripts/migrate_art_papers.py(nexus-z0o2p.25, RDR-223 Phase 3).nx t3 reidentifywas the RDR-108 verb that re-keyed legacy chunk ids; it re-upserted throughcol.upsert, which the service collection stub does not have, so--no-dry-runraisedAttributeErroron every real install, and a pgvector chunk id is its chash by schema. The ART migration script was a one-off over the retired SQLite T2 and catalog files.nx t3 backfill-manifestand the othernx t3verbs are unchanged. The verify-fill code indb/reconcileanddb/embed_migratewent too; neither had a caller. -
nx collection backfill-hashandnx catalog backfill's pass 4 (RDR-223 Phase 3 follow-up). The hidden verb re-upserted each chunk throughcol.upsertto addchunk_text_hashmetadata; the service collection handle has noupsert, so the verb raisedAttributeErroron every real install (the defect class that removednx t3 reidentify), and a pgvector chunk id is its chash by schema so there is nothing to backfill.nx catalog backfillloses its chunk_text_hash pass, which was already a no-op on the service-backed T3. -
The engine's
/v1/staging/*routes and thestagingschema (nexus-z0o2p.27, RDR-223 Phase 3). The RDR-180 land-then-transform landing zone (POST /v1/staging/load/{store},/embed_fill,/promote,/finalize,/clear,GET /v1/staging/counts) is gone, along with its promote code and the sweep's guard overstaging.document_chunks; changesetstaging-6-drop-landing-schemadrops the seven staging tables, discarding any rows landed there, and drops the schema unless something stands in the way: an object in the schema the changeset did not name, a view or foreign key outside it that depends on a named table, or a migrating role that owns neither. The boot goes on in each case and that object stays. The migration connection does not surface the changeset's NOTICEs, so the catalog is the only evidence of what was left (pg_namespaceandpg_tablesfor schemastagingreturn no rows after a complete drop). No 7.x client calls the routes. The one client that did was the Chroma-era migration in conexus 6.18.1 and earlier. Aimed at a managed cloud engine, that path is unsupported: after this change it has no routes to land through, and what it did against a current engine before is unmeasured (nexus-6g218). Against a local install it uses its own pinned engine and is unaffected. Irreversible by a binary rollback (nexus-z0o2p.44): once the walk has dropped the schema, an engine at v0.1.142 or older aborts at boot, because its always-run grants changesetgrants-nexus-diag-3grants on schemastagingand does not check that it exists, wherever the rolenexus_diagexists. To go back, create the schema first (CREATE SCHEMA staging, as the admin role), then start the older engine; that was rehearsed on a restore of production (the older engine boots, runs no new changeset, and re-grantsnexus_diag), and restoring from a point in time is the fallback. A local client downgrade to 7.67.0 meets the same refusal only where anexus_diagrole exists, which has not been checked for the local bundle.
Fixed
-
Backfill and the reindex delete-guard read stored chunks, not live ones (nexus-wbfpw.35, RDR-192 Phase 2 gate).
nx catalog backfill --from-t3, the RDR and paper backfills and thenx collection reindexpre-delete scan read through thelive(c)filter, so a collection whose chunks are all unowned looked empty to them: the backfills found nothing to register, andreindexskipped its "would destroy every chunk" refusal and deleted the collection. They now passinclude_non_live, and the backfills filter collections onstored_count. Reading stored chunks also brings back the chunks of documents deleted but not yet purged, so the verbs skip a path the catalog holds only as a deleted document (they used to register or re-index it as a new live one) and say how many they skipped. The match is exact: a source path equals the tombstone'sfile_path, made relative to its owner'srepo_rootwhere that applies, never a suffix, and each tombstone'srepo_rootmakes only its own relative form.reindexkeeps a source that any live catalog document in the same collection names, under any owner and in either path form (one file can be catalogued under two owners, and a deleted document's owner is only one of them), so it cannot purge chunks it then fails to rebuild; the backfills skip a deleted document's path unless a live document holds it under that same owner (delete a file's document, then index the file again). The guard fails closed:reindexrefuses when the catalog cannot be read or when every source is a deleted document, andreindex, the three backfills andnx catalog orphan-backfill(dt-link,synthetic,apply-csv, which now leave a deleted document's title groups alone) refuse on an engine whoseGET /v1/catalog/trashentries carry nofile_path, instead of running without the guard. Needs an engine newer thanengine-service-v0.1.142, which addsfile_pathto those entries and orders them bydeleted_atthen tumbler so a batch delete cannot hide a tombstone between pages; againstv0.1.142the verbs above refuse once the trash holds anything. The verb that writes,re-embed, stays on live rows. For a chunk no document ever owned, a client re-write would refresh the reap grace anchor and extend its life. For a chunk owned only by a deleted document that argument does not hold (under the plannednexus.chunk_orphaned_atdesign a refresh extends nothing); the reasons there are the Voyage cost of a re-embed and that patching a chunk of a deleted document is pointless.re-embednow says how many stored chunks stayed on their old vectors. -
MCP
store_listsays "N stored" (nexus-wbfpw.35). Its header and footer read "showing 1-20 of N" with N the stored chunk count while the rows are live ones; they now sayN storedasnx store listdoes, and the next-page hint appears only after a full page (a page of the engine's 300-row cap counts as full, solimit=500no longer reads as the end). -
nx collection reindexnow refuses, and asks for--force, on any collection holding hidden chunks with no source (nexus-wbfpw.35). The scan reads stored chunks, so chunks with no live catalog document (unowned, or owned only by a deleted document) count as sourceless alongside manualstore_putentries, where before they were invisible to it. The refusal says so and points atnx collection info, which reports live and stored counts; a collection of only such chunks is refused outright. -
The single-file index failure no longer says the chunks are searchable (nexus-wbfpw.34, nexus-wbfpw.35).
nx index pdfandnx index mdon a file with no catalog document wrote nothing since RDR-223 but said "indexed ... searchable but orphaned"; the message now says nothing was written, and for chunks written without a manifest row it says they are hidden until repaired. -
Engine: the hybrid search dispatch counts the rows search returns (nexus-wbfpw.35, changeset
vectors-023).text_gate_probe_<dim>kept the old tenant-wide tombstone anti-join afterlive(c)moved every other read path, so a gate that was selective among live rows could read as dense and take the HNSW-first plan. It now useslive(c). -
nx index repowrites an oversize file's chunks together with their owner rows, and fails the run when a file is deferred (nexus-z0o2p.14, RDR-223 P2.4). The fallbacks for a code file over 300 chunks, a prose or RDR file over the CCE batch and a PDF over one batch wrote chunks withupsert-chunksand the owner rows in a later manifest write, so a client that died between the two left chunks nobody owned. They now send chunks and owner rows in the same request, with the metadata merge the old upsert had (bib_*set bynx enrich bibsurvives a forced re-index; a key the indexer owns and dropped is cleared), and so does every normalnx index repochunk flush. A file deferred on a transient write error (429, 502, 503, 504, the embed timeout, a connection failure) is now named in a warning and makes the command exit non-zero; it used to exit 0. Needs the engine that carries themetadata_mergeroute fields; an older engine fails each flush loudly. -
nx upgradebackfills legacy notes that RDR-192 hid from search (nexus-wbfpw.41). A note stored before nexus-b6enc has a catalog document but no manifest row, and the engine'slive(c)predicate has hidden such a chunk since RDR-192 Phase 2; only the operator tenant had been censused and backfilled, so every other install kept invisible notes that the coming reaper would delete. The upgrade ladder has a new rung,rdr192-manifest-backfill: it censuses every non-quarantine collection, backfills each one holding a legacy-unmanifested chunk, and records completion innexus.ladder_completionsonly after a fresh census reads zero, with a census summary in the record. It deletes nothing and is safe to re-run. A census answer without its bucket totals, an empty listing over a catalog that holds manifest rows, an unreachable engine and an old engine all defer rather than read as clean. A note the backfill cannot heal defers the rung:nx upgradefinishes its other steps and exits 0, names the collections, the skipped-document counts per class and the remedy (no verb heals it, and its text is hidden fromnx store get, so re-put your own copy under the same title;nx t3 census-manifest-lessnow prints each legacy row's owner title), and is not even re-examined for 24 hours at the same package version (NX_RDR192_BACKFILL_RETRY=1retries now). A backfill that errors is never remembered and is retried at the next session start; an unwritable config directory or a census outage during verification defers instead of failing; a clean tenant pays for one census.detect()never takes a census, sonx doctorandnx upgrade --dry-runstay cheap.nx t3 backfill-manifest --only-gappedno longer manifests a document with more matched chunks than its registered chunk count or two chunks at one position (a note with a repeated piece, fewer matched than registered, is still healed), and re-reads the manifest just before writing. The engine side isRdr192BackfillGate, which the reaper (nexus-2x9xa) calls and which refuses to reap on a tenant without the record. Seedocs/migration-runbook.md. -
Engines v0.1.133 to v0.1.142 overstated the census
scope_chunk_totalon a collection where one chash is named by several manifest rows; fixed in v0.1.143 (nexus-wbfpw.60, .62). Identical text shared by N documents was counted N times (code__1-15read 8,913 for 5,641 chunks). The count is the denominator ofnx t3 gc's 0.25 floor and of thechunks=detail therdr192-manifest-backfillrung prints, so against those engines the floor ran up to about 58% looser than intended, and the reaper logged falsereaper_census_scope_much_largerwarnings. Chunksnx t3 gcmoves go to quarantine, so a move made under the looser floor can be restored. The new engine counts chunks; buckets, items and the other totals are unchanged and no changeset moves. -
Engine: Liquibase's history table is pinned to
public, and a split history refuses the boot (nexus-q81g7). Liquibase putsDATABASECHANGELOGincurrent_schema(), and the default search path makes that the schema named after the migrating role once one exists. A role namednexus,t1orstagingbooted once, created its own schema mid-walk, read an empty history there on the second boot and failed onCREATE TABLE nexus.memory. The migration session now pinssearch_pathtopublicand logs its role, schema and path (schema_migration_session). Ifdatabasechangelogexists in another schema with more rows thanpublic's, the boot stops before any changeset runs and namesALTER TABLE <schema>.databasechangelog SET SCHEMA public.nx doctorand the daemon readpublic.databasechangelogexplicitly. A role with no like-named schema, such as production'snexus_admin, already resolved topublicand is unchanged. -
nx index repodoes not bisect a 429 or a deadline-aborted flush (nexus-eoido). A failed multi-file flush was split and each half retried with a full budget, including on a 429 or an engine request-deadline abort, so one refused request became 1, 2, 4 and more requests that each re-embedded and re-billed. A throttle now fails the batch as one: its files are deferred to the next run's staleness check, theRetry-Aftergoes to the shared rate brake, and the throttled files are reported apart from failed ones. A client-side call deadline defers without pacing the brake. Every other failure still bisects, a deadline abort once. -
The local MiniLM model is provisioned once under a lock (nexus-ccre5). Several cold first uses made several downloads, and one could delete a peer's just-published files. Provisioning now takes an exclusive lock and rechecks inside it, so N cold callers make one download; transient HTTP errors (429, 5xx, transport) retry with backoff, a checksum mismatch still fails loud, and temporary archives older than a day are swept.