v7.75.0

Pairs with engine-service-v0.1.154 (REQUIRED_ENGINE_VERSION moves from 0.1.152; v0.1.153 was deployed to the managed service but never pinned). Search and indexing latency, in both modes. A default search no longer carries vectors, and the contradiction check is opt-in. Chunk counts are cached for 15 minutes. In local mode the cross-encoder rerank scores a capped set of 60 candidates, and indexing runs about 1.8 times faster. Three local-mode first-run defects are fixed.

Changed

• The per-collection chunk counts a default MCP search sizes its fan-out from are cached for 15 minutes instead of 60 seconds (search latency, Sam's decision 2026-10-08). The counts come from the full GET /v1/vectors/stats, about 380 ms on a local engine and 0.73 s on the managed service, and interactive searches are spaced more than a minute apart, so nearly every one paid it (measured: 724 ms cold, 209 ms warm, 588 ms after a 65 s wait). The names and registry rows keep their 60 s clock and refresh from the cheap catalog-only routing listing, which now carries the counts forward instead of discarding them. This process's own store_put, store_delete and new-collection registration still drop the counts at once. The one thing the longer window can get wrong: a collection that another process grew past the 3-chunk fan-out floor is skipped by the default fan-out for at most 15 minutes (a collection named explicitly is never affected). store_list prints the sizes, so it keeps the 60 s freshness (get_collection_counts(max_age=...)). Constant: mcp_infra._COLLECTION_COUNTS_TTL.
• In local mode the rerank scores a capped candidate set, not every candidate (search latency, Sam's decision 2026-10-08). nx search over more than one collection asks the engine to rerank, and in local mode the cross-encoder (ms-marco-minilm-l6-v2) spent 3.5 to 4.9 s of a 4.0 to 5.4 s search (M5 Pro, T2 nexus/search-latency-local-mode-2026-10-08) scoring all 135 candidates at up to 512 tokens each. In local mode search_cross_corpus now sends rerank_max_candidates = max(3 * n, 60) and the engine scores that many rows in vector order; the rest stay in the result, unscored and in vector order, behind the scored rows, so the result count and the page shape are unchanged, and a search with fewer candidates than the cap is scored in full. Cloud mode has no default cap: the latency evidence is the local cross-encoder, and the page-quality measurement on file (nexus-abdp2) was taken in the cloud, where the reranker is Voyage. A caller that filters the pool after retrieval (nx search --path, --max-file-chunks) is never capped, because with a cap most of the rows that survive the filter would come back unscored. The cap is per request, so a search the client splits or batches into several requests scores up to the cap in each. New optional request field rerank_max_candidates (integer >= 1, requires rerank) on seven engine handlers (eight routes counting the /query alias of /search): search-per-collection, /search, hybrid-search, search-metadata-scoped, search-topic-scoped, search-graph-hop and search-aspect-scoped; an engine that predates it ignores it and scores everything, so a local install gets the latency gain once REQUIRED_ENGINE_VERSION pins an engine that carries the field. The --lexical leg is not capped. Set search.rerank_max_candidates in .nexus.yml to change the cap in either mode (a positive integer fixes it; 0 scores every candidate). In local mode the page is now chosen from the top max(3n, 60) by vector order rather than from the whole over-fetched pool (up to 1000 rows), which nexus-abdp2 measured as pool-sensitive; the latency gain was measured, the page change was not. Scoring time alone, the local cross-encoder on 2,000-character chunks, warm, a dev Mac, two runs each: 135 chunks 3.2 and 3.6 s, 60 chunks 1.4 and 1.5 s, 30 chunks 0.70 and 0.79 s. That is the scoring step, not an end-to-end nx search, which was not re-measured. The default floor is 60, not 30: over 40 queries at n=10, a cap of 30 kept 6.5 of the uncapped top 10 and lost canonical answers in a hand check, and it dropped prose rows twice as often as code; 60 kept 9.0 of 10 and 90 matched uncapped (T2 nexus/rerank-cap-page-quality-2026-10-08). Sam chose 60.
• nx index repo in local mode writes with 2 to 4 file workers by default, scaled by core count, instead of 2, so more than one embedding batch is in flight (index throughput). A source file over the local 16-chunk cap is written one request at a time by its own worker (the flush pool never sees it), so with 2 workers a run averaged 1.13 requests in flight and had none in flight 34% of the time. Measured on a 900-chunk slice of this repo, local engine, bge-768 on 16 cores: 114 s at 2 workers, 94 s at 3, 92 s at 4 (mean 2.1 in flight), 82 s at 6, 80 s at 8; on the length-grouping engine below the step from 2 to 4 is 69 s to 62 s. The local default is min(4, max(2, cores // 4)) (2 on 4 and 8 cores, 3 on 12, 4 from 16 cores up; 2 when the core count is unknown), because the client and the engine share the machine and the engine admits cores / 2 concurrent embeds: 4 workers plus the 3 flush workers would hold every permit on an 8-core laptop and leave an interactive search to time out. Cloud mode keeps 2: the wider default was measured only against a local engine, and tests/e2e/index-throughput-bench/run.sh has not been run against the cloud. The widest default plus the 3 flush workers stays inside QUOTAS.MAX_CONCURRENT_WRITES (10); NX_INDEX_CONCURRENCY=N still overrides in both modes. The named values are DEFAULT_SERVICE_INDEX_CONCURRENCY (cloud, and the local floor) and LOCAL_INDEX_CONCURRENCY_CEILING in indexer_utils.py.
• The local engine groups the chunks of one embed request by length before the ONNX call (index throughput; engine change, takes effect with the next engine tag and, locally, the client release that pins it). Bge768Embedder ran a request of 16 chunks as one tensor padded to its longest row: on this repo's chunks that is 1.62x the real tokens (attention 2.03x). It now sorts the rows by token length, closes a group when more than 25% of it would be padding (or at the existing padded-area memory ceiling), runs one call per group and returns the vectors in input order: about 1.12x padding, about 4 calls per request. Same slice, local engine: embed route 6.4 to 11.6 chunks/s with one batch in flight and 9.4 to 16.7 with eight; nx index repo 114 s to 62 s (7.9 to 14.4 chunks/s together with the worker change; the box was shared, load average 9 to 32 during the runs). The embedding of a text moves by float rounding only (cosine to the same text embedded alone stays above 1 - 1e-6, pinned by Bge768BatchCompositionTest). Cloud users get it when the engine is deployed; local users get it only after a client release bumps REQUIRED_ENGINE_VERSION, because that constant drives PINNED_SERVICE_TAG and is the only thing that tells a local install which engine to download. Measurements: T2 nexus/index-embedding-throughput-2026-10-08.
• A default search no longer asks the engine for vectors (search latency, nexus-92q1p follow-up). search_cross_corpus sets include_embeddings on POST /v1/vectors/search-per-collection only under cluster_by="semantic"; 7.74.2 also set it for the contradiction check, so every default search carried a vector for each surviving row of the enrichment pool. Measured on the managed service (one sample per query, T2 nexus/search-latency-default-off-2026-10-08): default-corpus limit-10 MCP searches went from 10.2, 2.4 and 6.2 s to 5.2, 2.1 and 2.9 s.
• The contradiction check is off by default (search latency, nexus-92q1p follow-up). Enable it with search.contradiction_check: true. Its flag marks two near-identical chunks from different authors, which is a similarity signal rather than evidence of a contradiction, and a default search should not pay a vector fetch for it.
• The contradiction check runs in the MCP search render, over the page it displays, and fetches a vector only for the rows on that page that can flag, so a default search no longer carries a vector for every pooled row (search latency, nexus-92q1p follow-up). RDR-057 designed the check over the returned results; it had drifted onto the up-to-300-row enrichment pool, and each search-per-collection call then carried a vector for every surviving row (233 rows: 1.73 MB, 1.10 MB gzipped, about 1.5 s of transfer and parse beyond the 1.0 s server handler, in a 7.9 s default-corpus limit-10 MCP search). search_cross_corpus no longer sets _contradiction_flag and never fetches a vector for it; the only reader of the flag is the MCP search text render, which now calls flag_displayed_contradictions on the page after every boost, the file-diversity cap and the offset/limit slice, so a row a boost lifted onto the page is considered and a row off the page is not (nx search, query and structured output never rendered the flag and no longer pay for it). A pair flags only inside one collection, between two non-empty and different source_agent values. A row is fetched only when it carries a source_agent and its collection holds, on the same page, a row with a different one. The check costs one parallel get-embeddings call per collection that holds such a pair on that page, and none when no collection does; the rate of such pages is not measured. A collection with more than 30 such rows on the page is skipped, neither fetched nor checked (the pairwise cap now bounds a collection's share of one page). A fetch that fails leaves that collection's rows unflagged and never fails the search. The route is now asked for vectors only under cluster_by="semantic". Result count and order are unchanged.

Fixed

• A collection a process has just written for the first time is visible to that process at once (in-process collection cache). The engine lists only collections that hold a chunk. Registering a new collection dropped the process's collection cache, but the combined writer read the cache again for its per-request chunk cap before the first chunk landed, which refilled it without the new collection, and nothing dropped it after the write. For up to 60 s, corpus resolution, the search fan-out and get_collection_names in that process did not see the collection (the MCP server, or in-process nx index then nx search). A chunk write now drops the cache when the cache does not name the written collection, so a write to a collection already listed costs nothing. Found when the faster local embed path brought the RDR-196 retrieval bench's 15 index runs inside the 60 s window and the bench read an empty collection list after a successful index.
• nx index repo no longer defers taxonomy assignment for the whole of a first index (local-mode first run). The run decided once, at start, to defer assignment because the engine had restarted less than 600 s earlier, and held that decision to the end: an index started 102 s after nx init started the engine deferred all 22,848 chunks of a 2.9 h run. The deferral is now a deadline: it lifts when the warm-up window and the failure backoff after a recent loss have both passed (when both apply, the later one decides and the message names both), the next flush assigns again, and the end-of-run discovery and drain run once it has lifted. The in-run breaker after a lost batch still lasts the whole run. The messages state what happened: engine restarted 102 s before this run began (warm-up window 600 s) at start, and N chunk(s) deferred during the first 498 s of the run at the close, where the old line read as if the engine had restarted near the end.
• nx daemon service stop --with-pg also stops the MinerU server nexus started (local-mode first run). Indexing auto-starts mineru-api (about 420 MB resident) and the full-stack stop left it running; only nx mineru stop ended it. The stop now finds it from mineru.pid, the file nx mineru stop and nx uninstall already key on, runs the same verb (so the live command is re-checked before anything is signalled) and prints MinerU server stopped (PID n). A mineru-api with no pid file is the user's and is never matched by name; a --config-dir other than the default config dir does not reach the default dir's server. MinerU is stopped after Postgres, on every path including when Postgres is absent or fails to stop, so a slow MinerU drain cannot push nx uninstall's 30 s bound past pg_ctl.
• The aspect-worker daemon is no longer spawned, and refused, on every store or indexing step on a box without claude (local-mode first run). ensure_aspect_worker_daemon forked nx daemon aspect-worker start each time work was enqueued; with no claude on PATH the child refused at once (aspect_worker_daemon.missing_claude_credentials), 119 times in one measured run, leaving a 400 KB crash log. The spawner now checks that precondition first, spawns nothing, returns False, and logs aspect_worker_daemon.spawn_skipped_claude_absent once per process. Enqueueing is unchanged: rows wait in the service queue for a worker on a box that has claude. The daemon's own refusal stays as the backstop, and nx daemon restart-stale reports a drained worker it could not restart for the same reason instead of started pid False, naming the remedy (install Claude Code or put claude on PATH). nx doctor --check-aspect-queue prints a WARN line, not a failure, when rows are pending and claude is not on PATH: aspect extraction is idle: N pending, claude not on PATH; it prints nothing with an empty queue.