v0.32.1

Auto-released from merge of PR #192.

Fixed — a source disclaimer no longer counts against the citation score (#186)

When the synthesizer was transparent about which sources it discarded — "note that sources [3], [5], [7], [8], [9], [11], and [12] contain no information about ternary LLM accuracy and are not cited below" — the verifier pulled seven citation ids out of that aside, recall-checked each against the disclaimer sentence, and counted all seven as unsupported. Honesty about rejected sources became the single largest penalty an answer could take: one nightly canary run scored supportRatio 0.52 with 7 of its 10 unsupported citations coming from that one sentence.

A sentence is now excluded from citation accounting only when it both reads as a source-disclaimer (it names sources and asserts they carry nothing useful) and has no supported citation of its own. Both conditions are required, and that pairing is the safety property: one supported cite means the sentence is making a claim, so it is never skipped — the rule cannot bury a partially-hallucinated citation. The disclaimer test is deliberately an exclusion claim rather than any negation, so a genuine negative finding like "the trial showed no improvement [4]" is untouched. Excluded sentences are reported in verification.disclaimers and printed by the CLI rather than silently dropped, because an exclusion nobody can see is indistinguishable from a miscount.

Built + tested by auto-release.yml; npm publish runs tokenless in publish.yml via OIDC trusted publishing (provenance-attested).

v0.32.0

Auto-released from merge of PR #183.

Changed — XDG Base Directory layout for fresh installs (#179)

deepdive kept everything under a single ~/.deepdive/. Fresh installs now follow the XDG Base Directory spec: config at $XDG_CONFIG_HOME/deepdive/config.json (~/.config/…), the page cache at $XDG_CACHE_HOME/deepdive (~/.cache/…), and sessions at $XDG_STATE_HOME/deepdive/sessions (~/.local/state/…). Existing setups don't move: whenever a legacy ~/.deepdive/ directory exists it keeps winning for all three paths, and the DEEPDIVE_CONFIG / DEEPDIVE_CACHE_DIR / DEEPDIVE_SESSIONS_DIR overrides still beat everything. Resolution lives in one place (src/xdg.ts, injectable legacy check) and is pinned by test/xdg.test.mjs — including the spec rule that relative XDG_* values are ignored.

Built + tested by auto-release.yml; npm publish runs tokenless in publish.yml via OIDC trusted publishing (provenance-attested).

v0.31.1

Auto-released from merge of PR #170.

Added — continuous fuzzing of the trust-boundary parsers (ClusterFuzzLite)

deepdive runs over three streams of input it doesn't control: robots.txt bytes an arbitrary web server hands back, the raw text of every page it reads, and whatever the LLM returns. Each now has a Jazzer.js fuzz target pinning its fail-safe contract — parseRobotsTxt/isPathAllowed never throw or hang on an attacker-controlled robots.txt (the hand-rolled non-backtracking wildcard matcher's linearity claim is now machine-checked), extractContent always returns a well-formed result on hostile page text, and the LLM-output boundary (markdownToHtml/escapeHtml for --html export, parsePlan/parseCritique for the agent loop) never emits unescaped HTML and never fails with anything but a plain Error. ClusterFuzzLite runs the targets weekly in CI (cflite.yml); npm run fuzz is the fast local repro loop. This also closes the OpenSSF Scorecard Fuzzing check.

Changed — workflow tokens drop to read-only at the top level

auto-release.yml and codeql.yml granted their write scopes (contents, actions, security-events, …) at the workflow level, so every job in those workflows inherited them. The scopes now live on the single job that uses them and the top level is contents: read — closing the three open Scorecard Token-Permissions findings. No behavior change: same steps, same scopes, narrower blast radius.

Built + tested by auto-release.yml; npm publish runs tokenless in publish.yml via OIDC trusted publishing (provenance-attested).

v0.31.0

Auto-released from merge of PR #162.

Added — MCP server mode: deepdive mcp (#160)

deepdive now meets agents where they live. deepdive mcp runs a Model Context Protocol server on stdio exposing one tool, deepdive_research, so any MCP client (Claude Code, MCP-aware IDEs, agent harnesses) can hand its model a research tool that shows its work: every answer returns as markdown followed by the trust signals the CLI reports — source-trust mix, citations supported, confidence grade, and LLM cost. Nothing else in the MCP tool ecosystem reports the credibility of its own sources.

Design points, in the fleet's proven MCP shape: configuration flows through the same resolveConfig path as CLI flags (the client's env block supplies defaults; per-call arguments override), so the two surfaces cannot drift. Calls are stateless (sessions off) and each run carries a wall-clock cap (default 10m) enforced by AbortSignal.timeout — a server must never process.exit the way the CLI's runtime backstop does. stdout is protocol-only; sparse progress goes to stderr. A failed run returns honest tool output rather than a protocol error — NoSourcesError carries the #147 diagnostics (what the domain filter dropped, which fallback was skipped and why) so the calling model can widen the filter or rephrase instead of guessing.

New runtime dependencies (justified): @modelcontextprotocol/sdk — an MCP server cannot exist without the protocol implementation — and zod, the SDK's required schema layer. Both are regular dependencies so CI's --omit=optional installs keep the MCP tests importable.

Pinned by test/mcp.test.mjs: protocol-level tests over the SDK's in-memory transport pair with an injected research runner (no network, LLM, or browser), the tool-argument → CLI-flag mapping, the stats footer, schema-layer rejection of missing arguments, error-path rendering — plus a real-stdio smoke test spawning deepdive mcp to prove stdout stays a clean protocol channel end-to-end.

Built + tested + npm-published inline by auto-release.yml on this run. See the workflow log for the provenance-attested publish output.

v0.30.0

Auto-released from merge of PR #159.

Changed — the allow-domain hint becomes a directive, not a suggestion (#157)

v0.29.0's receipt protocol (the NIST question ×3 with --allow-domain=nvlpubs.nist.gov on the production runner) came back 1/3 — unchanged from baseline — and the diagnostics said exactly why: the hinted retry fired on every failing run ("10 queries"), but a bare host token appended to the query cannot steer an aggregator's ranking, and two of the fan-out's three backends can't respond to a host hint at all. Honest exit-3s are good; recovered runs are better. Three changes, all on the hinted-retry path:

• Engine-syntax backends get searchHinted (new optional SearchAdapter capability): searxng, duckduckgo, brave, and auto express the hint as site:<host> — a directive the engine enforces, not a relevance nudge it may ignore. Multiple allow hosts use the OR form. The adapter receives the RAW query plus a structured DomainHint; nothing is token-baked upstream.
• multi: dispatches the hint per sub-adapter: subs with searchHinted run their own form; fixed-domain subs are skipped (a literal-match API can't act on a hint, and the plain pass already asked it — the receipted waste was stackexchange's keyword ladder re-walking an unsatisfiable hinted string); open-web subs without query syntax get the improved token form. All subs skipped → the hinted pass resolves empty instead of failing. The stackexchange adapter now declares its one-site servesDomains (flagship sites map to their own domains — the #130 lesson, adapter-side), which also tightens the #147 fallback gate for it. A structurally deaf primary is not hinted at all.
• The token fallback gains the host's leading label (nvlpubs.nist.gov → also nvlpubs): the 7/2 probe that motivated #147 ranked the target #1 with the bare label where the v0.29.0 full-host token failed — engines tokenize hostnames URL-ish, but the label matches page text and URL fragments.

Pinned by test/domain-hint.test.mjs (formatters, per-leaf site: delegation, auto's primary→fallback hint path, SE serving-set map), new multi: dispatch cases (structured/skip/token routing; all-deaf → []), and agent cases (structured hint used and event display, deaf primary never hinted, token+label form). Receipt protocol unchanged: same question ×3 on the runner, target 3/3.

Built + tested + npm-published inline by auto-release.yml on this run. See the workflow log for the provenance-attested publish output.

v0.29.0

Auto-released from merge of PR #156.

Fixed — --allow-domain is no longer a coin flip (#147)

Live receipt on v0.27.1: the same question with --allow-domain=nvlpubs.nist.gov run three times gave one full answer and two exit 3s — while a raw search probe carrying the host name found the target PDF at rank 1 throughout. Two compounding causes, both fixed:

• Retry-with-hint on the empty-post-filter path. The planner (correctly) rewrites the question into semantic sub-queries and drops incidental hints like a host name — so whether any query happened to surface the allowed domain was planner-variance roulette, and the filter then dropped everything else. Now, when a round yields zero candidates under an allow list, the agent re-runs the round's queries once with the allowed host(s) appended as a relevance hint (most engines treat a bare host token as a strong signal; the filter stays the enforcement layer). Same shape as the #86/#131 keyword ladder: one extra pass, only on the would-have-failed path, and suppressed when the primary is rate-limiting. Surfaced as a search.hinted event, visible without --verbose.
• The structurally useless fallback pass is skipped, and says so. With zero candidates post-filter, the recovery pass ran the fallback adapters (default wikipedia) — whose results were then all dropped by the same domain filter, burning keyless calls on a guaranteed-empty outcome. Fixed-domain adapters (wikipedia, arxiv, pubmed, github — and multi: when every sub-adapter declares) now expose servesDomains; when that serving set can't overlap the allow list in either hostname-suffix direction, the fallback is skipped with a search.fallback-skipped event and the no-sources message names it.
• The exit-3 message stops lying. "returned 0 usable results" was wrong when search returned plenty and the filter ate them all — NoSourcesError now carries droppedByDomainFilter and fallbackSkipped, and the CLI renders "found N result(s), but the domain filter dropped every one" with filter-specific advice instead of "rephrase the question".

Pinned by test/agent-allow-domain.test.mjs (fake-adapter reproductions of all three behaviors, including the deny-only and open-web-fallback non-triggers) plus canServeAllowedDomain cases in test/domain-filter.test.mjs and servesDomains union cases in test/multi-search.test.mjs.

Changed — authority ranking learns to see topical relevance in uniform-tier pools (#148)

Two live receipts from the same night, one gap: rankByAuthority orders by WHO published, and once a candidate pool is uniformly trustworthy it stops discriminating exactly where it is the only ranking axis left — the slot-limited keep stage then fills on search-interleave order. An --search=arxiv run kept a December 2010 paper (all 10 candidates primary, nothing to break the tie) in a ternary-LLM answer set; a multi: ops run gave wikipedia 4 of 7 slots (all reputable, same story). Three additive pieces, measured per the #97 lesson:

• Relevance tiebreak inside equal authority tiers. rankByAuthority accepts an optional {terms, textOf} tiebreak: candidates tying on authority are ordered by distinct-term overlap between their title+snippet and the question/round content tokens — the same deterministic index-walk token machinery as #86/#131/#145. Authority stays the primary key (relevance never promotes a lower tier), equal-overlap ties keep search order (stable sort), and off remains a strict no-op — pools whose tiers already discriminate behave exactly as before. Threaded at BOTH ranking sites: the keep stage (agent.ts, reusing the #145 relevanceTerms) and the multi: fan-out merge (per-query keywords).
• Per-registrable-domain slot cap at the keep stage. At most 2 kept per registrable domain per round (en./de.wikipedia.org count as one host) — UNLESS honoring the cap would leave slots unfilled, in which case the best capped-out candidates take them. Encyclopedic context is worth a slot or two, rarely four. Applied only when authority ranking is on, so off stays the untouched measurement baseline.
• The bench grows a canonical column (the #148 measurement instrument). Authority boards measure trust composition, not topical fit — an all-primary kept set can still miss the one source that answers. Questions now declare canonicalSource (a URL regex for the source an expert would demand: nginx.org for the nginx question, an RFC for HTTP/3); both scoreboards report ✓/✗ per run. Reported, never gated, same stance as authority.

Pinned by new cases in test/source-authority.test.mjs (tiebreak inside/across tiers, stability, off-mode no-op, registrable-domain collapsing, cap-with-headroom), test/multi-search.test.mjs (fan-out tiebreak), and test/bench-authority.test.mjs (canonical column). Receipted on the branch build (bench/results/2026-07-11-pr155-*.md, off→prefer with the canonical column): academic 8P 3U → 11P 0U, niche-ops 2P 2R 4U → 6P 5R 1U, canonical ✓ on all four runs, aggregate primary/reputable share 73%→100% and 50%→92%.

Built + tested + npm-published inline by auto-release.yml on this run. See the workflow log for the provenance-attested publish output.

v0.28.0

Auto-released from merge of PR #149.

Changed — the per-source word cap now spends its budget on query-relevant spans, not the document head (#145)

The first real PDF ingest on a production runner (v0.27.1) fetched the 129-page NIST SP 800-63B-4, extracted all 18,265 words flawlessly — and then head-first truncation (words.slice(0, maxWords)) reduced it to title page, authors, abstract, and table of contents. The synthesizer, correctly refusing to cite what it could not see, answered from front matter and said so. Head-first is the worst heuristic for exactly the documents the source-authority axis works hardest to keep: long formal publications put their least informative words first.

New src/relevance-window.ts (deterministic, LLM-free, zero new dependencies): when a source exceeds --max-words-per-source, the budget now goes to (1) a head anchor for document identity, (2) the fixed-size word windows scoring highest on content-token overlap with the question and the round's queries — the same token machinery as the keyword ladder (#86), reusing trimPunctuation's index-walk normalization (the CodeQL-safe alternative to trim regexes) — and (3) document-order fill for any remaining budget, reassembled chronologically with … elision markers. Applied to both cap sites: the PDF path in agent.ts and extractContent (long HTML specs had the same failure mode). With no query terms or no matches, behavior degrades to exactly the old head-first cap. Pinned by test/relevance-window.test.mjs, including a fixture in the NIST failure shape — 2,600 words of front matter with the normative "SHALL NOT" text buried past the cap — asserting the old behavior misses it and the new behavior reaches it.

Built + tested + npm-published inline by auto-release.yml on this run. See the workflow log for the provenance-attested publish output.

v0.27.1

Auto-released from merge of PR #144.

Fixed — OpenSSH/OpenBSD project hosts were invisible to source authority (#142)

A live v0.27.0 run asked what changed in the latest stable OpenSSH release. The answer was fully cited — 18/18 citations supported, 16 of them grounded in the official OpenSSH release notes — yet the trust badge read mixed, because openssh.org scored unknown (0.4) while the set's one recognized "primary" was a marginally relevant Windows install guide on learn.microsoft.com. Beyond the badge, prefer's keep-stage ranking (#113) would rank the actual release notes below any recognized domain under slot competition. Same coverage class as #130 (canonical project sites with no docs. prefix), fixed the same way — three receipted entries in the curated primary list, no heuristics: openssh.com, openssh.org (the mirror search actually surfaces), openbsd.org (the parent project). Re-scoring the live run's five kept sources moves it from 1 primary / 4 unknown to 3 primary / 2 unknown. Pinned by a new case in test/source-authority.test.mjs.

Built + tested + npm-published inline by auto-release.yml on this run. See the workflow log for the provenance-attested publish output.

v0.27.0

Auto-released from merge of PR #134.

Added — the ops/infra canon becomes visible to source authority (#130)

The v0.26.1 before/after board (bench/results/2026-07-02-v0.26.1-source-authority.md) had one dead row: the niche-ops question kept 0 primary sources with authority ranking off AND on — ranking can only promote what scoring can see, and the scorer was blind to the very sources that answer ops questions. Two auditable coverage holes, both fixed by extending the curated lists (no heuristics, no denylist changes, per the module's boost-led philosophy):

• Official project-documentation hosts that carry no docs. prefix now score primary: nginx.org (the canonical source for the bench question's literal fix, proxy_buffer_size), apache.org, kernel.org, php.net, git-scm.com — the same class as the redis.io/postgresql.org entries already present.
• The Stack Exchange network's flagship sites live on their own domains — serverfault.com, superuser.com, askubuntu.com are not subdomains of stackexchange.com, so listing stackexchange.com never covered them and the canonical ops Q&A scored unknown. They now score reputable, the same class as the already-listed stackoverflow.com.

Fixed — StackExchange search returned zero results for natural-language queries (#131)

/2.3/search/advanced?q= is literal-match against the question corpus, so the planner's long natural-language queries routinely returned zero items — live-verified with the niche-ops bench question: 0 results on both stackoverflow and serverfault, even though the network is full of answers to that exact error. This is the same failure v0.23.0 fixed for wikipedia (#86), now closed for StackExchange with the same machinery: when the verbatim query finds nothing, the adapter walks the pure keywordLadder (4 → 2 → 1 leading content tokens) until a variant hits — at most 3 extra keyless-API calls, only on the would-have-been-empty path. Live before → after on the bench question: 0 → 12 results on both sites (first hit at nginx return 502 upstream). Pinned by three new adapter tests, including the real bench question.

---

Built + tested + npm-published inline by auto-release.yml on this run. See the workflow log for the provenance-attested publish output. (Body backfilled from the CHANGELOG — the extraction step shipped broken and left every auto-release body empty; fixed in #136.)

v0.26.1

Fixed — user-publishable docs. hosts no longer score as authoritative (#111)

• docs.google.com was scored primary (top authority, 1.0) by the docs. documentation-subdomain boost — but docs.google.com is the Google Docs app, where anyone can publish a document (/document/d/.../pub), not Google's product documentation. So an arbitrary user-published Google Doc ranked as authoritative as arxiv.org or a .gov source: exactly the fabricable-source-scoring-as-trustworthy failure mode the source-authority axis exists to catch (#111), leaking in through the prefix rule's own blind spot. The docs. / developer. prefix boost now skips a small, auditable DOCS_PREFIX_EXCLUSIONS set; excluded hosts fall through to neutral unknown (not punished, just not boosted). Google's real product docs live at developers.google.com / cloud.google.com and are unaffected (the latter is already a curated primary domain). Pinned by a new case in test/source-authority.test.mjs.

Added — the source-trust badge reaches the HTML report (#111)

• The HTML export now carries the source-trust signal. v0.26.0 surfaced source trust — the axis orthogonal to citation support: whether the sources themselves are credible, not just whether the claims are cited — in the CLI footer and --json, but not in the HTML export, so a fully-cited report built on content farms could be handed off looking clean. renderTrustBadge() adds the trust label to the report's meta line, reusing the same deterministic, LLM-free summarizeSourceTrust every other surface uses — one consistent signal everywhere. Only rendered when there is something to flag (mixed/low); a high-trust report stays clean, mirroring the CLI footer. The tooltip is HTML-escaped, with matching .trust-low/.trust-mixed styles. Pinned by new cases in test/html-export.test.mjs.

Added — source authority reaches the search stage (#111 P4)

v0.26.0 added domain-authority ranking at the keep stage — but the keep stage can only reorder candidates search already returned. On a farm-heavy topic a multi: fan-out could fill its result cap with content farms a backend ranked first and truncate a primary source that ranked low, so the keep stage never saw it. The --source-authority setting now also biases the fan-out merge: the deduped pool from --search=multi:… is reordered by authority (the same rankByAuthority primitive) before the result cap, so low-ranked primary sources survive into the pool. strict drops known farms there too, with the same min-keep floor; off and single-backend searches are unchanged (plain round-robin / the backend's own ranking). Completes the issue's P4 stretch goal — search-side bias toward primary sources — closing the gap the keep stage alone left open.

Fixed — synthesis no longer fails the whole run on an intermittent upstream stall (#104)

• The streaming synthesis call now retries an intermittent upstream stall instead of fast-failing the run. Since v0.25.2 synthesis always streams, bounded by an idle-token deadline — but callLLMStream wrapped only the initial connect in retry, so once the stream was flowing a mid-generation stall tripped the idle deadline and the TimeoutError killed the whole run. On a single-pass factual lookup, synthesis is the only synthesis call, so an upstream that stalls ~20% of the time failed ~20% of those queries (#104). callLLMStream now wraps the whole connect+stream in retry, gated on whether any token has reached the user: in buffered mode (--json / non-TTY — the path #104 measured) and before the first visible token in interactive mode, a stall is retried transparently with backoff; once visible tokens have streamed to a TTY the stall surfaces instead (a retry would duplicate already-printed output). The accumulated text is reset per attempt, so a recovered retry never concatenates a previous attempt's partial output.
• The connect timeout no longer aborts a healthy long generation. The connect deadline is now a clearable timer that bounds only time-to-headers and is cleared the instant the response headers land; past that, the body stream is governed solely by the idle-token deadline (and the user's abort signal). Previously the connect timeout signal stayed attached to the body, so a healthy generation longer than --llm-timeout-ms (120s default) could be aborted mid-stream — contradicting the client's stated "bounds only the connect" contract. Pinned by new test/llm-stream-retry.test.mjs (buffered-stall retry + fresh-buffer, pre-first-token retry, post-first-token no-retry, and a long-but-healthy stream that outlives the connect timeout).

---

Built + tested + npm-published inline by auto-release.yml on merge of PR #126. See the workflow log for the provenance-attested publish output. (Release notes restored by PR #128 — the release PR bumped the version without cutting the changelog section.)