VoidAccess 2.1.0

A security release for multi-user self-hosted installs, plus everything built since 2.0.3: a shared knowledge
tier and owned page index, new intelligence sources, safety parity between the CLI and the web app, an official
progress hook, a cloud-ready source deny-list, and many reliability fixes.

Upgrade: read docs/SECURITY-ADVISORY-2026-10.md, install 2.1.0, rebuild the web UI, and run
alembic upgrade head (migrations 0027 to 0036). Rotate JWT_SECRET if you want to end sessions issued before
the upgrade. Multi-user installs should upgrade promptly.

Privacy: nothing leaves your install. The new "shared" tier is shared between the users of one install, in your
own database, and only ever holds sanitized observations, never who searched for what. The cross-install ingest API
is off unless you set VOIDACCESS_INGEST_ENABLED=true on a corpus service you run yourself.

Security

• Fix self-hosted multi-user data exposure: shared ownership dependencies return
  404 for foreign investigation, entity, graph, analysis, export, actor and monitor resources.
• Add owned entity link context, investigation-scoped pages, private actor notes
  and aliases, database monitors and alerts, and user-filtered Chroma search.
• Add the user/admin role and voidaccess admin promote --email; administrative
  routes now require an administrator. Existing first users bootstrap as admin.
• Make forwarded-IP trust opt-in and add authenticated-user request limits.
• Move web sessions to httpOnly cookies; server proxies forward authentication,
  and logout revokes backend JWTs. Database revocation errors fail closed.
• Add opt-in VOIDACCESS_RSS_SNIPPET_ONLY=1 to avoid fetching article bodies.
• Add migration 0036_multiuser_security, idempotent YAML/vector import, two-user
  API tests, migration tests, route enumeration and browser-session tests.
• See docs/SECURITY-ADVISORY-2026-10.md for upgrade and attribution limitations.

Breaking changes

• Database: run alembic upgrade head (migrations 0027 to 0036). The ownership migration has no destructive
  downgrade; back up first (see the advisory).
• Administrative routes require the admin role.
• voidaccess.llm.generate_summary raises SummaryUnavailable instead of returning an error string.
• CLI JSON sources_used statuses renamed: fail -> error, skipped_no_key -> no_key; new disabled,
  skipped_by_allowlist, timed_out.
• voidaccess investigate --quiet prints nothing except fatal errors.

New: shared tier and owned page index (local to your install)

• Shared corpus tier (shared_corpus/): discovered onions, page snapshots, canonical IOCs and source reputation,
  kept apart from private investigation data, so one investigation's observations speed up the next (cache hits,
  seed expansion, pre-extracted entities) without revealing who searched for what. Migrations 0027, 0028,
  0030 to 0032, 0034, 0035.
• One sanitizer for every shared write (sanitizer/): content-safety drop, credential redaction and personal
  data redaction before anything reaches the shared tier; it fails closed (stores nothing) when it can't guarantee
  redaction. Plus a page-snapshot object store (snapshot_store/) and IOC hygiene helpers (hygiene/).
• Anti-poisoning trust gate (trust_gate/): deterministic quarantine, corroboration, source reputation and
  anomaly checks for the shared tier, with corpus-health views for the maintainer (observability/).
• Cross-install ingest API with Sybil-resistant corroboration (ingest/, api/routes/ingest.py, migration
  0033). Off by default (VOIDACCESS_INGEST_ENABLED=false).
• Owned, semantically ranked page index (owned_index/), filled from investigations and seeds
  (index_population/) and kept fresh by a background loop (freshness/) on a durable DB-backed job queue
  (jobqueue/, migration 0029), with an in-process rate limiter (ratelimit/).
• New sources and search improvements: CVE intelligence, EPSS exploit-probability scores and Exploit-DB; a much
  larger curated onion seed list; search result ranking and an engine cull (search/ENGINE_CULL.md); a reusable
  blocklist-feed matcher (feeds/); presence-only HIBP Pwned Passwords checks via k-anonymity
  (credential_presence/); a shared clearnet connection pool (netpool/).
• Benchmark tooling (benchmark/): engine probes with latency and unique-result share.

Changed defaults (self-hosters will notice these)

• More distinct onion pages per run. Onion links are now picked one per site and
  at most two per mirror family (sites whose addresses share their first seven
  characters), with real search-result links ahead of addresses a results page
  only mentions. The number fetched follows --depth: 15 (shallow), 25 (normal),
  40 (deep), overridable with VOIDACCESS_ONION_TOP_N (5-60). When failed fetches
  or mirrored copies leave slots empty, one extra batch backfills them from the
  next-ranked links; normal and deep also follow up to 10 / 20 links found on
  the fetched pages. Every extra fetch goes through the same URL and content
  safety gates.

• Source status can read partial: some of the source's requests worked and
  some failed. In the enrich.source trace event a partial source reads error
  (the trace vocabulary has no partial).

• Packaging: pip install voidaccess now ships 13 packages it previously
  omitted (shared_corpus, trust_gate, netpool, jobqueue, hygiene,
  sanitizer, ratelimit, observability, snapshot_store,
  credential_presence, feeds, ingest, benchmark) plus the new
  progress package. A wheel install used to crash on import. tests/test_packaging.py
  builds a wheel and checks it from a clean venv.

• CLI page cap: voidaccess investigate now honours MAX_LLM_PAGES_PER_INV
  (default unchanged at 10); it was silently ignored before.

• Summary loss: the CLI summary phase now retries once with a reduced context
  after a timeout, then falls back to a deterministic non-LLM summary. The first
  attempt gets 67% of VOIDACCESS_SUMMARY_TIMEOUT (60 of 90 s by default) so the
  retry fits in the same budget. The report JSON gains summary_kind
  (llm | fallback | none). The CLI also passes preset="threat_intel" by
  keyword; it was being passed in the entities position.

• Search engines: OSS and TorNet are quarantined (excluded from the
  fan-out, retained for re-probe). Live Tor probe, 4 samples each: p90 latency
  48.6 s / 76.1 s and unique-result share 0% / 7.2%
  (benchmark/artifacts/E1B_engine_probe.json). Kaizer, Anima, Tornado and
  Find Tor were already quarantined and probed at 0% success again.

• Tests: tests/test_setup.sh no longer contains a real-looking OpenRouter
  key or admin password.

• generate_summary raises (library API): it now raises
  voidaccess.llm.SummaryUnavailable (with the cause chained) when the LLM fails
  or there is no page content, instead of returning the string
  "Summary unavailable — LLM error: …" or "… No extractable content found.".
  A failed LLM could therefore be stored as, and reported as, a real summary.
  Callers must catch it; the CLI and the API do.

• Summary retry and fallback on every failure, in both entry points: the CLI
  retried only after a timeout; now any failure (SummaryUnavailable, another
  exception, an empty answer) retries once with reduced context and then falls
  back to the deterministic summary. The API now does the same through the shared
  voidaccess.summary_fallback.summarize_with_fallback (it used to store the
  error text). summary_kind is llm only when the model produced text. No
  user-facing text contains a raw exception message any more: the API's stored
  summary no longer ends in summary generation failed: <exception>, the CLI
  display line shows a generic message, and sources_used[...].error is the
  exception class name rather than its message.

• --quiet is now silent: it used to print a VoidAccess — <query> header,
  every step line, and then the full result panel. It now prints nothing at all
  except fatal errors, and those never echo the query (a too-short query used to
  be quoted back). Library warnings, the startup banner (on a terminal) and the
  optional-keys configuration warning are suppressed too. Output files are still
  written; their paths are in the completed event of --events-jsonl (file
  names embed the query, so they are not printed). JSONL and callback sinks are
  unaffected.

• sources_used is honest and richer (CLI JSON): statuses are now ok,
  error, timed_out, skipped_by_allowlist, disabled, no_key (plus
  skipped for "not run in this mode", e.g. --no-tor). Renames: fail ->
  error, skipped_no_key -> no_key. A source switched off by
  VOIDACCESS_ENRICHMENT_SOURCES or by its own off switch
  (PASTE_SCRAPING_ENABLED=false, GITHUB_SCRAPING_ENABLED=false, ...) no longer
  reads ok with zero results; a source whose every HTTP call failed reads
  error; a side source that raised reads error (it used to read ok). Every
  entry gains requests (HTTP calls actually made), and any source that made
  calls but was not reported before now appears. The summary panel's "Sources"
  count counts ok sources only. The API's flat string statuses are unchanged
  apart from the new disabled and skipped_by_allowlist strings.

• SIGTERM cancels voidaccess investigate cleanly: it used to kill the
  process with no event and no cleanup. Now the current item finishes (up to
  VOIDACCESS_CANCEL_GRACE, 3 s), the progress hook emits cancelled with an
  accurate stages_completed, the investigation row is marked cancelled, and the
  process exits 143 without writing any output file. Ctrl-C (exit 130) also marks
  the row cancelled (it used to stay running until the stuck-run sweep failed
  it).

• Output files are written atomically: the JSON and Markdown reports are
  staged as dot-prefixed temp files and renamed into place after the last cancel
  checkpoint, so a cancelled or killed-at-the-wrong-moment run can no longer leave
  a truncated report.

• Cancel checkpoints inside the long stages (API and CLI): a cancel request is
  now honoured between engines in the search fan-out, between pages in fetch and
  scrape, and between sources in enrichment, not only at stage boundaries. With no
  cancel requested nothing changes.

• Tests: tests/test_progress.py::test_rich_display_output_unchanged compares
  the live display (not quiet=True, which is now silent).

New, opt-in (no change unless set)

• Official progress hook (progress/): 8-stage events with numeric counts, Rich /
  JSONL / callback sinks, voidaccess investigate --events-jsonl PATH.
• VOIDACCESS_ENRICHMENT_SOURCES allowlist for enrichment sources.
• VOIDACCESS_DISABLED_SOURCES: a cloud deny-list. The named sources make no
  network call in any stage (Tor engines, paste sites, GitHub, GitLab, RSS,
  Telegram, blocklist feeds, historical intel, Pwned Passwords, enrichment) and
  report disabled. Unknown ids are a hard error. Recommended cloud value:
  openphish,sslbl,phishtank,ransomwhere,hibp_pwned_passwords,fbi_doj_press,pastebin,paste_ee,dpaste,rentry,gitlab,telegram
  (sources.source_control.CLOUD_DISABLED_SOURCES). Source ids are listed in
  docs/architecture.md section 13.18.
• VOIDACCESS_CANCEL_GRACE and VOIDACCESS_CANCEL_POLL_INTERVAL tune the cancel
  helper (utils/cancel.py); _run_investigation(..., cancel_token=...) lets an
  embedding runner cancel cooperatively.
• Per-source HTTP request counts: sources_used[...].requests, and
  source_requests, source_requests_by_step, sources_status and the output
  paths in the completed progress event's metrics.
• VOIDACCESS_SEARCH_WORKERS, VOIDACCESS_SCRAPE_WORKERS,
  VOIDACCESS_EXTRACT_CONCURRENCY, VOIDACCESS_LLM_CHUNK_CONCURRENCY, VOIDACCESS_LLM_CHUNK_TIMEOUT, VOIDACCESS_SEARCH_DEADLINE,
  VOIDACCESS_CONCURRENT_COLLECT; VOIDACCESS_ENRICHMENT_TIMEOUT is now applied
  on the CLI path when set. See docs/architecture.md section 13.17.
• benchmark probe artifacts report latency_p90_ms and unique_result_share
  per engine.
• Trace events (progress schema 2): voidaccess investigate --trace --events-jsonl PATH
  or VOIDACCESS_TRACE_EVENTS=1 adds a trace event for each refined query,
  search engine call, URL found/kept/dropped (with the reason), page fetch outcome,
  page entity counts, entity found (masked), enrichment source, graph progress and
  summary status, so a dashboard can animate a live run. Hosts are shown only in a
  shortened or defanged form, with names that spell out abuse content masked; no
  page text, paths, credentials or personal data,
  and anything the content-safety filter dropped is reported as an id and a reason
  only. Volume is capped per kind (VOIDACCESS_TRACE_CAPS, at most 3,000 per run),
  and traces are dropped first when a sink falls behind. JSON Schema:
  progress/schema/progress-v2.schema.json; docs/architecture.md section 13.19.
• Every progress event now carries schema: 2 (an added field; nothing else in the
  schema-1 events changed).

Safety

• Align CLI and API content-safety gates: Tor and side-source page text is
  scanned via shared gate_pages() before persistence, vector cache, extraction,
  or summarization; URL and entity gates fail closed on errors.
• Add VOIDACCESS_SAFETY_BYPASS_PROMPT (on default, off for cloud runners)
  to omit the LLM authorization preamble from refine, filter, and summary prompts
  without disabling mandatory content filters.

Fixed

• A failed fetch is no longer counted or stored as a page. The scraper used to
  return the search result's title as the text of a page it could not fetch (or
  that had no text), so those titles were stored, extracted and counted in
  pages_fetched. Now only a fetch that returned content is a page; the failure
  shows only as its page.fetched outcome.
• Source status is honest for fixed endpoints. Any non-2xx answer from a feed
  or a fixed API (ransomware.live, CISA advisories, justice.gov, the MITRE
  bundle, the blocklists) is a failure; a per-item lookup's 404 (Shodan
  InternetDB, HIBP, VirusTotal, NVD and similar) still means "no record". Calls
  made as await session.get(...) are counted too, and a feed that failed is not
  requested again in the same run. Sources that silently read ok while every
  call 404'd now read error.
• Dead endpoints replaced: ransomware.live groups and victim posts now come
  from its data feed (data.ransomware.live/groups.json and victims.json; the
  API's /groups, /group/{name} and /v2 victim endpoints are gone), CISA advisories from cybersecurity-advisories/all.xml, and
  justice.gov press releases from /news/rss. The last two are RSS and are now
  parsed as such (the justice.gov feed had never parsed).
• MITRE ATT&CK techniques are attributed only to a group the run found, by
  exact name or alias. Matching was by substring, so "Tor", "royal" or "bert"
  pulled in unrelated groups' techniques. Now a name must equal an intrusion
  set's name or alias (after a built-in alias table that now holds only names
  ATT&CK itself lists for the group; brand mappings such as REvil, GandCrab or
  Conti to Wizard Spider and DarkSide to FIN7, and the tool "Cobalt Strike",
  were removed); an alias that is also an ATT&CK software name is
  ambiguous and matches nothing; revoked sets are ignored. The overlay counts
  THREAT_ACTOR_HANDLE as an actor, runs only when the run has no
  MITRE_TECHNIQUE yet (it used to check a misspelt CVE type), adds one page per
  group, and tags every technique with its group.
• MITRE output now needs a corroborated actor. An actor drives the ATT&CK
  overlay and historical intel only if a pattern or the gazetteer extractor found
  it, it was seen on at least two different hosts, or the query names it. One
  entity-recognition hit on one page (a common word that is also an ATT&CK group
  name) used to add that group's techniques to most reports. The actor entity
  itself is still reported.
• ransomware.live and RansomLook group names match whole words only. A group
  name or listed alias must appear as a whole word in the query (at least three
  characters, case-insensitive); versioned names such as lockbit3 also match
  their stem ("LockBit"). Substring matching let a one-letter group name match
  unrelated queries and add its pages. ransomware.live is off in the hosted
  profile, so this affects self-hosted runs.
• Organisation names drop link-directory and markup noise. Values such as
  "Name - description" titles, markdown residue, slash lists, .onion names,
  directory words, base64 runs and dark-market listing words are removed before
  they are stored or sent to the NVD CPE lookup; truncated parentheses, link
  tails and leak-listing status words are trimmed first. On 379 labelled values
  from the regression runs: 97.8% precision, 75.9% recall.
• Enrichment no longer throws away finished work at its 59 s deadline. The
  CISA KEV / NVD / Shodan / VirusTotal / historical-intel (MITRE) / CVE-fusion
  group ran its steps one after another behind NVD, so it usually missed the
  deadline and all of its results were discarded (MITRE techniques, the CVE
  view and many relationships were lost). Now historical intel starts as soon as
  CISA, Shodan and VirusTotal are done, the CVE fusion as soon as NVD and KEV
  are (bounded at 8 s, falling back to the plain NVD/KEV pages), NVD stops
  starting new lookups after 35 s and keeps what it fetched, and the group
  returns its finished parts 2 s before the phase deadline. A slow source loses
  only its own results; the phase still ends by 59 s. The constants are in
  sources/enrichment.py (ENRICHMENT_DEADLINE_S, NVD_BUDGET_S, ...).
• A source a deadline cut off now reads timed_out, one that failed error,
  in sources_used and in the enrich.source trace events, including the
  sources inside that group (cisa_kev, nvd, cve_intel, epss,
  exploitdb, shodan_internetdb, virustotal, mitre_attack) and the
  other enrichment feeds. Before, a dropped source read ok with 0 results.