v4.2.5_sync

DonSeTch v4.2.5_sync

Changed

• The npm package page carries a real README instead of the stub: the fourth
  tool (web_screenshot), the feature list, the full platform matrix with
  OCR/rerank availability per platform, the install troubleshooting notes,
  the sponsor tiers, and links to the docs. package.json gained a
  funding field, so npm shows a sponsor link on the package page, and
  search keywords that describe what the package actually is.
• The repository README was rewritten: 1090 lines to 700, sponsors moved
  from dead last to the fourth section with a nav link from the top, the
  v3 and v4 "what's new" sections folded into the feature sections they
  describe, the WRB results replaced by a pointer to the WRB repo, and the
  comparison table cut from 18 rows to 12 plus a short Firecrawl head to
  head. Two stale numbers fixed: the tests badge said 1270 while the body
  said 727.

Fixed

• A crawl seeded on a page at the host root (/p0.html, /index.php)
  returned the seed alone, or nothing, with complete: true. The
  auto-scope rule that keeps /tokio on docs.rs inside /tokio/* was
  applied to the file name too, so the scope became /p0.html/* and
  every sibling link was filtered out before it reached the frontier.
  Separately, the quality gate skipped a low-quality page before
  harvesting its outlinks, and a hub page (a link list with almost no
  prose) is the lowest-quality page on a site and the one a crawl is
  seeded from. A root-level file now scopes to the host, and a skipped
  page still feeds the frontier (mnaza, dondai44423#250, fixes dondai44423#249).
• A resolver that could not answer (EAI_AGAIN: resolv.conf unreachable,
  SERVFAIL, a VPN flap) was reported as network.dns with
  errorKind: permanent and a do-not-retry action, the mirror of dondai44423#248:
  the outage read as a dead name. It is transient now, while a name that
  does not exist stays permanent. Search enrichment no longer demotes a
  live result when the resolver times out during the prefetch, and the
  ghost probe's failure class and the search task status label kept their
  signals (mnaza, dondai44423#251).
• Egress lane health learned the same variants: a proxy lane whose own
  name stops resolving is marked dead again, and a resolver timeout is
  counted as a timeout (the message reads "dns timeout", which the old
  contains("timed out") check never matched).

Full Changelog: https://github.com/axwfae/donsetch_noavx/commits/v4.2.5_sync

v4.0.0_sync

DonSeTch v4.0.0_sync

The agent web stack, rebuilt end to end. Same four tools, same
zero-config donsetch mcp default. Faster, stealthier, more
reliable, and honest about what it knows.

Removed

• web_answer (the evidence pack) and web_memory (the local vector
  store), by my call. A fixed search-then-read pipeline and a
  memory-of-everything store strip the agent's control over the loop
  (which link, which budget) and spend tokens on pages the agent
  would not have picked. The agent owns search + fetch; DonSeTch just
  makes each of those calls god-tier. The search prefetch that the
  pack spawned is kept (web_fetch consumes it: a speed win with zero
  new surface).

Fixed

• Persona locale is fail-closed sanitized before it can reach Chrome
  --lang, CDP navigator.languages injection, or an
  Accept-Language header. A corrupt on-disk persona or hostile
  persona.locale / LANG value can no longer break out of the JS
  array or inject header bytes; invalid values become en-US.
• A warm ghost whose viewport/locale no longer matches the incoming
  persona (quarantine re-mint) now relaunches instead of silently
  claiming the old identity. web_screenshot uses the same persona
  wire as tier-1.
• Cross-process host-pace rows are compared by host on read (FNV
  collision no longer shares a floor) and stale .tmp files from
  failed atomic writes are pruned.
• PDF glyph walk no longer panics when the first glyph's font has no
  PDFium-reportable name (Type3, missing BaseFont, over-long name).
  The mono-font check now bounds-checks the family index like the
  sibling dingbat flag; a crafted PDF can no longer abort the process
  (release panic = "abort") on fetch. (dondai44423#221)
• Search syndication dedup keeps punctuation in the title key, so
  C++ Tutorial and C# Tutorial (and dotted versions like
  Rust 1.75) no longer collapse into one result and silently drop
  the other. Only titles that are identical after case/whitespace
  normalization still dedup. (dondai44423#222)
• web_screenshot omitted full_page now defaults to viewport on
  both CLI and MCP (MCP used to silently mean full-page).
• Fetch CLI long-help cites --actions, not the dead
  --browser-actions.

Changed

• donsetch config show redacts bearer tokens and credential-bearing proxy
  values, including proxy pools, while still showing whether each field is
  configured and where its value came from.
• Browser version probes now apply one deadline to process exit and bounded
  stdout capture, terminate inherited-pipe descendants on Unix and Windows,
  and cap captured output at 64 KiB.
• Every runtime env read now flows through the typed config. The old
  env names keep their exact historical trigger semantics and still
  work in 4.0, but they are deprecated; donsetch doctor lists the
  ones your shell still sets and maps each to its config key. The
  hard cut lands in a later minor once the migration is done.

Added

• Egress fabric (v4 A): search, crawl, fetch, and ghost share one
  process-wide proxy pool with durable lane health. Burned and dead
  lines survive restarts in egress-health.json (kill:
  DONSETCH_NO_EGRESS_PERSIST). When a pool is configured, web_fetch
  sticks to one exit per host and rotates on 429 / 407 / connect-dead /
  timeout; the lane stays pinned for the whole redirect chain (never
  mid-200-session). Kill: DONSETCH_NO_FETCH_ROTATE. Personas get an
  exclusive lane bind: a burned or foreign-persona exit is never
  reused for a new mint, and quarantine frees the lane. Per-lane RTT
  EWMA feeds search pacing and surfaces in donsetch doctor --deep
  as one line per lane (ok / slow / burned / dead / auth) with a named
  fix. Daemon preflight still benches dead proxies before the first
  query; crawl skips benched lanes and reports outcomes into the same
  health world.
• Learning engine v2 (v4 B, local only): per-(engine, intent)
  search trust EWMAs drive roster order (search-trust.json v2; empty
  intent maps seed every intent from engine-global history so a
  restart never re-learns a walled engine from zero). Domain profiles
  stamp which egress class their cookies were learned on; a Warm
  clearance vault is refused when the live lane is the other class
  (home-IP cookies never ride a proxy exit). Crawl host ladders
  (429 storms, robots delays) persist to crawl-governor.json
  (cap ~2k hosts, 7d TTL). Enrich/prefetch success density feeds a
  capped domain quality prior (search-quality.json, ±0.08 after
  ≥3 samples; kill DONSETCH_NO_QUALITY_PRIOR; walls and timeouts
  never count). Agent-outcome feedback (must_contain miss / thin /
  SoftNotFound) soft-demotes class|host in outcome-feedback.json
  (±0.05 after ≥2 misses; default off until the 24h soak proves
  it; enable with DONSETCH_OUTCOME_FEEDBACK=1; never extra fetches).
  donsetch status has an improve line
  (warm-hits, walled, cooldowns, flaky, low-trust/quarantined,
  quality hosts, outcome demotes) and donsetch doctor --improve
  explains the loop in ~10 lines with live local receipts and kill
  switches. No MCP tool, no telemetry, no cross-machine sharing.
  A 24h soak battery still gates any public improve claim.
• Search v3: adaptive early-return cancels stragglers once ≥3
  independent index families already agree on a top-3 URL (kill
  DONSETCH_NO_SEARCH_EARLY). Mojeek empty-parse (blocked as HTTP
  200) burns engine trust harder so it leaves the default width
  instead of occupying a slot every query. Compact snippets trim to
  180 chars. Query compiler splits site: / filetype: / intitle:
  out of the free text and only sends each operator to engines that
  honor it (DDG lite strips them so BM25 never ranks the literal
  token); post-merge filters enforce site: and intitle:, and
  filetype: keeps matching URLs plus extensionless download
  handlers (kill DONSETCH_NO_QUERY_COMPILE). site:github.com,
  site:stackoverflow.com, and friends also join the fan-out as
  that site's own API. A byte-derived SERP instant layer lifts
  featured snippets / instant answers / knowledge panels out of the
  organic list into their own slot with the source URL always
  present (never invents text; not cached; kill
  DONSETCH_NO_SERP_INSTANT). Code verticals are query-gated and
  stress-aware: StackExchange only on Q&A/error language, MDN only
  on web-platform tokens, HN only on release/announce; under pool
  stress HN drops first.
• Doctor ultra (v4 D): broader local coverage, same fast default.
  New checks: config posture (NO_CONFIG_FILE + DONSETCH_CONFIG
  conflict, missing explicit file, layer report), search health
  (engine trust, quarantine, quality/outcome receipts, BYOK key
  state counts, C kill-switch flags), clearance stores (routes.json
  counts, handles, cookie vault), crawl stores (governor host
  ladders, page-history size warning past 5MB), DNS resolution
  (independent of HTTP, names AAAA presence), and a captive-portal
  probe under --deep (generate_204 must stay 204). State
  permissions now cover every secret-bearing store
  (ghost-state.json, routes.json, byok-keys.json) and
  auto-tighten to 0600. All new checks carry unit tests; live
  doctor on a real box is green with honest warnings.
• ML-DSA ClientHello parity (v4 E1): Chrome 151 pads
  signature_algorithms with ML-DSA 44/65/87 (IANA 0x0904/05/06).
  boring 5.2.0 finally names and verifies those codes, so the
  ChromeTrue wire advertises them (mldsa44:mldsa65:mldsa87: at
  the front). InterceptionSafe never sends them (a corporate MITM
  re-terminates without ML-DSA certs). Kill:
  tls.mldsa_sigalgs=false / DONSETCH_NO_MLDSA_SIGALGS.
• Persona wire divergence (v4 E2): per-domain personas now
  actually drive the wire. Tier-1 Accept-Language is persona-locale
  first (TLD/script kept as a lower-q preference so localized
  content still lands, without the classic en-US browser speaking
  perfect ru-RU JA4H tell). The ghost browser launches with the
  persona's viewport and --lang, and CDP navigator.languages
  matches. Quarantined personas fall back to the default wire.
• Crawl polish (v4 F): dataset JSONL rows carry
  dataset_version: 1 in structuredContent; resume-token TTL is
  fetch.resume_ttl_secs (default 7200, clamped 300..86400) instead
  of a hardcoded 2h. Cross-process host politeness: crawls share a
  per-host last-stamp file under <cache>/host-pace/ so a daemon and
  a CLI crawl of the same host respect one floor gap (kill:
  DONSETCH_NO_HOST_PACE_FILE; corrupt file = ignore). Canonical/
  binary skip and transient-vs-permanent retry were already on
  master; reconfirmed with tests.
• just win-check: type-checks the crate for x86_64-pc-windows-gnu
  from Linux (clippy, no linkage), both --no-default-features and
  the full feature set, so #[cfg(windows)] breakage from a
  Linux-only change surfaces before the push instead of in Windows
  CI. Needs mingw-w64; see CONTRIBUTING.md.
• One typed runtime config (src/config.rs, issue dondai44423#193): every knob
  lives in a layered struct, defaults < donsetch.toml
  (DONSETCH_CONFIG or <config-dir>/donsetch/donsetch.toml,
  skippable via DONSETCH_NO_CONFIG_FILE=1) < DONSETCH_<SECTION>__<KEY>
  env names. Values validate loudly at load; unknown TOML keys name
  the offending file. donsetch config show prints each knob with
  its value and origin (--markdown for the reference table,
  --legacy for the old-name map); donsetch doctor warns about
  deprecated env names active in the shell.
• Resurrection reaches transport-dead URLs: a TLS handshake against
  a parked domain, a dead DNS name and a refused port now gate
  archive=auto resurrection the same way 404/410 do, while
  timeouts, resets and protocol errors stay excluded (a snapshot
  must never launder an unknown, and a reset can be an IP-level
  block that an archived copy would paper over). The gate reads a
  ne...

v3.6.2_sync

DonSeTch v3.6.2_sync

Added

• Ghost SERP cascade lane: when the plain-HTTP fan-out AND its retry
  wave leave the merge thin (<3 working lanes or <15 hits), one browser
  render through the shared ghost hook fetches Google's 2026 JS-shell
  SERP (plain HTTP gets 0 result anchors; the browser render parses with
  the existing layered parser). Joins the merge as the independent
  google index family (5 keyless families instead of 4). Costs zero
  when healthy (never fires), trust/quarantine shared with the google
  base, honest google_ghost engine reports. Kill switches:
  DONSEEK_NO_GHOST_LANES, DONSEEK_FORCE_GHOST_LANE (dev bench).
• Persisted engine health: trust EWMAs + failure streaks live in
  search-trust.json, so a benched engine stays benched across daemon
  crashes instead of re-paying three failure lanes after every restart.
• Corroboration on the model surface: compact search lines carry the
  independent-index-family count per result (· 3 sources), the same
  math the ranking consensus uses.
• Ranking top-up on page truth: after enrichment swaps in real page
  titles/descriptions for the top slice, the cross-encoder gets a
  bounded ±0.1 nudge on close calls (DONSEEK_NO_TOPUP = A/B switch).
  Bench A/B: head-10 corpus MRR 0.75 → 0.80 with the top-up.

Fixed

• donsetch login post-login probe aborted the CLI process: the
  probe and the CDP endpoint poll built reqwest::blocking clients
  inside the tokio runtime context, which is a documented reqwest
  panic (and with panic = "abort", a hard process abort). Reproduced
  on master: probe_domain in a tokio test kills the process. Fixed
  by running the probe on a dedicated thread and converting the
  endpoint poll to an async client. Both paths regression-tested
  under the runtime. Credit: mnaza (dondai44423#122).
• Linux warm-connect (TFO) path shipped three real bugs in 3.6.x:
  (1) sockaddr_of returned a pointer to a local inside its own match
  arm: dangling-pointer UB, benign only by stack-layout luck (the
  shipped path "worked" by accident of stack reuse);
  (2) from_ne_bytes(...).to_be() reversed the IPv4 octets on every
  little-endian target, so warm TFO connects dialed octet-reversed
  addresses (203.0.113.7 → 7.113.0.203), always failed, and silently
  fell back to a plain connect: warm TCP Fast Open never actually
  worked on Linux in 3.6.x, and the fix makes warm connects genuinely
  faster (fewer round trips on repeat-navigation origins);
  (3) the raw libc::connect on a TFO-requesting socket is a real
  blocking syscall that ran inline on the executor thread: measured
  against loopback by the author at ~135s vs ~220µs for a plain
  connect, silently defeating the outer connect timeout. Fixed with
  spawn_blocking + honest fallback to a plain connect. Structs now
  returned by value, execute-test covers the executor-freedom pattern
  with a stated caveat. Credit: mnaza (dondai44423#123).
• search --json lost titles/snippets for machine consumers (compact
  contracts, v3.6.0): the model surface (structuredContent) stays
  compact, but the client-only com.donsetch/search-debug namespace now
  carries the full per-result machine view (title/url/snippet/score/)
  and the CLI --json re-materializes it into meta for pipelines.
  The in-repo search bench had silently gone 0/30 because of the loss:
  found live, restored, and the bench now measures real recall
  (snippet accuracy 96.7% over the 30-question corpus, MRR 0.80).
• consensus double-counted same-engine ranks in the JSON meta: a
  Yahoo URL at two ranks read as consensus 2 for one opinion. Now counts
  index families (same math as ranking), matching its own field name.
• Enrichment demoted slow-alive pages as dead links: transport
  timeouts no longer halve a result's score; only refused/DNS-dead/
  4xx/5xx answers do (slow ≠ dead).
• Single-flight stampede: two identical searches at different
  max_results each paid a full fan-out; the flight now keys on
  query+intent since the leader publishes the full top-12 anyway.
• Oversized/empty queries now fail fast with a clean message +
  next_action on BOTH paths (BYOK providers included), instead of
  burning a fan-out: donsetch search "<862-char query>" no longer
  reaches exa/engine endpoints.
• Search-health single write per search: trust snapshots now save
  once per completed search (not per engine outcome).

Changed

• Google News snippets now carry Publisher · date instead of a bare
  RFC-822 date string (a date alone says nothing about the story).
• search --json engines list dedups engine names (an engine surfacing
  a URL at two ranks is one opinion, same as the markdown surface).
• Bench harness: bench/search_quality.py results cache note + the
  search invocation verified against the current CLI arg surface.

Full Changelog: v3.2.0_sync...v3.6.2_sync

v3.2.0_sync

DonSeTch v3.2.0_sync

The search-legibility release: signals the merge already computed now reach the text channel, where every client can read them.

Changed

• Snippets are 200 chars, cut on a word boundary. 120 ended mid-word almost every time — sequence transduc, All You Nee, the attention on a live query. The cost was not lost context but a wasted web_fetch to learn what the snippet nearly said. The cut trims back rather than extending, so output stays bounded by the budget; if the character past the window is whitespace the window already ends cleanly and is kept whole; backing off is abandoned when it would cost more than a fifth of the budget, since a long URL or an unbroken CJK run would otherwise strip the snippet to nothing. The ellipsis is appended only when text was actually dropped, so it never promises content that does not exist. Trailing marks that join clauses (, ; : 、 ， 「 《 【) are removed before it; sentence terminators (. ! ? 。！？, shared codepoints across Chinese and Japanese) stay, because a cut landing after one means the snippet ended on a complete sentence. (dondai44423#40, @Mart-Bogdan)
• Each result names the engines behind it, with its blended score: engines: bing, ddg · score: 0.83. Which indexes agreed is what separates two equally plausible results — independent engines converging usually means canonical, a lone vertical hit often means tangential — and it was previously visible only in structuredContent. Names rather than a count: consensus there is sources.len(), which double-counts an engine that returned the URL at two ranks, while ranking counts index families. Deduped names are the honest version, and say which source. (dondai44423#40, @Mart-Bogdan)

Fixed

• Whitespace in titles and snippets is collapsed at merge. HTML-scraped engines normalized already; JSON-sourced hits did not — MDN summaries, BYOK provider snippets (Exa returns raw page text) and GitHub descriptions arrived with embedded newlines, breaking the three-space indent of the markdown list. Normalizing once in rank::merge, the single point every source flows through, also fixes the "longest snippet wins" and "shortest clean title wins" comparisons, which previously ranked on whitespace count: a newline-padded short snippet could beat a genuinely longer one. (dondai44423#40, @Mart-Bogdan)
• Clippy only ever linted Linux, and only lib and bins. The ~32 #[cfg(windows)] sites — all of ghost/proc.rs — were never compiled by the lint pass, and neither were the 50 #[cfg(test)] modules or tests/*.rs, which the default lib+bins pass skips. Clippy now runs on Windows as well, with --all-targets. macOS stays out deliberately: its only exclusive site is one target_os = "macos" block, everything else being unix (shared with Linux) or not(linux_like) (shared with Windows), so Linux+Windows already covers it.
• rust-toolchain.toml pins the local toolchain to 1.98, matching the CI/release pin. v3.0.0 pinned the CI side to end local-vs-CI clippy drift, but nothing pinned the contributor's side, so a local clippy of a different version reports a different lint set — findings that CI does not have, and misses that it does.

Full Changelog: v2.5.0_sync...v3.2.0_sync

v2.5.0_sync

Full Changelog: https://github.com/axwfae/donsetch_noavx/commits/v2.5.0_sync

v2.3.4_sync

Full Changelog: https://github.com/axwfae/donsetch_noavx/commits/v2.3.4_sync