v0.3.2

Added

• docs: new Mintlify docs site in docs/ — overview, quickstart,
  installation, browsers/challenges concepts, structured-extraction /
  scrape-at-scale / auth-and-login guides, full 51-tool reference, CLI + env
  reference, deployment, troubleshooting, contributing. Built with the Mintlify
  CLI (docs/docs.json); existing docs/*.md files left in place, untouched.
• docs: changelog-map page (docs/changelog.mdx) — version-anchored
  history, linked in the Project nav.

Changed

• docs: quickstart.mdx and README now show the full MCP client setup
  (stdio + HTTP transports, VS Code / Claude Desktop / Cursor configs,
  webrain_guide verification).
• skills: skills/webrain/SKILL.md upgraded to a full agent-skill contract -
  richer frontmatter, When to use, Recommended limits & token discipline,
  and a 7-step end-to-end How to invoke flow.
• docs: new Prerequisites section in README.md and the docs site.
• docs: ADR-001 graph note refreshed to 2026-08-05 state — 875 nodes /
  2287 edges, 3 entry points, new hotspots (vault.get fan-in 26,
  ensure_page_attached, send_cmd_with), 51 MCP tools (up from 34).
• style: cargo fmt on the mcp session-routing match indent.

v0.3.1

Added

• cli: webrain install --engine lightpanda now downloads the lightpanda
  binary from the lightpanda-io/browser GitHub release (raw asset, no
  archive) into the engine cache, mirroring --engine obscura.
  find_lightpanda() also discovers the cached build, so webrain lightpanda
  just works after install. Lightpanda publishes no Windows binary — on Windows
  the install bails with the Docker fallback (lightpanda/browser:nightly)
  instead of silently installing Chrome.

Changed

• cli: webrain install --engine <unknown> now errors with a clear message
  (try chrome, obscura, or lightpanda) instead of silently installing Chrome.

Fixed

• mcp: webrain_open_session(cdp_url=…) now actually routes tools to that
  session — every browser tool accepts an optional session_id argument.
  Previously routing only worked via the Mcp-Session-Id HTTP header, so
  open_session(cdp_url=obscura) never switched navigate/batch/setcookies and
  everything kept hitting the default Chrome backend.
• mcp: a browser kill/restart no longer wedges the cached backend forever —
  dead-socket errors (os error 10054 / connection reset / stream closed) now
  drop the backend so the next call reconnects fresh.
• core: the CDP WebSocket connect retries once with a longer budget (20s)
  after the initial 5s fail-fast, tolerating obscura's slow cold-start
  handshake.

v0.3.0

Added

• tools webrain_page_info —
  just-in-time page context (viewport/page size, scroll position,
  pixels/pages above & below, position %) so the LLM knows when to scroll.
• tools --state/--restore:
  webrain_save_state / webrain_restore_state — export/import a profile's
  auth state (cookies + localStorage) to state.json so logins follow you
  across machines.
• cli: webrain -v / --version / version prints the version.

Fixed

• core: webrain install failed with "the response body is larger than
  request limit" — ureq's read_to_vec() caps bodies at 10 MB, too small for
  the Chrome/Obscura engine zips. Now reads via the unlimited
  into_with_config() reader.

Changed

• docs: rewrote README — removed the table of contents and made it
  install-first with a "Why webrain?" comparison and a use-case section;
  added the Scoop extras install option.
• build: moved the Dockerfile into docker/ and added
  docker/docker-compose.yml (webrain MCP server + persistent
  vault/profile/cache volumes).
• tools: deduped the repeated JS→JSON parse chain in tools.rs into
  parse_json_str / arr_len helpers — no behavior change.

v0.2.0

Added

• cli: webrain upgrade — updates to the latest release. Delegates to
  Homebrew/Scoop when installed through one (brew upgrade webrain /
  scoop update webrain), otherwise self-updates the running binary in place
  from the latest GitHub release.
• install: one-line installers — scripts/install.sh (Linux/macOS) and
  scripts/install.ps1 (Windows) fetch the latest release binary per OS and
  put webrain on PATH:
  curl -fsSL https://raw.githubusercontent.com/prokopis3/webrain/main/scripts/install.sh | bash.
• dist: submitted to the official Scoop extras bucket (PR
  ScoopInstaller/Extras#18455) and published a Homebrew tap
  (prokopis3/homebrew-webrain; brew tap prokopis3/webrain && brew install webrain).
• docs: CONTRIBUTING.md (conventional commits + changelog-enforced PR
  policy), README badges (release/license/platforms/last-commit/stars),
  Quick Start + Traditional Selectors + Commands + Updating sections.

v0.1.1

Added

• docs: real all-OS install one-liners (PowerShell + curl against release
  binaries) and the working scoop bucket (prokopis3/scoop-webrain).

Fixed

• ci: the Linux release binary now builds on ubuntu-22.04 (glibc 2.35) —
  the previous ubuntu-latest build required glibc 2.39, so webrain-linux
  failed with "GLIBC_2.39 not found" on Ubuntu 22.04 / Debian 12 and older.

v0.1.0

[0.1.0] - 2026-08-04

Added

• workspace: Cargo workspace (resolver 3, edition 2024, Rust 1.85) with
  webrain-core / webrain-mcp / webrain-cli; MIT license; docs/
  ARCHITECTURE.md; Keep-a-Changelog CHANGELOG.md; CI + release +
  changelog-enforce workflows.

• core: Rust CDP browser-automation agent. webrain-core defines one
  BrowserBackend trait over CDP WebSocket (CdpBackend) that drives Chrome,
  Edge, Lightpanda, or Obscura with the same code; SessionPool per-session
  isolation; STEALTH_JS anti-bot injection; default-execution-context tracking.

• mcp: webrain-mcp stdio JSON-RPC server owning the CDP connection, with a
  25-tool dispatch table (webrain_eval, webrain_navigate, webrain_click,
  webrain_media, webrain_console, …).

• cli: webrain-cli single webrain binary, match-based subcommands
  (no clap) mirroring the MCP tool surface.

• core: page interaction — navigate, evaluate (arbitrary JS → JSON),
  click/type/press/scroll, snapshot, get_html, get_images,
  multi-tab (open_tab/close_tab), accessibility tree, overlay dismissal.

• core: capture — single + full-page screenshot (webrain_screenshot),
  PDF export, PixelRAG vision tiles (webrain_pixel), and a vision index with
  cosine VectorStore + embed Endpoint (webrain_vision_index / retrieve).

• core: extraction — webrain_extract_json (CSS-schema, zero-LLM),
  webrain_extract_regex (built-in patterns + custom {label, re}), and
  webrain_eval (JS → JSON).

• core: spider/crawl — BFS SpiderEngine (webrain_spider) and web search
  (webrain_search).

• core: batch + download — webrain_batch (fetch/extract/screenshot),
  webrain_download (streaming Body::into_reader, extension filter).

• cli: webrain doctor — full install diagnosis: version, MCP server, CDP
  ports (9222/9224/9225), engine discovery (chrome/lightpanda/obscura),
  encrypted vault, Python stealth sidecar, and a recommend line. Exit 0 when a
  browser is reachable. --doctor kept as an alias.

• core/cli: agent-browser-style engine install — webrain install downloads
  Chrome for Testing into a cache dir (WEBRAIN_BROWSERS_DIR) and that build
  wins discovery over system Chrome; webrain install --engine obscura downloads
  the latest Obscura release (--stealth picks the BoringSSL build).
  webrain lightpanda / webrain obscura spawn the CDP servers
  (launch_lightpanda/launch_obscura; binary from PATH / ~/.lightpanda /
  ~/.obscura / ~/.local/bin / WEBRAIN_LIGHTPANDA / WEBRAIN_OBSCURA).
  Windows .zip via the zip crate; linux/macOS .tar.gz via system tar.

• docs: README.md with all-OS install (cargo / homebrew / scoop /
  from-source), engine + MCP tool guides, marketplace/MCP-client setup, and repo
  logo (assets/webrain-logo.png).

• core/mcp: secure webrain_login — fully-automatic login from a local
  encrypted vault. The server decrypts the secret in-process and injects it into
  the browser via CDP; the value never passes through the model, chat, or logs.
  webrain_profiles lists vault entries (names only). Optional TOTP (RFC 6238)
  auto-injection when a site gates with 2FA.

• core/cli: webrain vault set|list|rm — enroll credentials with hidden
  prompts (never argv/chat). AES-256-GCM vault at %APPDATA%/webrain or
  ~/.config/webrain (vault.json index + 0600 vault.key), portable to any
  OS, no daemon. Optional TOTP seed at enroll.

• core: stealth hardening — PluginArray/MimeTypeArray rebuilt on the real
  prototype (a plain array is the classic detectable leak) with the standard PDF
  plugin names, navigator.connection (4g) stub, full window.chrome
  (app/csi/loadTimes) stub, permissions.query notifications reflection, plus
  CDP-level Network.setUserAgentOverride (real Windows Chrome 151 UA + Win32
  platform) and Emulation.setAutomationOverride on attach. Element snapshot
  redacts input[type=password] values.

• core/mcp: webrain_download engine="ytdlp" now works in the no-browser
  path too — it was silently forced onto the HTTP engine, so the advertised
  yt-dlp engine was dead over HTTP. One shared engines::download_ytdlp
  implementation serves both the stdio and HTTP transports.

• core/mcp: webrain_spider gains Scrapling AutoThrottle — adaptive
  per-domain delay tuned from observed latency (speeds up on fast servers,
  doubles on a blocked/error page, capped at autothrottle_max_ms, floored at
  delay_ms). Never guess a delay again.

• core/mcp: webrain_spider gains Scrapling crawldir checkpoint/resume —
  persists {queue, seen} every N pages; a later crawl with the same crawldir
  resumes from where it stopped. Checkpoint deleted on a clean (queue-drained)
  finish, kept when the crawl is capped/timed-out so resume continues.

• core/mcp: webrain_spider returns a stats block {elapsed_ms, pages_ok, pages_err, page_ms_total} — consistent with the batch stats block.

• core/mcp: webrain_sitemap tool — discover crawlable URLs from a site's
  sitemap (spider-rs crawl_sitemap / Scrapling SitemapSpider). Follows
  robots.txt Sitemap: → sitemap_index.xml → leaf sitemaps → every <loc>.
  Pure HTTP via the pooled agent, zero new deps (regex <loc> parse). Feed the
  returned URLs into webrain_batch/webrain_spider for a full crawl.

• core/mcp: webrain_spider gains Scrapling/spider-rs features:
  allow/deny URL regex filters (LinkExtractor allow/deny,
  spider-rs whitelist/blacklist), retry (re-fetch failed pages, 200ms backoff),
  delay_ms (polite crawl), and crawl_timeout_secs (hard wall-clock cap).
  Filters applied in the shared crawl loop — one spot covers every strategy.

• mcp: every tool response now carries ms (wall-clock elapsed) next to the
  existing tokens — per-tool-run latency + token cost at the one choke point
  (with_token_cost), both stdio and HTTP transports.

• core/mcp: batch results gain per-URL ms (tab-open→result wall-clock) and
  webrain_batch responses gain a stats block
  {total, ok, errors, ms_total} — the LLM sees at a glance which URL was slow
  and the whole run's cost, instead of counting result rows.

• core: webrain_navigate/snapshot now return a links field — deduped
  same-origin hrefs (≤200) via new LINKS_JS. One-call crawl/internal-link
  discovery (was: separate eval for hrefs).

• core: webrain_batch(op=extract|interact) results now carry a parsed
  data array (single-page extract_json shape) instead of a JSON string
  inside text (text kept for backward compat). Kills the data/text
  confusion an agent hits when tallying batch results.

• docs: agent guide + decision guide gain task-derived lessons — /ajax
  offset shortcut for load-more/infinite pages (fastest path, dedupe sliding
  windows), the async-eval-on-obscura null caveat (use op=interact), and the
  obscura Docker --host 0.0.0.0 requirement.

• core: adaptive selectors (Scrapling-style adaptive=True) — webrain_extract_json
  gains adaptive: bool. When the base selector matches 0 items (site redesigned / class
  renamed), the extractor auto-relocates to elements that still contain ≥2 of the field
  selectors, keeping only the deepest (row-level) candidates. Zero-LLM structural
  re-anchoring, all in-page via one evaluate().

• core: 3500-domain tracker blocklist — webrain_navigate/webrain_batch gain
  block_trackers: bool. Ported from anudeepND/blacklist via
  scripts/port_blocklist.ps1 into webrain-core/data/tracker_domains.txt, embedded at
  compile time (include_str!), lazy-parsed once. Applied to CDP only when opted in
  (~35KB over CDP per navigate) — the default fast path stays at the 28 wildcards.

• core: batch consolidation — 4 near-identical batch fns (fetch/extract/interact/
  screenshot, ~685–892 lines) collapsed into one generic batch_map<F, Fut> helper + 4
  thin wrappers (-74 net lines). One tab lifecycle (open → session → navigate → op →
  close) shared by every op, so a fix covers all callers.

• api: webrain_batch gains per_backend_concurrency — bounds tabs per CDP backend
  when cdp_urls is set (memory cap: total tabs = this × backends; default = concurrency).

• perf: bm25_filter now precomputes per-term doc-frequency once
  (O(docs·terms)) instead of re-scanning all docs per (doc, term) inside the
  score loop (O(docs²·terms)). Kills the flagged linear-scan-in-loop hot path.

• perf: hand-rolled base64_encode (24 ln, per-chunk allocations) replaced
  with base64::engine::general_purpose::STANDARD.encode — SIMD-accelerated
  stdlib, already a dep. Real speedup on the webrain_pixel tile path.

• docs: docs/adr/0001-webrain-architecture.md — Architecture Decision Record
  for the layered Cargo workspace (webrain-core engine + CDP backend, webrain-mcp
  transport with 34 tools, webrain-cli thin binary).

• mcp: session management tools — webrain_open_session, webrain_close_session,
  webrain_list_sessions. The LLM can now create named browser session pools
  (each with optional cdp_url for per-session browser routing), list active
  pools, and destroy them. This is the architectural unlock for auto-subagent
  orchestration: the LLM opens N sessions across different CDP_URLs, then farms
  MCP requests with different Mcp-Session-Id headers across parallel subagents.
  The existing Mcp-Session-Id routing + HttpState map already had the
  infrastructure — ~50 lines of MCP tool wrappers were added.
  CdpBackend::connect_with_url() added for per-session CDP routing.

• pdf: webrain_pdf_extract now uses the Firecrawl pdf-inspector engine
  (pure Rust, built on lopdf) instead of hand-rolled `ex...