Skip to content

v0.4.0 — Search resilience overhaul + CSP fix + href_pattern fix

Choose a tag to compare

View all tags
@That1Drifter That1Drifter released this 27 May 13:45
· 28 commits to main since this release
v0.4.0
1782a4b
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Release notes for 0.4.0 (2026-05-27). See CHANGELOG.md for the full structured entry.

Added

  • BROWSE_MCP_TAVILY_API_KEY opt-in Tavily Search provider (1000 req/mo free, AI-curated). Tried after Brave in the search provider chain. (#27)
  • Playwright-rendered DDG/Bing fallback for browser_search. When the fetch-based scrape returns 0 (anti-bot interstitial or layout drift), the search reruns inside the real browser context and parses the live DOM. Layered chain: Brave (key) → Tavily (key) → fetch DDG → fetch Bing → rendered DDG → rendered Bing. (#24)
  • Structured difficulty log entries with 600-char htmlExcerpt, page title, and URL when rendered DDG/Bing fallbacks return 0 results. Visible via browser_review_issues for drift debugging without a live repro. (#26)

Fixed

  • browser_extract_listings({ href_pattern: "/inventory/used" }) no longer throws Invalid flags supplied to RegExp constructor 'used'. href_pattern is now treated as a substring by default; the value is only parsed as a regex when wrapped in /.../flags AND flags validate against /^[gimsuy]*$/. Applied symmetrically to browser_links. (#22)
  • browser_read no longer fails with Refused to execute inline script on github.com, Cloudflare-challenged pages, and any site with strict script-src CSP. Readability source now runs through page.evaluate(<string>) (CDP Runtime.evaluate, which bypasses CSP) instead of page.addScriptTag which injects a real <script> element. (#23)

Changed

  • FRAGILITY NOTICE in src/search.ts and the README search section now lead with the API-key recommendation. The scrape rungs are documented as a best-effort backstop; the new fallback chain and telemetry are documented as the response to Cloudflare / TLS-JA3 fingerprinting that increasingly blocks Playwright clients.

Recommended setup for new installs

For reliable search, set one of:

Provider Free tier Env var
Brave Search API ~1k req/mo on the free credit BROWSE_MCP_BRAVE_API_KEY
Tavily Search 1000 req/mo, AI-curated BROWSE_MCP_TAVILY_API_KEY

Either is sufficient. The scrape fallback still works for zero-config installs.

Assets 2
Loading