Skip to content

Releases

v0.2.3

Choose a tag to compare

@alkaz-nodemaven alkaz-nodemaven released this 12 Nov 16:51
ed2cd91
  • One-line setup: pip install serpmaven or uv tool install serpmaven, then serpmaven login saves your proxy login once.
  • The browser installs itself on the first run (asks in a terminal), or with serpmaven install.
  • A failed link lookup is reported as a link_lookup_failed event with its reason instead of an empty link.
  • Every command-line option is documented in the README options table.
  • CI: test coverage floor, dead-code and complexity checks.

v0.2.2: SERP Maven

Choose a tag to compare

@alkaz-nodemaven alkaz-nodemaven released this 11 Nov 10:54
280e291

SERP Maven. The project is now serpmaven:

pip install serpmaven
serpmaven search "best running shoes" --nodemaven

The old names keep working: pip install gserp and pip install google-browser-scraper now install serpmaven, with their old commands, imports and GSERP_* / GBS_* variables. Docker image: ghcr.io/nodemaven/serpmaven.

Also in this release:

  • Old names: an exported function sharing a module's name (classify) is kept; released google-browser-scraper 0.2.x keeps importing over the new gserp; old commands name themselves in usage.

v0.2.1

Choose a tag to compare

@alkaz-nodemaven alkaz-nodemaven released this 10 Nov 13:03
05cb000

Changes

  • You can see what a run is doing. gserp search now prints its progress in plain words: the proxy IP warming up, each query served, an IP dropped and why, and a readable summary at the end. -v keeps the JSON events.
  • Setup errors are one line. A missing browser, a missing display on a Linux server, a missing query file or missing credentials each answer with one line saying what to run, instead of a long traceback.
  • gserp doctor ends with ready or not ready.
  • Less traffic on a warm-up. The local relay also refuses Chrome's own component downloads and the videos of a warm-up page, which no search needs.
  • A first search is no longer interrupted by a warm-up page that navigates on its own: the warm-up ends on a blank page.
  • Plain HTTP through the relay is one request per connection, so the block list applies to every request.
  • README: install steps with a virtual environment, PowerShell commands, and what a Linux server without a screen needs.

pip install -U gserp. google-browser-scraper 0.2.1 installs gserp 0.2.1.

v0.2.0: gserp

Choose a tag to compare

@alkaz-nodemaven alkaz-nodemaven released this 09 Nov 11:38
7c757ef

google-browser-scraper is now gserp. pip install gserp; the command is gserp, the import gserp, the image ghcr.io/nodemaven/gserp. pip install google-browser-scraper still works: it installs gserp and keeps the old command, import google_browser_scraper (with a warning) and the GBS_* variables.

Changes

  • Cookie sets on by default. Every 25 served queries, a proxy IP's Google cookies become the set new proxy IPs start from, so a run warms up its first proxy IP and rarely another. --cookies FILE starts from saved cookies, --cookies-out DIR saves them, --renew-cookies 0 goes back to copies of the first served profile.
  • Held links are followed. Links Google holds behind its "Verifying your request" page are asked again once the wait is over, so result links come back complete.
  • serve answers before every link is in. links=N resolves only the top N results, links_wait=S answers after S seconds with links marked pending, async=true answers at once; GET /searches/<id>.json returns the finished record.
  • Cleaner results. The "AI Mode replied:" label is dropped from ai_overview, and the unrelated-results check matches query words written with /, - or ..
  • --no-hybrid resolves links too.
  • Environment variables are GSERP_PROXY, GSERP_API_KEY, GSERP_BROWSER_ARGS; the GBS_* names are still read.

v0.1.2

Choose a tag to compare

@alkaz-nodemaven alkaz-nodemaven released this 30 Oct 15:42
8841192

Every result has its link again

Google's results pages now carry only a redirect for most results. Each link is resolved on the same proxy IP the page came from, so link is filled for every organic result again.

pip install -U "google-browser-scraper[nodemaven]"

v0.1.1

Choose a tag to compare

@alkaz-nodemaven alkaz-nodemaven released this 30 Oct 08:17
f16aa18

Poisoned results caught sooner

A proxy IP that slips unrelated results in between normal pages is now dropped too. The check looks at the last five pages instead of waiting for three bad ones in a row, so poisoned exits are dropped within a few pages.

pip install -U "google-browser-scraper[nodemaven]"

v0.1.0

Choose a tag to compare

@alkaz-nodemaven alkaz-nodemaven released this 29 Oct 09:14
79b42fd

Hybrid mode: 2x less traffic, 8x faster

A real browser gets past Google's checks; once it has, results pages are fetched directly over HTTPS with the browser's own cookies and headers. The browser steps in only when Google asks for its check again, and the scraper switches back on its own if anything changes. On by default (--no-hybrid turns it off).

  • 0.4 MB per search, down from 0.8 MB
  • 1.1 s per search, down from 9 s
  • 100% success rate

Poisoned results protection

Google sometimes answers scrapers not with a block but with a normal-looking page of results for a different query. The scraper checks every page against the query it asked, drops a proxy IP that starts returning unrelated results and asks again on a fresh one, so poisoned pages never reach your output. No extra requests or traffic.

Also new

  • AI Overview and map places (name, rating, reviews, address, phone, hours) in the JSON and in the MCP tool
  • Language and country per request in the HTTP API and the MCP tool
  • Faster HTTP API: answers in under 2 seconds once warm
  • Latest Chromium and TLS fingerprint
pip install -U "google-browser-scraper[nodemaven]"

v0.0.6.2

Choose a tag to compare

@alkaz-nodemaven alkaz-nodemaven released this 22 Oct 13:40
  • serve and mcp send the answer first; the upkeep a served query leaves (template copy, scheduled restart, retiring a capped exit) runs after it.
  • --prewarm searches one page on fresh exits until Google serves one, so the first request does not pay for a refused exit and a second warm-up. /health answers 503 warming until then, busy while a search holds the scraper and down when the worker has stopped.
  • Stopping serve or mcp skips upkeep still queued and waits for the job in hand, so the browser and temporary folders are not left behind.
  • A failure to create the cookie template folder no longer blocks later templates.

v0.0.6.1

Choose a tag to compare

@alkaz-nodemaven alkaz-nodemaven released this 21 Oct 18:29
  • The browser restarts every 50 results pages (was 100), so two browsers fit on a small server.
  • A search that runs out of memory or meets a crashed browser retires the exit instead of retrying on it, so the run goes on instead of hanging.
  • README: benchmark under the demo, neutral SERP API wording.

v0.0.6

Choose a tag to compare

@alkaz-nodemaven alkaz-nodemaven released this 21 Oct 13:28
  • The browser restarts every 100 results pages on the same profile and exit (--restart-every), so its memory stays flat over long runs.
  • A served exit is held for up to 300 searches (was 50).
  • Copies of a served profile carry the browser's HTTP cache along with the cookies, so a new exit starts with cheap pages; the copy is refreshed at each restart. After two refused copies carrying the cache, the run copies cookies only (--no-clone-cache from the start).
  • --nodemaven-demo: run on a temporary NodeMaven sub-user with a traffic cap (by @arslan-atajykov).
  • Every -v event carries a timestamp.
  • Docker image on GitHub Container Registry: ghcr.io/nodemaven/google-browser-scraper.
  • README: demo GIF, options table.