Repository navigation
Releases
v0.2.3
- One-line setup:
pip install serpmavenoruv tool install serpmaven, thenserpmaven loginsaves your proxy login once. - The browser installs itself on the first run (asks in a terminal), or with
serpmaven install. - A failed link lookup is reported as a
link_lookup_failedevent with its reason instead of an empty link. - Every command-line option is documented in the README options table.
- CI: test coverage floor, dead-code and complexity checks.
v0.2.2: SERP Maven
SERP Maven. The project is now serpmaven:
pip install serpmaven
serpmaven search "best running shoes" --nodemaven
The old names keep working: pip install gserp and pip install google-browser-scraper now install serpmaven, with their old commands, imports and GSERP_* / GBS_* variables. Docker image: ghcr.io/nodemaven/serpmaven.
Also in this release:
- Old names: an exported function sharing a module's name (
classify) is kept; released google-browser-scraper 0.2.x keeps importing over the new gserp; old commands name themselves in usage.
v0.2.1
Changes
- You can see what a run is doing.
gserp searchnow prints its progress in plain words: the proxy IP warming up, each query served, an IP dropped and why, and a readable summary at the end.-vkeeps the JSON events. - Setup errors are one line. A missing browser, a missing display on a Linux server, a missing query file or missing credentials each answer with one line saying what to run, instead of a long traceback.
gserp doctorends withreadyornot ready.- Less traffic on a warm-up. The local relay also refuses Chrome's own component downloads and the videos of a warm-up page, which no search needs.
- A first search is no longer interrupted by a warm-up page that navigates on its own: the warm-up ends on a blank page.
- Plain HTTP through the relay is one request per connection, so the block list applies to every request.
- README: install steps with a virtual environment, PowerShell commands, and what a Linux server without a screen needs.
pip install -U gserp. google-browser-scraper 0.2.1 installs gserp 0.2.1.
v0.2.0: gserp
google-browser-scraper is now gserp. pip install gserp; the command is gserp, the import gserp, the image ghcr.io/nodemaven/gserp. pip install google-browser-scraper still works: it installs gserp and keeps the old command, import google_browser_scraper (with a warning) and the GBS_* variables.
Changes
- Cookie sets on by default. Every 25 served queries, a proxy IP's Google cookies become the set new proxy IPs start from, so a run warms up its first proxy IP and rarely another.
--cookies FILEstarts from saved cookies,--cookies-out DIRsaves them,--renew-cookies 0goes back to copies of the first served profile. - Held links are followed. Links Google holds behind its "Verifying your request" page are asked again once the wait is over, so result links come back complete.
- serve answers before every link is in.
links=Nresolves only the top N results,links_wait=Sanswers after S seconds with links marked pending,async=trueanswers at once;GET /searches/<id>.jsonreturns the finished record. - Cleaner results. The "AI Mode replied:" label is dropped from
ai_overview, and the unrelated-results check matches query words written with/,-or.. --no-hybridresolves links too.- Environment variables are
GSERP_PROXY,GSERP_API_KEY,GSERP_BROWSER_ARGS; theGBS_*names are still read.
v0.1.2
Every result has its link again
Google's results pages now carry only a redirect for most results. Each link is resolved on the same proxy IP the page came from, so link is filled for every organic result again.
pip install -U "google-browser-scraper[nodemaven]"v0.1.1
Poisoned results caught sooner
A proxy IP that slips unrelated results in between normal pages is now dropped too. The check looks at the last five pages instead of waiting for three bad ones in a row, so poisoned exits are dropped within a few pages.
pip install -U "google-browser-scraper[nodemaven]"v0.1.0
Hybrid mode: 2x less traffic, 8x faster
A real browser gets past Google's checks; once it has, results pages are fetched directly over HTTPS with the browser's own cookies and headers. The browser steps in only when Google asks for its check again, and the scraper switches back on its own if anything changes. On by default (--no-hybrid turns it off).
- 0.4 MB per search, down from 0.8 MB
- 1.1 s per search, down from 9 s
- 100% success rate
Poisoned results protection
Google sometimes answers scrapers not with a block but with a normal-looking page of results for a different query. The scraper checks every page against the query it asked, drops a proxy IP that starts returning unrelated results and asks again on a fresh one, so poisoned pages never reach your output. No extra requests or traffic.
Also new
- AI Overview and map places (name, rating, reviews, address, phone, hours) in the JSON and in the MCP tool
- Language and country per request in the HTTP API and the MCP tool
- Faster HTTP API: answers in under 2 seconds once warm
- Latest Chromium and TLS fingerprint
pip install -U "google-browser-scraper[nodemaven]"v0.0.6.2
serveandmcpsend the answer first; the upkeep a served query leaves (template copy, scheduled restart, retiring a capped exit) runs after it.--prewarmsearches one page on fresh exits until Google serves one, so the first request does not pay for a refused exit and a second warm-up./healthanswers 503warminguntil then,busywhile a search holds the scraper anddownwhen the worker has stopped.- Stopping
serveormcpskips upkeep still queued and waits for the job in hand, so the browser and temporary folders are not left behind. - A failure to create the cookie template folder no longer blocks later templates.
v0.0.6.1
- The browser restarts every 50 results pages (was 100), so two browsers fit on a small server.
- A search that runs out of memory or meets a crashed browser retires the exit instead of retrying on it, so the run goes on instead of hanging.
- README: benchmark under the demo, neutral SERP API wording.
v0.0.6
- The browser restarts every 100 results pages on the same profile and exit (
--restart-every), so its memory stays flat over long runs. - A served exit is held for up to 300 searches (was 50).
- Copies of a served profile carry the browser's HTTP cache along with the cookies, so a new exit starts with cheap pages; the copy is refreshed at each restart. After two refused copies carrying the cache, the run copies cookies only (
--no-clone-cachefrom the start). --nodemaven-demo: run on a temporary NodeMaven sub-user with a traffic cap (by @arslan-atajykov).- Every
-vevent carries a timestamp. - Docker image on GitHub Container Registry:
ghcr.io/nodemaven/google-browser-scraper. - README: demo GIF, options table.