v0.4.32 — the connection limit holds at the start of a job

Free open-source Google SERP parser and rank tracker, powered by BlankTrail Proxy.

The connection limit now holds at the start of a job too.

What changed since 0.4.31

• A job on a limited profile starts its sessions a few at a time. A
  session Google has not yet answered in this job — a new one, one that rested
  since the last job, or one on another address — nearly always meets a check
  on its first request, and a check holds the solver's own connections to the
  proxy for half a minute or more. Such a request is now counted as the solve it
  is likely to be (the profile's Reserve for solving a captcha) until its
  answer comes back. At a limit of 30 with the reserve of 5, about six start at
  once and the rest follow as checks are passed, instead of all threads
  meeting their checks together.
• Solves the service runs on its own are counted. BlankTrail reports the
  solves on each port (solving in /api/v1/ports/conns); a solve whose
  request has already given up is now held as the reserve, its port's
  connections are never closed by the parser, and it no longer skews what the
  parser learns about how many connections a request costs.
• The /search interface on a limited profile remembers which sessions
  have been answered for as long as the identities kept warm stand, so its
  requests are not each counted as a possible solve.
• Measured on our stand, 20 threads at a limit of 30 with BlankTrail's
  idle-connection close: 0.4.31 went over the limit for up to 38 seconds at the
  start (peak 52) while fourteen checks were solved at once; this build went
  over it for one second (peak 32), never ran more than five solves at once, and
  finished the same 200 queries in 6 min 49 s instead of 14 min 29 s. One run —
  the speed-up depends on the hour's exits too — but the start no longer
  stalls on a wave of checks.
• A profile without a limit behaves exactly as before. BlankTrail 1.3.24 or
  newer is still recommended: it serves the light listing, reports the solves
  and closes idle connections to the proxy.

This build needs nothing installed. On Windows it is one file: download
gserp.exe, put it anywhere, double-click it. The console goes away, an icon
appears in the notification area and your browser opens at
http://127.0.0.1:8080. Everything the interface needs is inside that file, and
it keeps its history in a gserp.db beside itself. It listens on this machine
only.

On Linux and macOS, unpack the archive for your system and run ./start.sh.

It does need BlankTrail Proxy with a licence including Challenge Breaker.
Google serves a JavaScript shell — zero results — to any plain HTTP client, no
matter how many proxies are behind it; results appear only after the solver has
carried the session through the challenge.

Which file

System	File
Windows, ordinary PC	gserp.exe — the whole program, nothing to unpack
Windows on ARM	gserp-arm64.exe
Linux, ordinary PC or server	gserp-linux-amd64.tar.gz
Linux on ARM	gserp-linux-arm64.tar.gz
Mac with Apple silicon (M1 and later)	gserp-macos-apple-silicon.tar.gz
Mac with an Intel processor	gserp-macos-intel.tar.gz

The archives hold the program, a starter script, both READMEs and the licence.
The zipped Windows builds are here too, for whoever wants the READMEs beside
the program.

Asset names carry no version, so a link to
releases/latest/download/<name> always fetches the newest build. Which
version you have is what gserp version prints.

Upgrading from 0.4.31: replace the program and start it; nothing in the history changes.

Upgrading from 0.4.2 – 0.4.30: replace the program and start it. The
history moves to schema version 34 on the first start; a copy of the database
before it costs nothing.

Upgrading from 0.4.0 or 0.4.1: replace the program and start it. The
history moves to schema version 34 on the first start — the jobs table is built
again to take the new filter — which on ten million results took a few
seconds. A copy of the database before the first start costs nothing.

Upgrading from 0.3.0 or older: replace the program and start it. The
history moves to schema version 34 on the first start — the jobs table is built
again so it can hold the new kind — and no setting has to be filled in again.
On a history of ten million results this took a few seconds; a copy of the
database before the first start costs nothing and is worth having.

Windows will warn about it

SmartScreen judges a program by the reputation of whoever signed it, and these
builds are unsigned: there is no certificate to have a reputation, so the
warning is not something the build can avoid. More info -> Run anyway.

What can be checked is that the file you have is the file published here.
SHA256SUMS.txt is beside the downloads:

Get-FileHash .\gserp.exe -Algorithm SHA256          # Windows
sha256sum -c SHA256SUMS.txt                         # Linux
shasum -a 256 -c SHA256SUMS.txt                     # macOS

Windows also marks anything downloaded, which is what raises the prompt.
Unblock-File .\gserp.exe clears it, once, per file.

macOS keeps downloaded programs quarantined in the same way. If it refuses to
open the file, clear the mark once with xattr -d com.apple.quarantine gserp in
the unpacked folder.

What the program does

• Four kinds of job: parsing, position check, index check and search
  suggestions. Organic results, ads and related queries, each exportable on
  its own, laid out field by field.
• Browser interface in English and Russian, light or dark, with a walk through
  it for whoever has just installed it.
• Proxy profiles: named sets of exits — a file, a URL, or the VPN gateways a
  BlankTrail subscription carries — optionally behind a first hop, with a job
  naming the one it goes out through.
• Sessions of its own: Google identities kept with their cookies and their
  address, rested between requests and written down as they go.
• Exports as CSV, JSON Lines and TXT; deduplication by URL or by host.
• HTTP API under /api/v1/, and a SerpApi-compatible GET /search a program
  written against that service reaches by changing the base address alone.

Measured on a live fifteen-thousand-address list with result addresses kept:
about 1100 pages a minute at a hundred threads, 30–40 captchas per thousand
pages once the start's wave of checks has passed. The start is the slow part — every session that rested between jobs is
checked again on its first request — and that is the shape to expect.

Full documentation: README
· по-русски

MIT licence. Provided as is, without warranty; observing the terms of service
of the sites you point it at, and the law where you are, is yours.