v0.4.34 — a solve counted as three connections by default

Free open-source Google SERP parser and rank tracker, powered by BlankTrail Proxy.

A solve is counted as three connections by default, and a job on a limited profile starts quickly again.

What changed since 0.4.33

• The Reserve for solving a captcha is 3 by default, not 5. Five was the
  most a solve was seen to open; once BlankTrail closes idle connections a
  solving port holds about three. The reserve is also how many new sessions a
  limit starts at once (each is counted as a possible solve until Google
  answers it), so at a limit of 30 about ten start together instead of six.
• Existing profiles: a profile with no connection limit that still carries
  the old 5 takes the new 3 on the first start (schema version 35). A profile
  with a limit set, or any other figure, is left as it is — set 3 on the
  proxies screen if you want the faster start there too.
• Measured on our stand, 20 threads at a limit of 30 with the reserve of 3: the
  limit held (four seconds above it at most, peak 32), no query failed, the
  first result came after 33 seconds (0.4.33: 167) and the 200 queries took
  7 min 5 s (0.4.33: 9 min 9 s). One run.
• If your solver opens more connections than three for a captcha, raise the
  reserve: counting too few lets a job past the limit while the solves last.

This build needs nothing installed. On Windows it is one file: download
gserp.exe, put it anywhere, double-click it. The console goes away, an icon
appears in the notification area and your browser opens at
http://127.0.0.1:8080. Everything the interface needs is inside that file, and
it keeps its history in a gserp.db beside itself. It listens on this machine
only.

On Linux and macOS, unpack the archive for your system and run ./start.sh.

It does need BlankTrail Proxy with a licence including Challenge Breaker.
Google serves a JavaScript shell — zero results — to any plain HTTP client, no
matter how many proxies are behind it; results appear only after the solver has
carried the session through the challenge.

Which file

System	File
Windows, ordinary PC	gserp.exe — the whole program, nothing to unpack
Windows on ARM	gserp-arm64.exe
Linux, ordinary PC or server	gserp-linux-amd64.tar.gz
Linux on ARM	gserp-linux-arm64.tar.gz
Mac with Apple silicon (M1 and later)	gserp-macos-apple-silicon.tar.gz
Mac with an Intel processor	gserp-macos-intel.tar.gz

The archives hold the program, a starter script, both READMEs and the licence.
The zipped Windows builds are here too, for whoever wants the READMEs beside
the program.

Asset names carry no version, so a link to
releases/latest/download/<name> always fetches the newest build. Which
version you have is what gserp version prints.

Upgrading from 0.4.31 – 0.4.33: replace the program and start it. The history moves to schema version 35 on the first start, which only changes the reserve of profiles with no limit; a copy of the database before it costs nothing.

Upgrading from 0.4.2 – 0.4.30: replace the program and start it. The
history moves to schema version 35 on the first start; a copy of the database
before it costs nothing.

Upgrading from 0.4.0 or 0.4.1: replace the program and start it. The
history moves to schema version 35 on the first start — the jobs table is built
again to take the new filter — which on ten million results took a few
seconds. A copy of the database before the first start costs nothing.

Upgrading from 0.3.0 or older: replace the program and start it. The
history moves to schema version 35 on the first start — the jobs table is built
again so it can hold the new kind — and no setting has to be filled in again.
On a history of ten million results this took a few seconds; a copy of the
database before the first start costs nothing and is worth having.

Windows will warn about it

SmartScreen judges a program by the reputation of whoever signed it, and these
builds are unsigned: there is no certificate to have a reputation, so the
warning is not something the build can avoid. More info -> Run anyway.

What can be checked is that the file you have is the file published here.
SHA256SUMS.txt is beside the downloads:

Get-FileHash .\gserp.exe -Algorithm SHA256          # Windows
sha256sum -c SHA256SUMS.txt                         # Linux
shasum -a 256 -c SHA256SUMS.txt                     # macOS

Windows also marks anything downloaded, which is what raises the prompt.
Unblock-File .\gserp.exe clears it, once, per file.

macOS keeps downloaded programs quarantined in the same way. If it refuses to
open the file, clear the mark once with xattr -d com.apple.quarantine gserp in
the unpacked folder.

What the program does

• Four kinds of job: parsing, position check, index check and search
  suggestions. Organic results, ads and related queries, each exportable on
  its own, laid out field by field.
• Browser interface in English and Russian, light or dark, with a walk through
  it for whoever has just installed it.
• Proxy profiles: named sets of exits — a file, a URL, or the VPN gateways a
  BlankTrail subscription carries — optionally behind a first hop, with a job
  naming the one it goes out through.
• Sessions of its own: Google identities kept with their cookies and their
  address, rested between requests and written down as they go.
• Exports as CSV, JSON Lines and TXT; deduplication by URL or by host.
• HTTP API under /api/v1/, and a SerpApi-compatible GET /search a program
  written against that service reaches by changing the base address alone.

Measured on a live fifteen-thousand-address list with result addresses kept:
about 1100 pages a minute at a hundred threads, 30–40 captchas per thousand
pages once the start's wave of checks has passed. The start is the slow part — every session that rested between jobs is
checked again on its first request — and that is the shape to expect.

Full documentation: README
· по-русски

MIT licence. Provided as is, without warranty; observing the terms of service
of the sites you point it at, and the law where you are, is yours.

v0.4.33 — a stricter, more exact connection limit

Free open-source Google SERP parser and rank tracker, powered by BlankTrail Proxy.

A stricter, more exact connection limit.

What changed since 0.4.32

• The service's solves are matched to the parser's port by port. A solve
  BlankTrail runs on a port where the parser has a request waiting is that
  request's solve, counted once; one on a port the parser has left is a solve
  nobody waits for, and holds a reserve of its own. 0.4.32 let new sessions
  start beside such leftover solves without counting them, which could take a
  profile over its limit when those sessions met checks too.
• A new session waiting too long is no longer passed by. Once a request
  for a fresh session has waited ten seconds, ordinary searches stop taking the
  places ahead of it — unless the room it needs is held by something the
  parser cannot give back (another program on the same profile, connections
  the service keeps), in which case nothing is held back for it.
• Sessions being kept warm, and a request the parser carries to another
  address, are counted as the possible solves they are.
• How many connections a request costs is learned more accurately while
  checks are being solved.
• The start of a job on a limited profile is slower than in 0.4.32. On our
  stand, 20 threads at a limit of 30 with the reserve of 5: the limit held
  (four seconds above it at most, peak 33), no query failed, and the job took
  9 min 9 s against 6 min 49 s for 0.4.32 in an hour with twice as many checks
  — six checks at once fill a limit of 30, and no new session starts until
  one of them is solved. A larger limit, or a smaller Reserve for solving a
  captcha if your solver opens fewer connections, starts more at once.
• A profile without a limit behaves exactly as before.

This build needs nothing installed. On Windows it is one file: download
gserp.exe, put it anywhere, double-click it. The console goes away, an icon
appears in the notification area and your browser opens at
http://127.0.0.1:8080. Everything the interface needs is inside that file, and
it keeps its history in a gserp.db beside itself. It listens on this machine
only.

On Linux and macOS, unpack the archive for your system and run ./start.sh.

It does need BlankTrail Proxy with a licence including Challenge Breaker.
Google serves a JavaScript shell — zero results — to any plain HTTP client, no
matter how many proxies are behind it; results appear only after the solver has
carried the session through the challenge.

Which file

System	File
Windows, ordinary PC	gserp.exe — the whole program, nothing to unpack
Windows on ARM	gserp-arm64.exe
Linux, ordinary PC or server	gserp-linux-amd64.tar.gz
Linux on ARM	gserp-linux-arm64.tar.gz
Mac with Apple silicon (M1 and later)	gserp-macos-apple-silicon.tar.gz
Mac with an Intel processor	gserp-macos-intel.tar.gz

The archives hold the program, a starter script, both READMEs and the licence.
The zipped Windows builds are here too, for whoever wants the READMEs beside
the program.

Asset names carry no version, so a link to
releases/latest/download/<name> always fetches the newest build. Which
version you have is what gserp version prints.

Upgrading from 0.4.31 or 0.4.32: replace the program and start it; nothing in the history changes.

Upgrading from 0.4.2 – 0.4.30: replace the program and start it. The
history moves to schema version 34 on the first start; a copy of the database
before it costs nothing.

Upgrading from 0.4.0 or 0.4.1: replace the program and start it. The
history moves to schema version 34 on the first start — the jobs table is built
again to take the new filter — which on ten million results took a few
seconds. A copy of the database before the first start costs nothing.

Upgrading from 0.3.0 or older: replace the program and start it. The
history moves to schema version 34 on the first start — the jobs table is built
again so it can hold the new kind — and no setting has to be filled in again.
On a history of ten million results this took a few seconds; a copy of the
database before the first start costs nothing and is worth having.

Windows will warn about it

SmartScreen judges a program by the reputation of whoever signed it, and these
builds are unsigned: there is no certificate to have a reputation, so the
warning is not something the build can avoid. More info -> Run anyway.

What can be checked is that the file you have is the file published here.
SHA256SUMS.txt is beside the downloads:

Get-FileHash .\gserp.exe -Algorithm SHA256          # Windows
sha256sum -c SHA256SUMS.txt                         # Linux
shasum -a 256 -c SHA256SUMS.txt                     # macOS

Windows also marks anything downloaded, which is what raises the prompt.
Unblock-File .\gserp.exe clears it, once, per file.

macOS keeps downloaded programs quarantined in the same way. If it refuses to
open the file, clear the mark once with xattr -d com.apple.quarantine gserp in
the unpacked folder.

What the program does

• Four kinds of job: parsing, position check, index check and search
  suggestions. Organic results, ads and related queries, each exportable on
  its own, laid out field by field.
• Browser interface in English and Russian, light or dark, with a walk through
  it for whoever has just installed it.
• Proxy profiles: named sets of exits — a file, a URL, or the VPN gateways a
  BlankTrail subscription carries — optionally behind a first hop, with a job
  naming the one it goes out through.
• Sessions of its own: Google identities kept with their cookies and their
  address, rested between requests and written down as they go.
• Exports as CSV, JSON Lines and TXT; deduplication by URL or by host.
• HTTP API under /api/v1/, and a SerpApi-compatible GET /search a program
  written against that service reaches by changing the base address alone.

Measured on a live fifteen-thousand-address list with result addresses kept:
about 1100 pages a minute at a hundred threads, 30–40 captchas per thousand
pages once the start's wave of checks has passed. The start is the slow part — every session that rested between jobs is
checked again on its first request — and that is the shape to expect.

Full documentation: README
· по-русски

MIT licence. Provided as is, without warranty; observing the terms of service
of the sites you point it at, and the law where you are, is yours.

v0.4.32 — the connection limit holds at the start of a job

Free open-source Google SERP parser and rank tracker, powered by BlankTrail Proxy.

The connection limit now holds at the start of a job too.

What changed since 0.4.31

• A job on a limited profile starts its sessions a few at a time. A
  session Google has not yet answered in this job — a new one, one that rested
  since the last job, or one on another address — nearly always meets a check
  on its first request, and a check holds the solver's own connections to the
  proxy for half a minute or more. Such a request is now counted as the solve it
  is likely to be (the profile's Reserve for solving a captcha) until its
  answer comes back. At a limit of 30 with the reserve of 5, about six start at
  once and the rest follow as checks are passed, instead of all threads
  meeting their checks together.
• Solves the service runs on its own are counted. BlankTrail reports the
  solves on each port (solving in /api/v1/ports/conns); a solve whose
  request has already given up is now held as the reserve, its port's
  connections are never closed by the parser, and it no longer skews what the
  parser learns about how many connections a request costs.
• The /search interface on a limited profile remembers which sessions
  have been answered for as long as the identities kept warm stand, so its
  requests are not each counted as a possible solve.
• Measured on our stand, 20 threads at a limit of 30 with BlankTrail's
  idle-connection close: 0.4.31 went over the limit for up to 38 seconds at the
  start (peak 52) while fourteen checks were solved at once; this build went
  over it for one second (peak 32), never ran more than five solves at once, and
  finished the same 200 queries in 6 min 49 s instead of 14 min 29 s. One run —
  the speed-up depends on the hour's exits too — but the start no longer
  stalls on a wave of checks.
• A profile without a limit behaves exactly as before. BlankTrail 1.3.24 or
  newer is still recommended: it serves the light listing, reports the solves
  and closes idle connections to the proxy.

This build needs nothing installed. On Windows it is one file: download
gserp.exe, put it anywhere, double-click it. The console goes away, an icon
appears in the notification area and your browser opens at
http://127.0.0.1:8080. Everything the interface needs is inside that file, and
it keeps its history in a gserp.db beside itself. It listens on this machine
only.

On Linux and macOS, unpack the archive for your system and run ./start.sh.

It does need BlankTrail Proxy with a licence including Challenge Breaker.
Google serves a JavaScript shell — zero results — to any plain HTTP client, no
matter how many proxies are behind it; results appear only after the solver has
carried the session through the challenge.

Which file

System	File
Windows, ordinary PC	gserp.exe — the whole program, nothing to unpack
Windows on ARM	gserp-arm64.exe
Linux, ordinary PC or server	gserp-linux-amd64.tar.gz
Linux on ARM	gserp-linux-arm64.tar.gz
Mac with Apple silicon (M1 and later)	gserp-macos-apple-silicon.tar.gz
Mac with an Intel processor	gserp-macos-intel.tar.gz

The archives hold the program, a starter script, both READMEs and the licence.
The zipped Windows builds are here too, for whoever wants the READMEs beside
the program.

Asset names carry no version, so a link to
releases/latest/download/<name> always fetches the newest build. Which
version you have is what gserp version prints.

Upgrading from 0.4.31: replace the program and start it; nothing in the history changes.

Upgrading from 0.4.2 – 0.4.30: replace the program and start it. The
history moves to schema version 34 on the first start; a copy of the database
before it costs nothing.

Upgrading from 0.4.0 or 0.4.1: replace the program and start it. The
history moves to schema version 34 on the first start — the jobs table is built
again to take the new filter — which on ten million results took a few
seconds. A copy of the database before the first start costs nothing.

Upgrading from 0.3.0 or older: replace the program and start it. The
history moves to schema version 34 on the first start — the jobs table is built
again so it can hold the new kind — and no setting has to be filled in again.
On a history of ten million results this took a few seconds; a copy of the
database before the first start costs nothing and is worth having.

Windows will warn about it

SmartScreen judges a program by the reputation of whoever signed it, and these
builds are unsigned: there is no certificate to have a reputation, so the
warning is not something the build can avoid. More info -> Run anyway.

What can be checked is that the file you have is the file published here.
SHA256SUMS.txt is beside the downloads:

Get-FileHash .\gserp.exe -Algorithm SHA256          # Windows
sha256sum -c SHA256SUMS.txt                         # Linux
shasum -a 256 -c SHA256SUMS.txt                     # macOS

Windows also marks anything downloaded, which is what raises the prompt.
Unblock-File .\gserp.exe clears it, once, per file.

macOS keeps downloaded programs quarantined in the same way. If it refuses to
open the file, clear the mark once with xattr -d com.apple.quarantine gserp in
the unpacked folder.

What the program does

• Four kinds of job: parsing, position check, index check and search
  suggestions. Organic results, ads and related queries, each exportable on
  its own, laid out field by field.
• Browser interface in English and Russian, light or dark, with a walk through
  it for whoever has just installed it.
• Proxy profiles: named sets of exits — a file, a URL, or the VPN gateways a
  BlankTrail subscription carries — optionally behind a first hop, with a job
  naming the one it goes out through.
• Sessions of its own: Google identities kept with their cookies and their
  address, rested between requests and written down as they go.
• Exports as CSV, JSON Lines and TXT; deduplication by URL or by host.
• HTTP API under /api/v1/, and a SerpApi-compatible GET /search a program
  written against that service reaches by changing the base address alone.

Measured on a live fifteen-thousand-address list with result addresses kept:
about 1100 pages a minute at a hundred threads, 30–40 captchas per thousand
pages once the start's wave of checks has passed. The start is the slow part — every session that rested between jobs is
checked again on its first request — and that is the shape to expect.

Full documentation: README
· по-русски

MIT licence. Provided as is, without warranty; observing the terms of service
of the sites you point it at, and the law where you are, is yours.

v0.4.31 — a connection limit for a proxy profile

Free open-source Google SERP parser and rank tracker, powered by BlankTrail Proxy.

A connection limit for a proxy profile, and no busy loop on a profile with no exits.

What changed since 0.4.30

• A proxy profile can have a connection limit. The proxies screen has two
  new boxes: Connections to the proxy (0 — no limit) and Reserve for solving
  a captcha (5 by default). Every job and the identities kept warm on that
  profile share the limit. Once it is reached the parser holds new requests
  back instead of failing them: searches and captchas being solved go first,
  hidden addresses are read on what is left, and the number of address reads at
  once follows the room. A solve is counted as the reserve, because the solver
  opens connections of its own, and its connections are never cut.
• The parser corrects its count from BlankTrail's. It learns how many
  connections to the proxy the service opens per request and reads the
  service's count — every two seconds while the profile carries traffic, every
  ten when it is idle — from the light listing /api/v1/ports/conns; an older
  BlankTrail without it is read from the whole port listing, less often.
• It is a soft limit. Connections the service holds by itself — retries,
  the solver's own, idle ones — are counted and hold new requests back, but the
  parser cannot close them, so the count can briefly go over the limit until
  BlankTrail lets them go. On our stand, 20 threads at a limit of 30 peaked at
  37 connections where the same job without a limit reached 106, at the same
  speed. Use BlankTrail 1.3.24 or newer: it serves the light listing and
  closes idle connections to the proxy two seconds after the last client
  leaves, which keeps those overshoots short.
• The job page shows Connections to the proxy: U of L, captchas being solved:
  S, and the new-job form warns when the threads asked for are more than the
  limit leaves room for. Ports open on the job page now counts the ports used
  for reading hidden addresses as well. Nought is no limit, and a profile
  without one behaves as before.
• A profile with no exits no longer loops. A job on a profile that names no
  exits asked the service for a fresh fingerprint over and over — about 1500
  requests every two seconds — and the same happened on a list with every
  address at its session limit. Now no ports are opened for such a profile: its
  jobs wait in the queue, start by themselves once exits are added, and can be
  stopped or deleted while they wait. The banner says which jobs are waiting,
  and settings can be saved meanwhile.
• Search suggestions go direct again, as before, and do not wait for a
  profile's exits.

This build needs nothing installed. On Windows it is one file: download
gserp.exe, put it anywhere, double-click it. The console goes away, an icon
appears in the notification area and your browser opens at
http://127.0.0.1:8080. Everything the interface needs is inside that file, and
it keeps its history in a gserp.db beside itself. It listens on this machine
only.

On Linux and macOS, unpack the archive for your system and run ./start.sh.

It does need BlankTrail Proxy with a licence including Challenge Breaker.
Google serves a JavaScript shell — zero results — to any plain HTTP client, no
matter how many proxies are behind it; results appear only after the solver has
carried the session through the challenge.

Which file

System	File
Windows, ordinary PC	gserp.exe — the whole program, nothing to unpack
Windows on ARM	gserp-arm64.exe
Linux, ordinary PC or server	gserp-linux-amd64.tar.gz
Linux on ARM	gserp-linux-arm64.tar.gz
Mac with Apple silicon (M1 and later)	gserp-macos-apple-silicon.tar.gz
Mac with an Intel processor	gserp-macos-intel.tar.gz

The archives hold the program, a starter script, both READMEs and the licence.
The zipped Windows builds are here too, for whoever wants the READMEs beside
the program.

Asset names carry no version, so a link to
releases/latest/download/<name> always fetches the newest build. Which
version you have is what gserp version prints.

Upgrading from 0.4.2 – 0.4.30: replace the program and start it. The
history moves to schema version 34 on the first start; a copy of the database
before it costs nothing.

Upgrading from 0.4.0 or 0.4.1: replace the program and start it. The
history moves to schema version 34 on the first start — the jobs table is built
again to take the new filter — which on ten million results took a few
seconds. A copy of the database before the first start costs nothing.

Upgrading from 0.3.0 or older: replace the program and start it. The
history moves to schema version 34 on the first start — the jobs table is built
again so it can hold the new kind — and no setting has to be filled in again.
On a history of ten million results this took a few seconds; a copy of the
database before the first start costs nothing and is worth having.

Windows will warn about it

SmartScreen judges a program by the reputation of whoever signed it, and these
builds are unsigned: there is no certificate to have a reputation, so the
warning is not something the build can avoid. More info -> Run anyway.

What can be checked is that the file you have is the file published here.
SHA256SUMS.txt is beside the downloads:

Get-FileHash .\gserp.exe -Algorithm SHA256          # Windows
sha256sum -c SHA256SUMS.txt                         # Linux
shasum -a 256 -c SHA256SUMS.txt                     # macOS

Windows also marks anything downloaded, which is what raises the prompt.
Unblock-File .\gserp.exe clears it, once, per file.

macOS keeps downloaded programs quarantined in the same way. If it refuses to
open the file, clear the mark once with xattr -d com.apple.quarantine gserp in
the unpacked folder.

What the program does

• Four kinds of job: parsing, position check, index check and search
  suggestions. Organic results, ads and related queries, each exportable on
  its own, laid out field by field.
• Browser interface in English and Russian, light or dark, with a walk through
  it for whoever has just installed it.
• Proxy profiles: named sets of exits — a file, a URL, or the VPN gateways a
  BlankTrail subscription carries — optionally behind a first hop, with a job
  naming the one it goes out through.
• Sessions of its own: Google identities kept with their cookies and their
  address, rested between requests and written down as they go.
• Exports as CSV, JSON Lines and TXT; deduplication by URL or by host.
• HTTP API under /api/v1/, and a SerpApi-compatible GET /search a program
  written against that service reaches by changing the base address alone.

Measured on a live fifteen-thousand-address list with result addresses kept:
about 1100 pages a minute at a hundred threads, 30–40 captchas per thousand
pages once the start's wave of checks has passed. The start is the slow part — every session that rested between jobs is
checked again on its first request — and that is the shape to expect.

Full documentation: README
· по-русски

MIT licence. Provided as is, without warranty; observing the terms of service
of the sites you point it at, and the law where you are, is yours.

gserp 0.4.30

Draft. Write the notes here, then publish.

gserp 0.4.29

Draft. Write the notes here, then publish.

gserp 0.4.28

Draft. Write the notes here, then publish.

gserp 0.4.27

Draft. Write the notes here, then publish.

gserp 0.4.26

Draft. Write the notes here, then publish.

gserp 0.4.25

Draft. Write the notes here, then publish.