v0.2.0

Behaviour changes for scripts.

• A search the site answers with its own "nothing matched"
  (search-count:0) now exits 4 (no_results, an empty result) in all
  three browser engines. It exited 5 ("never obtained"), which said the
  opposite of what happened.
• A --concurrency run in which some planned pages were never fetched by
  any worker now ends partial (exit 6), listing them in pages_failed
  and the sidecar's pages_unattempted. It used to report complete and
  exit 0 with those pages missing.
• The Selenium and pyppeteer engines now exit 5 when every navigation
  failed (a dead network, a timeout), like the Playwright engine. They
  reported exit 3 (blocked), sending the reader after a proxy problem.
• A paid captcha solve is now bought at most once per page. With the
  default --retries 2, one gated page used to buy a token on every
  attempt (three solves); the run now stops at exit 3 after one.
• --retries, --delay and --retry-delay below 0, and --pages or
  --concurrency below 1, are now usage errors (exit 2) naming the flag,
  in all engines and scraper_api_client.py. --retries -1 used to run
  no navigation at all and report exit 3. --retries 0 (try once) is
  unchanged.
• Output files are written to a temporary file in the same directory and
  renamed into place. A new file gets the mode open() would have given it
  under your umask; an existing file keeps its mode.

Fixed

• An interrupted write (a crash, a kill, a full disk) no longer destroys the
  previous good output: out.json, out.csv and out.meta.json are each
  written atomically. Measured before the fix: an 852-byte out.json came
  back as 12 bytes of invalid JSON.
• In the Selenium and pyppeteer engines, rotating to the next proxy exit now
  relaunches the browser. --proxy-server is fixed at launch, so the old
  code advanced the pool and kept using the first exit. Both engines also
  rotate on a blocked page 1 when a pool is given, as Playwright does.
• diff_runs.py now has the source_changed bucket its documentation
  described: a price difference that comes with a price_source difference
  (the API's exact figure against a card's rounded one) is reported there and
  does not trip --fail-on-change.
• The credential scan in .github/ci_checks.py tested every ancestor of the
  checkout against its skip list, so a clone under a directory named venv
  scanned 0 files and reported clean. It now tests paths relative to the
  repository, fails when it scanned nothing, and also reads .json, .csv,
  .toml, .cfg and .sh files.
• CONTRIBUTING.md, the bug-report and site-change issue templates and the
  review workflow's prompt now describe Indiegogo (modes search,
  campaign, rewards, the inlined search payload and the
  searchProjectsForCards POST) instead of another site's pages, flags and
  modes.
• The exit-code tables in README.md and TROUBLESHOOTING.md now say what exit
  4 and 5 mean in the code: 4 is "the site answered and nothing matched", 5
  is "the content was never obtained", including a served page that parsed
  to nothing (parser_found_nothing).
• The help text and docs no longer say Scraping Browser credentials last
  "about a day"; how long one lasts varies, and a 401 is the signal to fetch
  a fresh endpoint.
• --proxy-block-retries is accepted for compatibility with the other
  scrapers in this family; its help now says it has no effect here.
• The daily canary was red from 2026-10-07 to 2026-10-12 although the run
  itself was healthy (72 rows, complete, 3 of 3 pages). Its sanity check
  rejected pct_funded <= 0, and a real campaign that raised 10,000 against
  a goal of 900,000,000 rounds to 0.0. The check now fails on a negative
  percentage, or one computed from a goal that is not positive; zero is a
  figure. The parser was right and is unchanged, with a check pinning the case.
• Scraper API: waitFor is now sent as a JSON object. Measured
  2026-09-23 against scraper.2captcha.com/tasks/sync: the JSON-encoded
  string this client sent (the form older docs described) is refused with
  HTTP 422 "params.waitFor must be an object" -- and the task is still
  billed ($0.0005) -- so every run with --wait-text or its sibling wait flags
  failed with exit 5. The object form answers HTTP 200.
• Scraper API: the target page's status is read from http_code. The
  response's status field is the API's own verdict string ("success"),
  not the target site's HTTP code, so a target 403/503 was never seen by
  this client. http_code (an int) is read now, with status kept as a
  fallback only when it is an int.
• A rendered enterprise widget went undetected. The iframe rung matched
  recaptcha/api2/anchor; this site serves recaptcha/enterprise/anchor,
  where the sitekey can be only in the frame's k= parameter.
• A sitekey inside the site's own <captcha-*> mount went undetected.
• category is localised and differed BETWEEN MODES. The same German
  campaign read Produktivität from --mode search (the API's localised
  name) and Productivity from --mode campaign (a numeric code resolved
  through an English table).
• pyproject extras had drifted from requirements-*.txt
  (playwright>=1.44 against >=1.40.0, selenium>=4.20 against
  >=4.15.0). CI installs the requirements files, so the drift was invisible.
• The offline suite's credential check scanned gitignored files and
  failed on a --dump-html capture containing a site nonce — a file that
  cannot be committed. .github/ci_checks.py had already learned this;
  the suite now matches it.
• .gitignore did not cover this repo's own default output prefixes.
• Leftovers from the repo this one was bootstrapped from. .gitignore
  listed transfermarkt-scraper's transfermarkt_products.* output prefixes
  (this repo's own indiegogo_projects.* was already listed) and described
  sample_output.* as a --mode market-values run; it is --mode search.
  tools/waf_probe.sh and tools/verify_browser_api.sh were transfermarkt's
  AWS WAF probes — transfermarkt URLs, TRANSFERMARKT_* variables that this
  repo's loader does not read, and a --mode market-values this repo does
  not have. Nothing referenced either, and this site's gate is a Cloudflare
  managed challenge rather than AWS WAF, so they are removed rather than
  ported.

Added

• CSV output prefixes any text cell that begins with =, +, -, @, a
  tab or a line break with an apostrophe, so a spreadsheet does not run it as
  a formula. Numbers are untouched, JSON keeps the site's text, and the
  sidecar's csv_cells_escaped counts the cells that differ.
• category_code on Campaign — the site's numeric, locale-independent
  catalogue id. Join on it; category is display text.
• Checks pinning all of the above, each verified to fail when its fix is
  reverted.

Changed

• The daily canary no longer skips green when the runner is blocked (exit
  3): the README's claim is that a local browser needs no key and no proxy
  here, and bare runners have been served on every scheduled run checked, so
  a block is now a failure. Its exit 4 and 5 messages match the codes above.
• scraper_api_client.py warns on stderr when the key is passed with --key
  (visible to ps, kept in shell history). TWOCAPTCHA_KEY in .env is
  unchanged and remains the documented way.

Correction to v0.1.0's release notes. They said the 2Captcha balance
was "unchanged to five decimal places". That was measured across the four
bare-browser runs and is true of them, but it is not true of the session as
a whole: the balance moved $0.0024 over the day, which is Scraping
Browser session usage from --cdp-endpoint runs. The claim that matters is
narrower and now stated that way — 2Captcha's own statistics endpoint
reports 0 captcha solves and $0.00000 for the day. A run without
--cdp-endpoint still costs nothing at all.

The site has a SECOND captcha, configured but never rendered

Asked plainly which captcha this site uses, and answered by measuring. There
are two, at different layers:

1. Cloudflare Turnstile, in a managed challenge at the edge. This is the
   one v0.1.0 documents: met, self-cleared, never solved.
2. reCAPTCHA Enterprise, wired into the application and enabled on every
   page the site serves — site key 6LeRruUr…, the enterprise loader on
   recaptcha.net (not google.com), and an empty <captcha-widgets> mount
   point. It is never shown to an anonymous reader.

"No challenge rendered" is not "no captcha configured", so the useful
question was whether this repo would RECOGNISE it. Constructing the shapes
this site's own captcha would take found two the static detector missed.

The proxy path, now exercised live

v0.1.0 shipped with --proxy untested against a real gateway -- there was
no credential to test with. There is now, and the path works, but testing it
found a defect first.

• Fixed: a pasted proxy-LIST line exited 1 (crash) instead of 2 (bad
  usage). Proxy lists are scheme://host:port:login:password; this
  expects http://login:password@host:port, and the extra colons land in
  the port. proxy_pool already validated it and raised with a message
  naming the fix -- but no engine caught ProxyError at its entry point, so
  it reached the interpreter as a traceback. proxy_pool's own comment said
  this should be exit 2; now it is, in all three engines, with the password
  absent from the message.

Measured 2026-09-17 through a 2Captcha residential gateway:

• The site served content on the FIRST response, with no challenge --
  better than the datacentre path, which met one on 2 of 4 runs.
• -region-XX is honoured: de → Munich (M-net), us → Reston (AT&T),
  gb → Glossop (Plusnet), tr → Istanbul (Türk Telekom), all residential
  ISPs. The Scraping Browser's country- segment was NOT honoured on the
  accounts tested (a country-us login exited from Turkey), which is worth
  knowing before trusting either.
• One credential mints many exits: 5 -session- logins → 5 distinct
  addresses (RU, US, US, IN, LK).
• --concurrency 3 gave each worker its own session and returned 144
  rows over 6 pages, (page, position) unique, status complete.
• A dead exit rotates rather than retrying: a pool with an unreachable
  entry first reported ERR_PROXY_CONNECTION_FAILED, rotated to the next
  exit and completed with 24 rows -- the "a proxy failure is not a timeout"
  invariant, exercised rather than asserted.
• Selenium's documented limitation confirmed: it cannot authenticate a
  proxy, says so, and then legitimately returns 39 bytes and exit 3 where
  the other two engines returned 24 rows through the same exit.
• No credential reached any log across every run above.

Verified, not assumed

• --fingerprint applies a real user agent, locale and timezone
  (de-DE/Europe/Berlin for a German fingerprint — not the en-DE bug
  this family shipped four times).
• --concurrency 3 over 6 pages returns 144 rows with unique
  (page, position) and no duplicate skus.
• /de/ and /zh/ each return 48/48 fully populated rows.
• SECURITY.md said this project has no releases or version tags; it has
  both. "Supported versions" now names the latest release and main.
• captcha_solver.py's docstring pointed at a "No DataDome solver" section
  that does not exist in this repo (it came with the copied core). Removed.