v0.3.0

Proxy rotation. --proxy was a single static string applied once at browser launch and never changed — the shape of a demo, not of the thing proxies are bought for. This release adds the pattern that makes a pool worth holding.

cat > exits.txt <<'LIST'
# one proxy URL per line; blanks and # comments ignored
http://ACCOUNT:PASSWORD@HOST:9999
http://ACCOUNT:PASSWORD@HOST:10000
LIST

python3 playwright_scraper.py --url "$URL" --pages 20 \
  --proxy-file exits.txt --proxy-rotate per-page --proxy-shuffle

Flag	Default	Description
--proxy-file	–	One proxy URL per line. Wins over --proxy, and logs that it did.
--proxy-rotate	per-run	per-run: one exit per run. per-page: a new exit every page.
--proxy-shuffle	off	Don't have concurrent runs all start on the first line.
--proxy-block-retries	2	Retry a challenged page from this many other exits before giving up.

Three decisions where the cheap version would have been wrong

A rotation relaunches the browser. Swapping the proxy under a live session would be cheaper and wrong: cookies a bot manager issued against exit A, replayed from exit B, are a stronger signal than either address alone. Each exit gets a genuinely fresh cookie jar and storage. per-page therefore costs a browser start per page, and per-run stays the default because a session that changes address mid-flight is itself suspicious.

An unusable exit rotates rather than burning retries. Found live against an unreachable proxy: Chromium reports this as a PWError (net::ERR_PROXY_CONNECTION_FAILED), not a PWTimeout — so before this it escaped as an unhandled traceback, which is the likeliest failure the first time anyone points --proxy-file at a real list. A timeout deserves another try at the same exit; a dead proxy deserves a different one.

Credentials never reach server. Playwright passes that string to Chromium as a command-line switch, so a user:pass left in it would sit in the browser's argv for anything that can run ps. They go in the username/password fields instead. Logs mask credentials but keep host and port — which exit a run used is the point of the log, and is not the secret.

A malformed proxy list is rejected at load with the offending line named and exit 2, rather than surfacing as a connection failure on page 1 with nothing pointing at the cause.

Scope

Rotation is Playwright-only; --proxy still works on every engine. Both are ignored with --cdp-endpoint, where the remote browser brings its own exit.

Verified

183 offline checks (17 new, covering validation, credential masking, argv safety, wrap-around rotation, and proxy-error-vs-timeout classification). Live: the no-proxy path unchanged at 174 products over 2 pages; a deliberately dead 2-exit pool produced the full intended sequence — unusable exit → rotate → unusable exit → rotate → clean exit 4, no traceback, credentials masked throughout. A successful run through a working proxy was not tested — no live proxy credentials were available.

See CHANGELOG.md and the Proxies section.