v0.1.0 — rebuilt on the 2scraper template

This replaces the previous scraper entirely. If you were using the old
requests + lxml code (main.py, scraper.py, parser.py,
marketplace.py, output.py), nothing carries over: the CLI, the module
layout and the output format are all different. There is no upgrade path and
no compatibility shim — pin the previous commit if you depend on it.

Two changes to be aware of before you script against this:
--2captcha-key is now --twocaptcha-key, and a key never belongs on a
command line anyway (put it in .env); and a run that finds nothing now
writes nothing and exits 4, instead of overwriting your last good output
with an empty file. Pass --allow-empty if an empty result is the answer you
want recorded.

First release on the architecture shared across this family of scrapers.

What it does

Mode	Reads
--mode listing (default)	Search results (/s?k=) and best-seller grids (/zgbs/), paginated
--mode product	One /dp/{ASIN} page — brand, seller, bullets, images, availability
--mode reviews	The reviews Amazon renders to a visitor with no account (~13 per product)

All 21 marketplaces, chosen by the hostname of --url — there is no
--marketplace flag, so a flag and a URL cannot disagree about which site a
run is reading.

Four engines: Playwright (primary), Selenium, pyppeteer, and a
browserless client for the 2Captcha Scraper API. They share the page-state
policy (page_flow.py) and the status/exit-code mapping
(output_writer.finish_run), so they cannot drift on what a page means or
what a run reports.

It works with no key, no proxy and no account

Measured 2026-09-08 from a residential European exit, plain local Chromium:

$ python3 playwright_scraper.py \
    --url "https://www.amazon.com/s?k=bluetooth+headphones&i=electronics" --pages 3

Parsed 30 rows from page 1.   Price coverage on page 1: 24/30 (80%).
Parsed 30 rows from page 2.   Price coverage on page 2: 22/30 (73%).
Parsed 30 rows from page 3.   Price coverage on page 3: 25/30 (83%).
Saved 77 products -> amazon_products.json     # 90 rows, 13 duplicates dropped
Wrote run metadata -> amazon_products.meta.json (status=complete)

Amazon's front door met that run twice and neither time cost anything: an AWS
WAF JavaScript challenge (a browser clears it in 3.9 s by itself) and a 503
throttle (a retry cleared it). The README lists what the paid products actually
buy on top of that.

Amazon answers a request five ways, and four want different responses

Signal	Response
HTTP 202 + awswaf.com/challenge.js	Wait. Not a block, not solvable, unbilled
HTTP 503 "Sorry" / "Tut uns Leid"	Retry, then a different exit
/errors/validateCaptcha	The only path a 2Captcha key is for
/ap/signin in the final URL	Stop — no proxy or solve changes it
anything else	Parse it

Collapsing those into one "blocked" is how a run either gives up on a page it
would have got, or pays for a challenge no solver can answer.

Notable in the parser

• No JSON-LD path, because Amazon publishes none — 0
  application/ld+json blocks on search pages, best-seller grids and product
  pages. Rows come from data-asin, with a /dp/{ASIN} URL-pattern fallback.
• The url column is rebuilt as https://www.{host}/dp/{ASIN}: a
  sponsored tile's link is a click tracker with the product path
  percent-encoded inside it, so reading hrefs both misses sponsored products
  and stores trackers.
• Currency follows the exit IP, not the domain. From a European exit,
  amazon.co.uk and amazon.co.jp both quote EUR — and the converted price drifts
  with the exchange rate, which is what diff_runs.py --price-tolerance-pct
  exists for (default 0, so nothing is swallowed silently).

Honest gaps

• The image-captcha solve path is unproven against the live site. 47
  navigations across five marketplaces produced WAF challenges and 503s and
  never an image captcha, so it is exercised only by an offline fixture written
  from Amazon's documented markup.
• scraper_api_client.py is untested against amazon.com — no Scraper API
  subscription was available.
• brand is null on every listing row (Amazon's search markup has no
  brand field), and there is no prime column (no Prime marker appeared on
  any tile captured from a cross-border exit, so it would have been null on
  every row).
• --mode reviews has no --pages: /product-reviews/{ASIN} redirects to
  /ap/signin on page 1, so the paginated history is not available to an
  anonymous visitor at all.

Testing

239 offline checks (python3 smoke_test.py) with fixtures cut from real
captures and verified to parse identically to the untrimmed originals; a daily
canary against a real 3-page listing with floors set from the measurements
above; tests.yml on Python 3.9 and 3.12, plus engine-smoke as one venv per
engine so each is tested at the version users actually install.

Full detail, including the nine bugs that only a live run found, is in
CHANGELOG.md
and PR #1.