v0.2.0
First release verified against the live site. A real run of playwright_scraper.py on the README's own known-good category URL, from a residential IP with no proxy, no API key and no paid product, returned 96 products and cleared Akamai silently — confirming the README's central claim, which every earlier release had only fixture-level evidence for. The documented geo-redirect was confirmed at the same time (.com/shopping → .com/de/shopping, EUR, localised titles), as were the compounded-discount figures the tests were built against.
Changed
- Output is now sixteen columns, not fifteen:
Productgainedprice_source. Anything parsing the CSV header or asserting a column count needs updating. - A JSON-LD offer with no
priceCurrencynow yieldscurrency: nullinstead of a guessed"USD", and the DOM discount-overlay no longer overwrites a currency the structured data stated explicitly.
Added
price_source— says how much to trustprice:jsonld+dom(the rendered tile was found and reconciled — trustworthy),jsonld(structured data only; the tile was missing or disagreed, so on a discounted item this may be the pre-promo price), ordom(CSS fallback, no cross-check). The same column previously held all three with no way to tell them apart.- Run metadata sidecar — every run that writes output also writes
<out>.meta.jsonwithstatus(complete/partial/failed),stop_reason, pages requested vs. completed, and the start/final URL. A failed run writes no sidecar, so it can't contradict the previous run's still-intact output. - Exit code 6 for a partial run — a timeout or challenge partway through pagination still saves what it gathered but no longer looks identical to a complete run. The site's own pagination running out still exits 0: there was nothing more to fetch.
- 3-letter ISO currency codes in the fallback parser (
AED 100,100 CHF), matched against an allowlist of real ISO 4217 codes so a size chart (XXL 100) can't become a phantom price. Such tiles previously matched nothing and were dropped as "not a product", losing every product on those locales. diff_runs.pyrefuses an assortment diff when either side's sidecar says the run was partial (products on never-fetched pages would otherwise read as delisted);--forceopts out. It also reports asource_changedbucket for a price that differs alongsideprice_source— two snapshots that rendered differently, not a site-side change — which--fail-on-changeignores.- The parser logs DOM price-confirmation coverage per run, warning below 90%.
Fixed
canary.ymlnever recorded the scraper's exit code. GitHub Actions runsrun:steps underbash -e, so a non-zero exit aborted the script before the line that captured it — leaving the entire exit-3-vs-4-vs-124 interpretation dead in exactly the cases it existed for. The canary now also sanity-checks the data it fetched (≥10 products, ≥90% with a non-null price) rather than treating "exit 0" as success.
See CHANGELOG.md for the full list and README.md for usage.