Skip to content

v0.1.0

Choose a tag to compare

View all tags
@jehrr jehrr released this 07 Sep 10:47
· 24 commits to main since this release
v0.1.0
9d9429d
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

First tagged release.

Engines: Playwright (recommended), Selenium, pyppeteer, and a browserless HTTP client via the 2Captcha Scraper API — same CLI, same parsing core, same JSON/CSV output.

Parsing: JSON-LD primary, CSS + URL-pattern fallback. Two-stage discount correction (tile prices overlaid on JSON-LD) recovers original_price/discount_pct on discounted listings.

Optional 2Captcha integrations: captcha solving (reCAPTCHA v3/v2, auto-reconciled against the site's own widget), the Scraping Browser API over --cdp-endpoint, proxies, and fingerprints — all optional, none required to run against a listing page.

Since the initial commit:

  • Cross-page dedup by sku (dedupe_by_sku in output_writer.py), wired into all three browser engines' pagination loop.
  • diff_runs.py — diffs two JSON outputs by sku (added/removed/changed), for scheduled price/assortment monitoring.
  • .github/workflows/canary.yml — a daily live run against farfetch.com, since the main tests workflow is deliberately offline-only.
  • pyproject.toml (installable via pip install .[playwright] etc.), tests/test_smoke.py (pytest entry point), Dockerfile/.dockerignore.

See README.md for full usage, flags, and exit codes.

Assets 2
Loading