v0.2.2 — fingerprint_client reads .env

A one-defect patch, and the defect is that a documented mechanism was not
applied on one path.

fingerprint_client.py's --key defaulted to
os.environ.get("TWOCAPTCHA_KEY") and only that. So a key put in .env —
exactly as §3, the README and .env.example instruct — worked for every
engine and failed here with "No API key".

It now reads through env_config.env_value, calling load_env() itself
because this is a standalone entry point that no engine has necessarily run
first.

Why the loader and not os.environ

Measured both ways with TWOCAPTCHA_KEY=your_2captcha_api_key_here exported:

	What the user saw
before	Fingerprint API rejected the key (401) — note this is a separate subscription from captcha solving
after	TWOCAPTCHA_KEY is still set to the placeholder from .env.example — treating it as unset

The old message sent the reader off to check a subscription they never
needed.

Provenance

Found on a sibling repo's first live --fingerprint run, then checked
across the family before patching: five repos had it, one had already fixed
it. This release makes all seven identical.

Pinned by a check verified to fail on the old code — that it loads .env
itself, reads through the loader, does not read os.environ directly,
that a placeholder still reads as unset, and that the help string does not
interpolate its default, which is one substring away from printing a live
credential to anyone who types --help.

Nothing else changed. No flag, no default, no output column.

v0.2.1 — the fingerprint's device scale factor is applied

If you use --fingerprint, this one changes what the browser reports.

playwright_context_kwargs mapped the user agent, the locale, the timezone
and the screen onto the browser context and ignored deviceScaleFactor,
which the fingerprint API returns beside them.

Measured 2026-09-11 against the live API and a live browser:

	fingerprint says	browser reported, before
user agent	Chrome/146 on Windows	✅ same
locale	nl-NL	✅ same
timezone	Europe/Amsterdam	✅ same
screen width	1536	✅ same
devicePixelRatio	1.25	1 ❌

That is the mismatch the flag exists to prevent, on an axis any
fingerprinter reads for free and for nothing: the paid identity said one
thing and the browser said another, on every run, silently.

Playwright takes it as its own context option, so the fix is to pass it.
Verified in a live browser both ways and pinned in the offline suite.

Found while auditing a new sibling repo against the family notes — all
five repos in this family had it.

🤖 Generated with Claude Code

v0.2.0 — DataDome fallback, a second storefront, and two checks that were reporting more than they did

Two changes an existing user will notice

diff_runs.py now refuses a cross-currency comparison. Diffing a US run against a European one previously produced a diff — and reported every row as changed, at a constant 0.935 ratio that was the exchange rate rather than the seller, exiting 0. --fail-on-change would have fired on a catalogue that had not moved. If you diff two storefronts in a pipeline, that pipeline now stops with a message. --force restores the old behaviour.

The canary no longer runs on a schedule. It is workflow_dispatch only, so its badge reports the last run someone asked for rather than last night's.

The DataDome fallback, wired — and measured

The Scraping Browser API remains the primary path. When it is not available, DataDomeSliderTask is now a real fallback rather than a mention in a README.

What the wiring cost, and why it is opt-in: the solve requires a proxy, and it is only worth paying for a t=fe challenge — t=bv means the address is banned and no solve will clear it. The engines read that verdict from the live frame URLs, not from the page HTML, and that distinction is the whole feature. The top-level dd object holds rt='i' for two minutes or more while the iframe has already moved to t=fe at around seven seconds; a detector reading only the HTML never sees a solvable challenge and the paid path is unreachable.

Solving is off unless asked for. The default spends nothing.

The second storefront, verified

The family notes ask for a second country site. The US storefront is now captured and pinned, and it turned up a guard the family's usual protection could not provide: the siblings catch a cross-storefront diff through source, which holds a country hostname — but Etsy is one host, so source was etsy.com on both sides and the protection silently did not apply.

Same shop, two exits:

	US exit	German exit
currency	USD 39/39	EUR 39/39
the same mug	$33.50	€35.84
shop_location	Wooster, Ohio	Ohio, Vereinigte Staaten
titles matching	—	35 of 39

sku, brand, shop_id, ships_from, rating and in_stock were identical, as they must be. Etsy translates some listing titles.

Fixed — checks that reported more than they did

The fixture privacy check examined an empty string. It collected its corpus by a FIX_ prefix while every fixture in the suite is named by suffix, so twelve patterns — JWTs, session ids, click-tracking keys, DataDome blobs — ran against "". Twelve checks reported green over 150 KB of committed real captures that nothing had read.

The captures are clean; with the collection fixed all twelve still pass, now over 150,487 characters. What changed is that we know it, and an assertion underneath now fails if the corpus is ever empty again.

The banned-wording scan reached only the repo root, leaving both Claude workflows, tests.yml, canary.yml and the four issue templates unscanned.

A proxy password could reach a log. captcha_solver._redact masked key= query parameters but not credentials in a URL's userinfo — and this module now takes a proxy, so its errors can quote one. Masking is global and keeps host and port, because which exit failed is the useful half.

The credential check was dead code, was failing, and was weaker than the workflow's own grep. One implementation now, and CI runs it.

The canary failed on any non-zero exit, so a blocked run or a refused endpoint — access conditions that test nothing about this code — reported the same red as a real defect. They are warnings now, with "the canary did NOT test anything" written into the run summary. Red is reserved for a crash, bad usage, or the case worth having a canary for: it got in and parsed zero rows.

Checks

571 offline checks, 0 failures. Docker image built and run in CI. Three engine venvs, each pip check-clean.

v0.1.0 — rewrite as a member of the 2scraper family

The first release of etsy-scraper in its current form: a rewrite that replaced the three standalone scripts with the shared architecture the rest of the family uses.

Tagged retroactively at the commit it describes, so the CHANGELOG's link to this tag resolves. For what is current, see v0.2.0.

What it does

Three modes over one output contract:

mode	reads	anchor
listing	search results, category pages, shop fronts	data-listing-id
product	a listing's own page	JSON-LD Product
shop	a shop front plus its facts	both

Sponsored placements are flagged rather than dropped: the ls=a URL parameter agreed with the visible label on 400 of 400 tiles across two languages, which is what makes it a structural signal instead of one that happens to work on one locale.

The parser inverts the family's usual order

Its siblings read JSON-LD first and fall back to the DOM. Etsy publishes an ItemList that is partial — 8 of 64 listings on a search page, 61 of 65 on a category page, 36 of 40 on a shop front — so structured data is the enrichment here and the DOM tile is primary. Trusting the ItemList would have silently dropped 88% of a search page.

Access

Etsy sits behind DataDome, and the repo says plainly what does and does not get in. Measured 2026-09-09, live:

approach	result
curl from a hosting ASN	HTTP 403
headless Chromium, hosting ASN	HTTP 403
real Chrome, hosting ASN	HTTP 403
headless Chromium via a residential exit	HTTP 403
Scraping Browser API, fresh profile	HTTP 200, 64 listings

The fourth row is the one worth reading twice: a residential proxy alone was not enough for an automated browser here.

Also in this release

• Exit codes 0/1/2/3/4/5/6, produced identically by all three engines through a shared finish_run()
• A <out>.meta.json sidecar per successful run, recording which pages failed by number
• 27 locale prefixes with their currencies, taken from Etsy's own hreflang set
• Three engines, one behaviour — though only two reach Etsy; see the README's table for why Selenium cannot