Skip to content

v0.1.0 — rewrite as a member of the 2scraper family

Choose a tag to compare

View all tags
@jehrr jehrr released this 10 Sep 08:40
· 10 commits to main since this release
v0.1.0
6704d17
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

The first release of etsy-scraper in its current form: a rewrite that replaced the three standalone scripts with the shared architecture the rest of the family uses.

Tagged retroactively at the commit it describes, so the CHANGELOG's link to this tag resolves. For what is current, see v0.2.0.

What it does

Three modes over one output contract:

mode reads anchor
listing search results, category pages, shop fronts data-listing-id
product a listing's own page JSON-LD Product
shop a shop front plus its facts both

Sponsored placements are flagged rather than dropped: the ls=a URL parameter agreed with the visible label on 400 of 400 tiles across two languages, which is what makes it a structural signal instead of one that happens to work on one locale.

The parser inverts the family's usual order

Its siblings read JSON-LD first and fall back to the DOM. Etsy publishes an ItemList that is partial — 8 of 64 listings on a search page, 61 of 65 on a category page, 36 of 40 on a shop front — so structured data is the enrichment here and the DOM tile is primary. Trusting the ItemList would have silently dropped 88% of a search page.

Access

Etsy sits behind DataDome, and the repo says plainly what does and does not get in. Measured 2026-09-09, live:

approach result
curl from a hosting ASN HTTP 403
headless Chromium, hosting ASN HTTP 403
real Chrome, hosting ASN HTTP 403
headless Chromium via a residential exit HTTP 403
Scraping Browser API, fresh profile HTTP 200, 64 listings

The fourth row is the one worth reading twice: a residential proxy alone was not enough for an automated browser here.

Also in this release

  • Exit codes 0/1/2/3/4/5/6, produced identically by all three engines through a shared finish_run()
  • A <out>.meta.json sidecar per successful run, recording which pages failed by number
  • 27 locale prefixes with their currencies, taken from Etsy's own hreflang set
  • Three engines, one behaviour — though only two reach Etsy; see the README's table for why Selenium cannot
Assets 2
Loading