Skip to content

Releases: 2scraper/spinny-scraper

v0.1.1 — fingerprint_client reads through the loader

Choose a tag to compare

@jehrr jehrr released this 11 Sep 20:22

A one-defect patch, converging this repo onto the shape the whole family now
shares.

The .env half was already fixed in 0.1.0 — found the first time
--fingerprint was run live here — but with env_config.apply() after
parsing, which leaves --key's argparse default reading os.environ
directly and so skips the placeholder rule on the exported-variable path.

Measured with TWOCAPTCHA_KEY=your_2captcha_api_key_here exported:

What the user saw
before Fingerprint API rejected the key (401) — note this is a separate subscription from captcha solving
after TWOCAPTCHA_KEY is still set to the placeholder from .env.example — treating it as unset

The old message sent the reader off to check a subscription they never
needed.

The same defect, in its plainer .env form, was found in five sibling repos
in the same pass and fixed there — farfetch
0.4.3
,
amazon 0.1.3,
mediamarkt 0.1.8,
etsy 0.2.2,
tokopedia 0.1.4.
The check that pins it is byte-identical in all seven and was verified to
fail on the old code.

Nothing else changed. No flag, no default, no output column.

v0.1.0 — first release as a 2scraper-family scraper

Choose a tag to compare

@jehrr jehrr released this 11 Sep 14:25

Replaces the April 2026 prototype entirely. The two share no data
contract: prices are numbers, spinny_id is sku, kilometers is
km_driven, there are exit codes and a run sidecar, and --mode api is
gone. If anything consumed the old output it needs rewriting rather than
adjusting — see the CHANGELOG's "Replaced, not extended".

Scrapes Spinny used-car city listings, filtered listings and car detail pages
through Playwright, Selenium or pyppeteer, with JSON/CSV output, a
run-metadata sidecar and this family's exit-code contract. 627 offline
checks
, two CI workflows, and every number in the README measured and
dated.

The one thing to understand: --pages scrolls

A Spinny listing has exactly one page. ?page=N is silently ignored —
pages 1, 2 and 3 of one URL return byte-identical first cards under HTTP
200 — and there is no rel=next, no numbered anchor and no "load more"
anywhere. So --pages N means N batches of 20, reached by scrolling, and
--concurrency above 1 is refused with that reason in every engine.

A planner that built ?page=N would have fetched page one N times, found no
new car id, concluded the listing was exhausted, and reported a complete
run holding a twentieth of the catalogue.

You need nothing paid

Measured 2026-09-11: an ordinary local Chromium on a home connection, no API
key and no proxy, returned 62 cars at 100% price coverage. No refusal of
any kind was observed on this site — from that connection, an Amsterdam
datacentre exit, a Chennai residential exit, a bare GitHub runner or the
Scraping Browser API. The canary therefore runs daily with no credential,
unlike every sibling repo's; its first dispatch returned 82 cars.

Live-verified

  • All three engines return the same 62 car ids with zero differing
    columns
    .
  • The luxury listing returns 40 of the 40 cars the site advertises.
  • --mode detail joins its listing row on 15 columns, and adds the exact
    odometer, the owner count, the colour, the seating capacity and the
    insurance details.
  • Every paid path was run: Scraping Browser API (62 cars), residential proxy
    on an Indian exit (62 cars), Fingerprint API (applied live), Scraper API
    (200, 1.87 MB, 0 cars — by construction, the grid is not in the first
    response).

Findings worth knowing

  • A listing page carries no structured data about its cars — five JSON-LD
    blocks and not one names a vehicle. A detail page does, and spells the
    price offers.Price, with a capital P.
  • Four prices per car, on two bases. Mixing them computes a plausible
    negative discount, so every run reports the all-in-versus-displayed share.
  • reCAPTCHA Enterprise v3 loads on every page and challenges nothing, so
    recaptcha and g-recaptcha were removed from the inherited marker set: a
    marker that matches every page of the site it guards is worse than no
    marker.
  • A listing URL with no city renders nothing while advertising the
    national total — /used-suv-cars/s/ says "2549 Used SUV cars in India" and
    serves none of them.

Defects the live runs found, each now pinned by a check

fingerprint_client could not read .env; a --proxy the run was about to
ignore could still end the run; the readiness wait reported a timeout on a
page that had painted (off by one, all three engines); pages_completed
disagreed between engines on identical runs; and a stalled lazy load reported
a complete run holding 22 of an advertised 1546 — now scroll_stalled,
status partial, exit 6.

Full detail in CHANGELOG.md.