Skip to content

v0.1.0 — Maison KOSÉ

Choose a tag to compare

@jehrr jehrr released this 18 Sep 11:40
· 13 commits to main since this release

First release. Reads Maison KOSÉ (maison.kose.co.jp), Kosé's own online store.

The domain this repo is named for does not resolve. kose.com is registered (GoDaddy, 1998, paid to 2029) and delegated to two nameservers that answer REFUSED for the zone, so 8.8.8.8 and 1.1.1.1 both return SERVFAIL and no client anywhere gets an address. That is a broken delegation, not a bot manager. Kosé's storefront is maison.kose.co.jp — where www.kose.co.jp sends a visitor with its own meta refresh — and that is what this scraper reads.

You need no account

Measured 2026-09-18 from a datacentre address, and re-measured the same day from a bare GitHub runner by the canary in this release:

request result
curl/8.5.0 HTTP 200, 90,310 bytes
python-requests/2.31.0 identical
a Chrome User-Agent identical
no User-Agent header at all identical
12 pages back to back, no delay 12 × HTTP 200

Zero captcha markers of any vendor across 13 captures. The solver, proxy and fingerprint paths are all wired and all verified working — they simply buy nothing here today. Nothing in this repo claims a captcha on this site could not be solved; there is none to solve.

What it does

Two modes over three routes — category listings, tag facets (which cross brands) and product pages — through four engines. All four were run live; the three browser engines returned 48 of 48 identical rows on the same two pages.

216 rows over four live runs, with title, url, sku, brand, price, currency, in_stock, image_url and tax_rate on 216 of 216. Prices ¥495 – ¥264,000.

Site traps it handles

  • The brand segment in every URL is decorative. /site/xyz/c/c15/ returns byte-identical bytes to /site/cosmedecorte/c/c15/, so brand is read per tile — one tag run mixed 11 brands across 72 rows.
  • A category listing walked past its end serves its LAST PAGE AGAIN, byte-identical and HTTP 200, rather than 404ing or emptying. Runs plan against the site's own N/Mページ counter and still stop on "no new sku".
  • &pageno= and &page= are silently ignored on a tag listing; only ?p= paginates it.
  • 5 of 20 product pages canonicalise to a DIFFERENT sku — /g/gJQBA/ is the 60 g cream and points at the 20 g /g/gJQGA/. A row's URL is built from the page's own sku; the canonical goes in variant_of.
  • A product page reuses the listing's tile class on <div>s for staff-review cards, 15 on one measured page. The tile selector is element-qualified and sees zero.
  • Japan charges two consumption-tax rates and prices here are tax-inclusive, so tax_rate is a column: 2 of 171 tiles carried the reduced 8%.

Known limitation

--cdp-endpoint (the Scraping Browser API) is not live-verified in this repo — the profile available when this was built had expired, and those credentials live about a day. Every other path was run live.

423 offline checks; the canary runs a real 3-page scrape daily from a bare runner with no secrets, because that is the README's central claim.