v0.1.0 — justjoin.it scraper

First release. Reads justjoin.it, the Polish IT job
board, through Playwright, Selenium, Puppeteer or the 2Captcha Scraping
Browser API over CDP.

pip install -r requirements.txt -r requirements-playwright.txt
python3 -m playwright install chromium
python3 playwright_scraper.py --category python --pages 5 --format csv

That command needs no account, no key and no proxy.

Three modes

• --mode listings (default) — the board through justjoin.it's own
  offers endpoint, with offset pagination and nine filters measured to
  change what the site sends.
• --mode offer — one offer in full: the description body, per-skill
  levels, the company website and size, the country. Enumerated from the
  site's own sitemap (10,646 offer URLs), from --url, or from a previous
  run's output via --slugs-file.
• --mode facets — the board's own taxonomy with its own counts, in a
  second row schema. These counts are not capped, which is what makes
  them the tool for planning runs that reach past the per-query ceiling.

What running it found

All measured 2026-09-18 from a netcup VPS in Nuremberg (AS197540).

Nothing on the read path is gated. HTTP 200 to curl with no headers at
all, to python-requests, to wget and to a headless Chromium. Sixteen
candidate bot-challenge markers counted at zero across seven captures. One
exception, and it is a denylist entry rather than a gate: a
Python-urllib/* User-Agent gets HTTP 403.

The captcha is reCAPTCHA v2 — on the application form, and on no route
this scraper reads. The site publishes its own key
(googleRecaptchaV2Key), and the only string referencing it lives in the
job-application form's i18n namespace. In a live browser window.grecaptcha
was undefined on a rendered listing, an offer page and the endpoint alike:
the script is never even loaded on the read path.

The endpoint the front end calls is not the host the page config names.
api.justjoin.it — the site's own baseApiUrl — answers HTTP 503 from
nginx to everything. The route that works is the same-origin proxy at
justjoin.it/api/candidate-api/.

The salary is not employmentTypes[0]. The employer's own figure
carries currencySource: "original" and sits elsewhere on 321 of 1,000
listing rows and 29 of 40 detail records. On the detail route 16 of those 40
would get a wrong number, not just a wrong currency — 39.15 CHF printed
for a job offering 180 PLN.

from is the rate normalised to a month while unit names the
employer's unit. Reading the two adjacent keys together — the obvious
thing to do — says "18,480 PLN per hour" for a job paying 110 an hour, on
307 of 582 quoted entries. The scalar columns carry the quoted rate; the
monthly figure lives in columns that say salary_monthly_*.

?page=2 on a rendered listing returns page 1 again under HTTP 200, so
page_url() refuses to invent one and --route ssr plans a single page.

Every query is capped at 10,000 while the board held 19,381, and
from=10000 is an HTTP 500 rather than an empty page. The sidecar records
the whole arithmetic, because "complete" and "exhaustive" are different
words.

The default ordering is not a paid placement — checked rather than
assumed. isSuperOffer was spread through the result (9–34 per hundred at
five offsets) rather than banked at the top, so the scraper keeps the site's
own default sort. The sort is recorded, because on a capped result set it
decides which offers are in the file: two orderings of one query overlapped
on 5 of their first 100 ids.

Two defects fixed in the family core

Both inherited, both found by running the code rather than reading it.

• The two parity engines called mask(pool.current) unguarded in the
  block-retry path while the Playwright engine guarded it, so any retryable
  state on a run without --proxy died with AttributeError instead of
  retrying.
• scraper_api_client.py could not import at all — it called a parser entry
  point that had been renamed, so --help itself was an ImportError.
  Caught by the suite's signature-binding check.

Verified

All three engines produce byte-identical rows and identical sidecars for
the same command. 573 offline checks, and a daily canary that runs a real
three-page scrape from a bare GitHub runner with no secrets — expected to be
green, because that is what keeps the README's central claim honest.