Repository navigation
Behaviour changes for existing users.
--route ssrnow refuses, with
exit 2, any filter other than category, city, remote and keyword (it used
to accept them and ignore them).diff_runs.pynow refuses a run with no
sidecar unless--forceis passed.--retries 0is refused.CSV output now prefixes a cell that starts with
=,+,-,@, a tab, CR or LF with an apostrophe (JSON is
unchanged), and the sidecar gainscsv_cells_escaped..env.example
no longer setsJUSTJOIN_URL; aJUSTJOIN_URLfrom the environment that
names a different mode than--modeis now ignored with a log line
instead of exiting 2.
Fixed
-
An offer queue no longer ends at the first delisted offer (all three
engines). A 404 on one offer was read as the end of the run. -
--mode offerworks in Selenium and pyppeteer, and pyppeteer no
longer crashes without a proxy pool. -
--route ssrno longer accepts filters it ignores, and no longer
records a sort, order or total the page does not carry. -
--mode facetsis partial when a group is missing, and the parity
engines now request the categories. -
diff_runs.pyrefuses runs without a sidecar, and tells an absent
filterskey from an empty one. Engines always recordfilters. -
The Scraper API client writes a sidecar and picks the row class by
mode (its facets CSV had the wrong header). -
Output writes are atomic. A failed write no longer replaces the
previous good JSON, CSV or sidecar with a truncated one. -
CSV cells are formula-escaped. Every string in a row is written by an
employer, and a spreadsheet runs a cell starting with=,+,-or
@as a formula. Escaped in the CSV only, after list cells are joined;
numbers stay numbers and JSON keeps the site's bytes. The count goes in
the sidecar ascsv_cells_escaped, under any caller-provided field of
the same name. -
JUSTJOIN_URLno longer breaks--mode offerand--mode facets.
The listings URL that.env.exampleshipped was read as if--urlhad
been typed, so a copied example made both modes exit 2. The engines now
record what was typed before reading the environment; only a typed URL is
checked against--mode. The example now shows the variable commented
out. -
diff_runs.pysees changes in a facets run. It compared the job
columns on facet rows, which have none of them, so anoffer_count
going from 100 to 250 reported "0 changed". Facets runs are now compared
ontitle,offer_count,is_cumulativeandoverlaps_other_keys. -
The credential scan works from any parent directory. Its skip list
was matched against the absolute path, so a clone under a directory
namedbuild,envorvenvscanned 0 files and reported clean. It now
matches the path inside the repo, and the suite runs the scan from such a
copy with a planted key. -
--dump-htmlhelp pointed at aTROUBLESHOOTING.mdthis repo does not
have; it now points at the README section on traps that look like bugs. -
Engine comments said the paid captcha path runs only for a
challenge
state that does not exist here; they now describe what
page_flow.STATE_POLICYdoes. -
.gitignorecovers--dump-htmlpage files of any extension
(*.page[0-9]*), alive/scratch directory and HTML dumps at the root. -
CI tests Python 3.13, which the README badge already claimed.
-
Comments, docstrings and changelog entries no longer cite internal notes
that are not public; each rule is stated in place. A suite check keeps it
that way. -
Scraper API:
waitForis now sent as a JSON object. Measured
2026-09-23 againstscraper.2captcha.com/tasks/sync: the JSON-encoded
string this client sent (the form older docs described) is refused with
HTTP 422 "params.waitFor must be an object" -- and the task is still
billed ($0.0005) -- so every run with--wait-textor its sibling wait flags
failed with exit 5. The object form answers HTTP 200. -
Scraper API: the target page's status is read from
http_code. The
response'sstatusfield is the API's own verdict string ("success"),
not the target site's HTTP code, so a target 403/503 was never seen by
this client.http_code(an int) is read now, withstatuskept as a
fallback only when it is an int.
Leftovers from the repos this one was bootstrapped from, each describing
another site as if it were this one:
-
CONTRIBUTING.mddescribed Mercor's sources, markers, captcha, live-run
notes and scope (and Wellfound's pagination and pay strings). Rewritten
for justjoin.it from this repo's own README measurements. -
SECURITY.mdasked for "mercor-scraper security" in the subject line and
named Mercor in the out-of-scope list; it also said the project has no
releases, and pointed at a README section that does not exist. -
The issue templates were Etsy's: DataDome advice,
etsy.comexample
URLs, a JSON-LD evidence recipe for a site whose parser reads JSON, and
links to aTROUBLESHOOTING.mdthis repo does not have. -
requirements.txtwas headed# mercor-scraper. -
.gitignoreand.dockerignoreignoredwellfound_jobs.*instead of
this repo's own default output prefixjustjoin_jobs. -
smoke_test.py: the sidecar check used Mercor run metadata
(mercor.com,/explore, 390 records); it now uses a real
--route ssrsidecar from this repo. The suite's--helpdescription
said "mercor-scraper offline suite", and three comments described
Mercor's routes as this site's. -
captcha_solver.py's docstring pointed at a "No DataDome solver" section
that does not exist in this repo (it came with the copied core). Removed.