Repository navigation
v1.0.0
The headless-browser API is now fully async, headless_chrome is gone, and the only GPL
dependency is gone with it. Every fingerprint value the crate emits is now measured from a
real browser rather than transcribed or inferred.
This is a breaking release, and the first to commit to a stable API. The public enums
most likely to gain variants — Error, BrowserKind, ChallengeKind, ScraperEvent,
Action, Outcome, RotationStrategy, Ja4Error, EscalateReason, DemoteReason,
Transition — and the ClientHints struct are now #[non_exhaustive], so future additions
(Edge support, new challenge kinds, new events) will not require a 2.0. Enums closed by an
external specification (SameSite, Confidence, Transport) stay exhaustive.
⚠️ Breaking changes
-
The browser API is async and tab-free.
headless_chrome::Tabno longer appears in
any signature; a first-partyPagereplaces it.0.4 1.0 scraper.new_stealth_tab()?scraper.new_stealth_page().await?tab.navigate_to(url)?; tab.wait_until_navigated()?;page.navigate_and_wait(url, timeout).await?scraper.detect_challenge(&tab)?scraper.detect_challenge(&page).await?scraper.solve_challenge(&tab)?scraper.solve_challenge(&page).await?scraper.rotate_profile()?scraper.rotate_profile().await?CloudScraper::human_type_str(&tab, text)?CloudScraper::human_type_str(&page, text).await?CloudScraper::human_move_mouse(&tab, x, y)?CloudScraper::human_move_mouse(&page, x, y).await?GenericSolver::solve_cloudflare_turnstile(&tab)?GenericSolver::solve_cloudflare_turnstile(&page).await?Remove any
spawn_blockingwrapping these calls — it is no longer needed, and
wrapping an async call in it is worse than useless. -
BrowserProfile::random()now claims Chrome 153, not Chrome 124–126. This is a
behavioural break with no compile error: code asserting on the User-Agent string, or
pinned to a specific fingerprint, changes silently. The version is the one actually
captured and the one the launched binary reports; the old profiles advertised a browser
several years older than the one rendering the page, which feature detection alone
exposes. -
Errorgained aCdpvariant and is now#[non_exhaustive], so an exhaustive
matchon it needs a_arm. -
wreq-utilremoved, soEmulationvalues no longer reach this crate's API. Use
stealthscraper_rs::emulation::{chrome, safari_27, for_kind}instead. -
The
browserfeature no longer pullsheadless_chrome,tokio-tungstenite,
rand_distrorwreq-util. Code relying on those arriving transitively must depend on
them directly. -
MITM TLS moved to BoringSSL, so
rustls,tokio-rustls,rcgen,ringand
aws-lcare gone from the graph. Only affects consumers that relied on them
transitively; a consumer binary keeping rustls for its own clients is unaffected.
The pure domain modules (challenge, proxy_pool, geo, state, events) keep their
shape, so a no-features consumer only needs the #[non_exhaustive] note above.
Added
- Own async CDP client (
cdpmodule):LaunchConfig/launch, aCdpTransportthat
demultiplexes replies byidand events bymethod, and a typedBrowserHandle/Page
surface of ~25 methods. Chrome is launched over--remote-debugging-pipe(no scannable
TCP debug port) and without--enable-automation.Runtime,DOM,Log,Debugger
andProfilerare never enabled — asserted by test, where the previous claim to that
effect was false. - Dual-mode sessions (
StealthSession,identity,http_scraper): solve the first
challenge in a real browser, then carry the whole identity — profile, cookies (RFC 6265
scoped), client hints, locale and the same egress proxy — onto a plain HTTP transport and
shut the browser down. It re-escalates automatically on a fresh challenge or a
near-expiry clearance cookie. Steady-state scraping runs with no Chrome process. - Measured emulation table (
emulationmodule): complete Chrome and Safari emulations —
TLS, HTTP/2 SETTINGS in wire order, connection window,HEADERSpriority, pseudo-header
order, and the header set and order — every value captured from the browser it claims to
be. - JA4 measurement apparatus (
ja4,tls_capture): aClientHelloparser and JA4
computation, plus passive capture through the MITM proxy, so any browser's real
fingerprint can be read off rather than guessed. - Capture tooling:
examples/capture_fingerprint(TLS),examples/capture_h2(HTTP/2
frames, and header names as plaintext with--h1),examples/capture_hints(User-Agent
Client Hints, including a console snippet for machines with no Rust toolchain), and
examples/emulation_roundtrip(proves an entry reproduces its browser). JsonStateStore: a durableStateStorein the default build, needing no
dependency. Writes go to a sibling temp file, are flushed, thenrenamed over the target,
so a crash leaves either the previous complete file or the new one.- BoringSSL certificate authority (
camodule): one ephemeral in-memory CA per process,
leaves minted on demand and cached per host, replacing a fresh self-signed certificate per
CONNECT. The browser is pinned to it with--ignore-certificate-errors-spki-listrather
than having certificate checking disabled wholesale. impersonation_client(&BrowserProfile) -> wreq::ClientBuilderand apub use wreq
re-export, both in the default build. Many "protected" endpoints gate on the
TLS/HTTP-2 fingerprint alone, with no JavaScript challenge; against those a browser is
pure overhead and this is the two-line path. The builder already carries the measured
emulation plusUser-Agent,Sec-CH-UA*andAccept-Languagefrom the same profile.
Re-exportingwreqmeans a consumer configuring the builder cannot end up on a different
wreqmajor from the one the emulation was measured against.pslas a dependency, for the Public Suffix List that RFC 6265 cookie scoping needs.
Which names are public suffixes is data, not a derivable rule —co.ukis one and
example.comis not, and both have two labels — so a heuristic would be a guess of exactly
the kind this crate's fingerprint values were rewritten to remove. It is a leaf:pslplus
psl-types, no further dependencies, MIT/Apache-2.0.cert_compressionmodule: brotli and zlib certificate-compression codecs (RFC 8879).
wreq5 took an enum of algorithms and supplied the codecs;wreq6 takes
&dyn CertificateCompressorand ships none. Thecompress_certificateextension is part
of theClientHello, so omitting it would change the JA4. No new dependencies —brotli
andflate2were already in the graph — and decompression is bounded, because the peer
controls the compressed bytes.- The declared MSRV is now verified.
rust-version = "1.95"was a claim no job
tested — every CI job pinned current stable — while being load-bearing enough to have
ruled out a dependency upgrade during the advisory work. ACheck MSRVjob now runs
cargo check --all-features --all-targets --lockedon it, reading the version out of
Cargo.tomlso the job cannot drift from the claim. Verified against 1.95.0: it builds
clean. - Automated stealth audit (
tests/stealth_audit.rs, 14 live checks) and wire-level
regression tests for TLS (ja4_egress), HTTP/2 (h2_egress) and headers
(http_leg_headers).
Changed
-
Breaking:
CloudScraper's browser API is async and tab-free.new_stealth_tab()→
new_stealth_page().await, andsolve_challenge,detect_challenge,rotate_profile
and everyhuman_*helper are nowasync.spawn_blockingis no longer needed or
wanted. -
Breaking:
wreq-util(GPL-3.0) removed. Measured through the same harness, its newest
Chrome entry put byte-identical TLS and HTTP/2 on the wire to values captured directly
from the browser, so the GPL table bought nothing; the Safari entry it replaced was worse.
The crate is now MIT + Apache-2.0 throughout. -
Breaking:
BrowserProfile::random()claims Chrome 153 — the version actually
captured, and the version of the binary that renders the page. It previously claimed
Chrome 124–126 while launching a much newer browser, which feature detection alone
exposes. -
MITM TLS termination and certificate minting moved from rustls + rcgen onto BoringSSL,
whichwreqalready links.rustls,tokio-rustls,rcgen,ring,aws-lc,
rustls-webpki,yasnaandpemare gone from the dependency graph: three
cryptographic backends became one. -
wreqnow builds withgzip/deflate/brotli/zstd, so theAccept-Encodingthe crate
advertises is one it can actually decode. -
The impersonation client is
wreq 6(6.0.0-rc.31), and the TLS backend moved with it
fromboring2/tokio-boring2tobtls/tokio-btls— the successor bindings by the
same author, whichwreq 6links. Staying onboring2would have meant two vendored
BoringSSL builds in one binary. The move also resolves what would otherwise have shipped
as known issues:wreq 5.xis entirely yanked on crates.io, so a new consumer could
not have resolved the manifest andcargo updatecould not run at all, andlru 0.13
carried two unsoundness advisories (RUSTSEC-2026-0002, RUSTSEC-2026-0253). The graph
shrank from 151 to 140 crates (154 → 143 withbrowser).All 13 fingerprint assertions pass unchanged — same JA4, same HTTP/2 SETTINGS, wire
order, window, priority and pseudo-order, through an entirely different TLS stack. That is
what the measurement apparatus was built for: the migration is verified rather than hoped.wreq 6.0is still a release candidate. The re-exportedwreqis therefore an RC major,
which is a deliberate, recorded choice: the alternative was shipping 1.0 on a fully yanked
dependency. -
urlis now a direct dependency:wreq 6no longer re-exportsUrl, andhttp::Uri
cannot edit userinfo, which proxy-credential redaction needs. Already in the graph, so it
costs nothing.
Fixed
- The HTTP leg advertised a different browser from the profile. The emulation supplied
its ownUser-AgentandSec-CH-UA, which silently overrode the profile's, so a session
that demoted from browser to HTTP changed identity mid-flight — the exact failure a shared
identity exists to prevent. - Every Cloudflare-served page was reported as a challenge. A bare
cf-turnstilewidget
or acf-rayheader was enough. Detection now requires interstitial markers, and
vendor-only evidence returnsChallengeKind::Nonewith the evidence attached. - A clean page was never demoted without a clearance cookie, pinning the browser open
and defeating the memory saving that dual-mode sessions exist for. - The window reported an 800×600 screen while being 1920×1080.
screen.*is now
overridden coherently with the window. - Two stealth hooks were themselves the signal.
navigator.pluginsand
pdfViewerEnabledwere being synthesised over values the browser already reported
correctly; they are left alone, and a test asserts the injected script does not mention
them. Canvas and audio noise is now seeded per identity and applied once per buffer, where
re-perturbing on every read was itself detectable. wait_for_loadsubscribed after issuing the navigation — a real race, not a flaky
test. Split intowatch_load/navigate_and_wait/reload_and_wait.element_centerreturnedSome((0, 0))for a hidden element, so a human click could
be aimed at something invisible. Zero-area boxes now returnNone.- Client-hint values that were reasoned to rather than observed. All five were wrong:
WindowsplatformVersion(15.0.0→19.0.0), macOS (14.6.1→27.0.0), the GREASE
brand ("Not-A.Brand";v="99"→"Not_A Brand";v="8"), the brand order, and the
fullVersionListbuild (153.0.0.0→153.0.8010.53, since no real Chrome reports a
zeroed build).full_version_listalso restated the brand list instead of deriving it, so
the two could disagree — something a page can check directly. - A click could miss a target it had just scrolled to. A wheel event starts a scroll, it
does not finish one: the page keeps moving for several frames after the last notch, so
measuring the element straight away returned its mid-animation position and the click
landed where the target had been.human_scroll_into_viewnow waits forwindow.scrollY
to stop changing before returning. Found as a test that failed roughly one run in four
under full-suite load and passed every time in isolation — the load only widened a window
that was always there. cargo deny check advisoriesfailed. The dev-dependencyreqwestpulled
rustls 0.23.41, which carries CVE-2025-61730; the fix needsrustls >= 0.23.45, which
MSRV 1.95 cannot resolve to. The test call sites now usewreq, which this crate already
links, soreqwestandtokio-rustlsare gone as dev-dependencies and the vulnerable
crate is removed rather than ignored.rustls,ringandaws-lcare now absent from
every graph, dev and build included, not just the shipped one.- A server could scope a cookie to a public suffix.
Cookie::parse_set_cookie
enforced that aDomainattribute domain-matched the host that sent it, but not RFC 6265
§5.3's further rule that the domain must not itself be a public suffix. A response from
attacker.comcould setDomain=com, and the session-global jar then attached that
cookie to every later request to any.comhost — a shared identity carries cookies
across hosts by design, which is what made the missing check reachable. The same section's
IP-literal rule was missing too, letting a response from1.2.3.4claim.2.3.4and reach
9.2.3.4. Both are now enforced, with the RFC's exception that aDomainequal to the host
is accepted and stays host-only rather than being refused. Found by a security review of
this branch. restore()left a running browser speaking for the identity it replaced.
StealthSession::restorerebuilt the HTTP transport from the new identity but kept any
browser already running — and that browser was launched from the previous profile, so
its User-Agent, TLS fingerprint, stealth injection and locale all came from there. A
caller who escalated and then restored got the browser leg presenting one identity and
the HTTP leg another: the mid-session change this type exists to prevent, happening
silently. The browser is now shut down alongside the transport, with a new
DemoteReason::IdentityReplacedso the event says why.- The
Cookieheader was ordered by the jar, not by path. RFC 6265 §5.4 sends longer
paths first, and browsers do. Two cookies can share a name while differing in path, so a
server reading the first occurrence — what most frameworks do — could get a different
value from the one the browser leg of the same session sent. The ordering is also
something no browser produces, which in a crate built on matching the browser exactly is
a signal in its own right. - A panic in a caller's
updateclosure disabled the state store.
JsonStateStore::updateruns caller-supplied code while holding its lock, and the lock
was taken with.expect, so one panic in their closure poisoned it and every later
get/put/remove/updatepanicked too — taking the scraper down over a bug it had
already survived. It now recovers, as every other mutex in the crate already did. - A failed page could be left open.
clear_with_browserclosed its page only on the
success path, so a navigation timeout or a solver error leaked the tab for the life of
the browser — andfetchcan pass through it twice per request, so a run of failures
accumulated renderer memory inside the one process the dual-mode design exists to keep
small. The page is now closed on every path. - A CDP domain was recorded as enabled before the browser acknowledged it. With two
tasks sharing onePage, the second saw the domain as ready whileenablewas still in
flight, subscribed, navigated, and then waited out its whole timeout for a lifecycle
event the browser never sent. The domain is recorded only once the call returns, and a
refusal is retried rather than remembered. - JA4 selection no longer depends on a
Chrome/120string match that never fired, which had
every request emitting a Chrome 120 fingerprint under a Chrome 124–126 User-Agent.
Removed
headless_chromeand its dependency tree (auto_generate_cdp,tungstenite,which,
winreg,walkdir,ureq,derive_builder,tempfile),wreq-util,rustls,
tokio-rustls,rcgen,ring,aws-lc,regex,bytes,cookie_store(direct),
tokio-socks(direct) andrand_distr. Thebrowsergraph went from ~199 crates to 143,
the default build from ~160 to 140.
Known limitations
- Edge is not modelled. An Edge User-Agent carries a
Chrome/token and so parses as Chrome,
which would report theGoogle Chromebrand under anEdg/User-Agent. Edge also carries
a different version per brand infullVersionList, which the current hint struct cannot
express. - Chrome's TLS fingerprint is reproduced except for extension
0xca34and three ML-DSA
signature schemes. The cipher hash matches exactly; the extension count (16 vs 17) and the
signature-algorithm hash do not. Re-tested onbtls, the two halves have different causes:
0xca34(trust_anchors) is in btls's BoringSSL and reachable via
SSL_CTX_set1_requested_trust_anchors, which sends the extension even with zero ids. The Rust
bindings for it are already merged upstream — btls#168
and wreq#1273, which adds
TlsOptions::trust_anchors— but neither is in a published version yet, so it cannot be
switched on here. When one is, it is a single builder call and a re-measuredCHROME_JA4.
The ML-DSA schemes remain genuinely absent from the TLS layer and are not covered by either.