Repository navigation
Releases
v0.2.0
0.2 replaces most of the 0.1 API with fewer types: configuration moves onto SessionBuilder setters, and each job has one name. The upgrade guide maps the changed 0.1.6 names. The sections below list new features, behavior changes, removals without a replacement, and fixes.
Breaking changes
Upgrading from 0.1.6
See UPGRADE.md for the migration steps and complete old-to-new API map.
Added
SessionBuilder::run_per_core, which runs one single-threaded runtime per CPU core for maximum request rate; the cores share cookies, cache, tokens and HSTS and keep their own connections.RequestBuilder::textandRequestBuilder::bytessend the request and read the decoded body in one.await.- Resumable
RequestBuilder::download_to, strong ETag checks, SHA-256 verification, and download progress callbacks. A dropped transfer is retried under the retry policy and resumes when the server sends a strong ETag. - An opt-in, bounded HTTP cache with conditional revalidation,
Varymatching, and mutation invalidation. RequestBuilder::events,Events, andEventfor parsed SSE withLast-Event-IDrecovery. The retry policy bounds reconnects in a row; each delivered event resets the count.SessionBuilder::refresh_tokenshares an asynchronous bearer-token refresh after concurrent 401 responses, scoped to the base URL origin.Crawl::saveandCrawl::loadpreserve pending and buffered pages, duplicate history, crawl policies, and retry waits.HostLimits,Page,Respect,Scope, andSkipReasonimplement serdeSerializeandDeserialize.- A Chrome 157 profile,
Browser::Chrome157, from Chrome 157.0.8103.38 on Linux and 157.0.8103.39 on macOS and Windows, with TCP and HTTP/3 captures for each desktop. - A Brave (Chromium 157) profile,
Browser::Brave157, from captures of Brave 1.99.55 (Chromium 157) on macOS, Linux, and Windows. - Edge and Opera are
Browservariants:Browser::Edge145toBrowser::Edge157, andBrowser::Opera129toBrowser::Opera139, on the Chrome captures.Browser::all()lists them. - A Firefox 159 profile,
Browser::Firefox159, from captures of Firefox 159.0 on macOS, Linux, and Windows. Browserparses from its id withFromStr, as in"chrome-154".parse().Browser::chromium_major()gives the Chromium base of a Chromium browser, andBrowser::next_hello()gives the next distinct ClientHello in the same product.session.save(path)writes the browser, the profile id, the proxy URL without its password, the cookies, and the TLS tickets, Alt-Svc entries, and HSTS store as one JSON file.Session::load(path)returns aSessionBuilder;build()fails withis_profile_changed()when the saved profile id differs, then restores the connection state.session.autosave(path, every)saves on a cookie or state change and onshutdown().SessionimplementsSerializeandSessionBuilderimplementsDeserialize, so your own data goes beside the session in your own struct. Version 1 device files load.session.to_builder()returns aSessionBuilderwith the session's settings and cookie jar. The new session shares the pool, the TLS session cache, the host limits, and the shutdown state with its parent. A new browser gets its own partition of the pool, and a change to a connection setting gives it a new pool.SessionBuilder::ja3,ja4_r,akamai, andheader_orderlayer a fingerprint on the browser. The session reports the layered profile inprofile()andprofile_id().Retry::delay(attempt)gives the backoff wait for a retry loop outside a request.Fingerprint::from_json(text)parses an echo service's JSON report, andsent.diff(&observed)lists the fields that differ.RequestBuilder::websocket()opens a WebSocket with the request's headers.Sessionimplementstower::Servicewith thetowerfeature.Redirectconverts from ausizelimit, so.redirect(3)follows up to 3 redirects.- A reference page on docs.rs for each core type:
Session,SessionBuilder,RequestBuilder,Response,Error,Browser,Platform,Proxy,Tab,Retry,Timeout,Redirect,cookie::Jar,Crawl, andtrace::Trace. The crate root is an overview with the job map. AGENTS.md, for coding agents that write code with Leyline.docs/guide/bundled-profiles.mdlists every bundled profile and brand overlay, generated from the profile files.Crawl::agent(name)obeys the robots.txt group and meta robots name of a named crawler, andCrawl::max_body_size(n)reads pages larger than the session's cap without a new session.blocking::Sessionruns requests on the calling thread for code without an async runtime:blocking::Session::new(builder), thensession.block_on(future), with everySessionmethod throughDeref. Its direct HTTP/1.1 connections read and write blocking sockets, and every timeout still applies. A blocking session isSendbut notSync; build one per thread.leyline-bssl-tokio:RecordStream::try_map_iomoves a TLS 1.3 record stream onto another transport for the same connection.Retry::budgetlimits retries to a share of requests, so retries cannot multiply the load on an overloaded upstream or on the client itself.HostLimits::max_pendingandHostLimits::max_total_pendinglimit the requests that wait for a slot. Without one, a request queue onmax_total_in_flightpast origin capacity lets every request reach the origin as its deadline ends, and goodput falls to zero.pool_max_lifetimestops reusing a connection a set time after it opened, so traffic moves after a DNS or load balancer change. Without it, a busy pool stays on the old address.max_pending_per_hostlimits the HTTP/1.1 requests that wait for a connection to one host. A request over the limit fails at once with the newKind::Overloaded, whichgateway_statusmaps to 503 andretrydoes not resend. The default is 1024, as in Envoy, so one slow host cannot hold an unbounded queue;max_pending_per_host(None)removes the limit.- New connections to a host start on the next address of each family in turn, so a pool spreads its connections across every address DNS returns instead of opening them all on the first.
- An address that keeps failing to connect moves to the back of the dial order for 30 seconds, longer each time it fails again, so new connections stop waiting on a dead address.
TcpProfilehas one getter per value:ttl(),mss(),window_size(),df(),window_scale(),no_delay(), andoptions().
Changed
- A GET or other idempotent request with a replayable body, on an HTTP/2 connection idle past
http2_ping_after_idle, is sent right after its liveness PING and does not wait a round trip for the ACK. A missing ACK fails the connection athttp2_ping_timeout, and the request is resent on a new connection. Other requests still wait for the ACK. - BoringSSL is pinned to
5b83e5d, the revision in Chromium's DEPS at tag157.0.8103.38, with six carried patches. The CRL security backport is removed because the fix and its cleanup are included upstream. - A browser profile id no longer includes
profiles/bare.toml, so an edit to the bare profile does not change browser profile ids. A saved session from before this change reports a changed profile id. - The bare profile advertises a 64 KiB HTTP/2 maximum frame size.
BodyisSend + Syncand is the one body type for requests and responses:bytes_stream()and thetowerservice both return it.- A request uses less CPU: it parses its URL once, keeps its method, URL, and headers in typed values that the response shares, and TLS reads the socket in 16 KiB batches. BENCHMARKS.md compares leyline with other HTTP clients over HTTP/2 and HTTP/1.1.
- HTTP/2 writes the requests that callers queue together in one write, and while requests wait for a stream slot it reads up to four frames before it writes.
- A session without a browser profile opens another HTTP/2 connection to a host when every open one is at the server's stream limit, up to
max_connections_per_host. Before the server announces its limit, a connection takes 100 requests, as Chrome and Go assume. A browser profile keeps one connection per host unlessmax_connections_per_hostis set, andmax_connections_per_host(1)keeps any session on one. - After a TLS 1.3 handshake, leyline seals and opens the connection's records itself and BoringSSL handles only the handshake and session tickets. A profile that sets a record size limit keeps BoringSSL's record layer, as does TLS 1.2.
- Sessions without a browser profile, client certificate, or custom CA files share one TLS context per trust setting. Each session keeps its own TLS session cache.
- Tracing has one
session.executespan per request, andpool.hitis recorded on it. Thetransport.*andpool.send_request*spans are gone. - An HTTP/1.1 or HTTP/2 response header value that is not valid UTF-8 reaches
Response::headers()with its bytes unchanged. It was replaced with U+FFFD before. Session::shutdown()returns a future. It still cancels at once;.awaitit to wait until every autosave writer has made its last save. A call without.awaitwarns.- Brave, Firefox, Safari, and Safari on iOS 18 and later race HTTP/3 by default, as Chrome does, from their captured HTTP/3 hellos. Their profile ids change.
- Every session applies the built-in block rules. A Cloudflare or AWS WAF challenge counts against its proxy in a proxy list, and
Attempt::is_block()sees it, without ablock_rulescall. Error::kind()is the one error classification.KindgainsBodyLimit,Dns, andProxyTarget, andKind::gateway_status(). A connect timeout isKind::Timeout, andis_connect()stays true.DisplayandDebugof anErrorprint the kind. The variants are in a new order, so aKindcast to an integer gives a different va...
v0.1.6
Added
- A Brave (Chromium 156) profile,
Browser::Brave156, from captures of Brave
1.98.60 (Chromium 156) on macOS, Linux, and Windows. - A Chrome 156 profile,
Browser::Chrome156, from captures of Chrome
156.0.8078.48 on macOS, Linux, and Windows. ChromiumBrand::Operacovers Chrome 154 and sends Opera 138 there.
Changed
Browser::default()andBrowser::latest(Family::Chrome)send Chrome 156, and
Browser::latest(Family::Brave)sends Brave 156.- BoringSSL is pinned to
2ea3200, theboringssl_revisionin Chromium's DEPS
at tag156.0.8078.47, and carries 6 patches.
Fixed
- An HTTP/2 connection whose peer stopped reading kept taking new requests
when the request it stalled was small enough for the socket buffers to
hold, so each later request on it waited for its own timeout. A request
that times out before its response headers arrive now makes the pool send
aPINGbefore it reuses that connection, and the pool opens a new
connection when thePINGgets no answer withinh2_ping_timeout.
v0.1.5
Added
Session::crawlreturnsCrawl, a polite stream ofCrawlPages that obeys
robots.txt and each page's robots rules by default, with five settings:
ignore_robots()turns them off;respect(Respect)adds AI-training
opt-outs (AiTraining);scope(Scope);limit(pages); and
host_limits(HostLimits). By default it sends one request at a time and one
a second per host where the session's ownHostLimitsleave it unset,
pauses a host on 429 or 503 and tries the page up to three times, sends
every request through both the session's ownHostLimitsand the crawl's
pace, keeps the site'sCrawl-delayout of the session's own requests,
waits out a robots.txt outage before it skips a site, reads sitemaps,
follows redirects as links, and drops error pages, files, block pages, and
copies by canonical URL or text, preferring the canonical page over an alias
found first.CrawlStatscounts eachSkipReason.Session::robotsandRobotsread robots.txt by RFC 9309: group matching,
with every group record (includingCrawl-delay,Content-Signal, and
Content-Usage) closing the run ofUser-agentlines, longest match with
*and$, percent-encoding, the 4xx and 5xx rules, up to five redirects,
and a 512 KiB limit.Robotsalso readsContent-Signaland the IETF draft
Content-UsageasPurpose,Crawl-delay, andSitemaplines;
Robots::allows_usetakes the URL.html::textreturns the visible text of a page, one block per line, with
<pre>kept as written and script, style, and head text left out. An
unclosed<head>ends at the first element that cannot be in it.html::Directivesreadsnoindex,nofollow,none, andnoaifrom
<meta name="robots">, a meta tag named for the agent, and
X-Robots-Tag.html::canonicalreturns the page's canonical link.- A Firefox 158 profile,
Browser::Firefox158, from captures of Firefox 158.0
on macOS, Linux, and Windows.
Changed
-
html::links,html::forms, and other readers built on the HTML scanner
now keep text after a comment or a bare<(<a>a < b</a>givesa < b),
skip</followed by a non-letter to the next>, and keep a namespace
prefix in tag names (fb:like), as the HTML standard does. -
A
HostLimits::hostoverride now applies to IPv6 literal hosts; before, it
never matched them. -
leyline-bssl-syscompiles BoringSSL with thecccrate from the source
lists BoringSSL ships, so a build needs only a C and C++ compiler and Git:
no CMake. On Windows, NASM is optional; without it the build links the
NASM objects that ship in the crate. The CMake-only variables
CMAKE_TOOLCHAIN_FILE,CMAKE_GENERATOR, and
LEYLINE_BSSL_COMPILER_EXTERNAL_TOOLCHAINno longer apply; useCC,
CXX,CFLAGS, andCXXFLAGS. The build rebuilds BoringSSL when its
source, the compiler, or the compiler flags change, and reuses the cached
library otherwise, so a restored build cache never links a stale BoringSSL.
v0.1.4
Added
- A Brave (Chromium 155) profile,
Browser::Brave155, from captures of Brave
1.97.56 (Chromium 155) on macOS, Linux, and Windows. - A Chrome 155 profile,
Browser::Chrome155, from captures of Chrome
155.0.8059.40 on macOS, Linux, and Windows. ChromiumBrand::Operacovers Chrome 153 and sends Opera 137 there.
Changed
Browser::default()andBrowser::latest(Family::Chrome)send Chrome 155, and
Browser::latest(Family::Brave)sends Brave 155.- BoringSSL is pinned to
b780f19, theboringssl_revisionin Chromium's DEPS
at tag155.0.8059.39, and carries 6 patches.
v0.1.3
Security
- A saved TLS session restored into a session with different trust
settings could resume there, and a resumed handshake skips the
certificate check, so pins and custom roots did not apply to it. Each
saved TLS session is now bound to the trust its session loaded: the
pins, the contents of every root fromadd_ca_der,add_ca_file,
SSL_CERT_FILE, andSSL_CERT_DIR, and the contents of the client
certificate and key, hashed from the same bytes the TLS context loaded.
It resumes only in a session that loaded the same trust, and
Session::fresh_poolkeeps the trust of the session it copies. Saved
state contains TLS session secrets; store it with the same protection
as credentials. A saved ticket for a proxied connection no longer holds
the proxy user name; it holds a hash of the proxy credentials, as the
accounts guide states.
Added
RequestBuilder::read_until(limit, done)streams the response and returns
aPrefixRead: the decoded prefix and aStopReason(PredicateMatched,
LimitReached, orEndOfBody). It keeps the request's redirects, retries,
authentication, host limits, anderror_for_status().
Response::read_untilkeeps its signature, and now callsdoneonly
after a chunk that adds decoded bytes, asRequestBuilder::read_until
does.examples/stock_monitor.rs.
Fixed
- A
.stream()body that failed on a timeout, a decode error, or shutdown
kept its connection and, for a decoded body, its host-limit slot until
the caller dropped the stream. The failed stream now releases both at
once, and the trace reports the real outcome instead ofDropped. WebSocketBuilder::connectandSession::preconnectignored
Session::shutdown(). Both now fail with the shut-down error after
shutdown, including a preconnect that would do nothing, and a WebSocket
handshake takes a host-limit slot while it runs. An established
WebSocket connection stays open.- An HTTP/2 connection whose peer stopped reading blocked the whole
connection inside a socket write: no response, reset, ping, or shutdown
got through. The connection now keeps reading and cancelling while
output waits, stops taking new work and reading new frames while its
write buffer is full, and parks an upload instead of queuing it past
that budget. After a second without write progress it takes no new
requests, so the pool opens a fresh connection and queued requests are
refused unsent and retried there; it closes once no stream needs it. - An HTTP/3 connection waiting for stream credit moved every queued
request into an unbounded queue, and each failed stream started a task
that could wait forever on a full body channel. Queued requests now stay
in the bounded request channel, and a failed stream reports its error
through a reserved slot. - An HTTP/1.1 upload did not read the response until the whole request
body was sent, so a server that answered 413 early and stopped reading
left the request hanging until its timeout. Leyline now reads the
response while it uploads, stops the upload at a final response or when
the server closes its side, and closes that connection. A body that
yields many chunks without pausing no longer keeps the request from
seeing the response, its timeout, or shutdown. - A streamed HTTP/2 or HTTP/3 upload whose body yielded empty chunks
without pausing kept its upload task running after the request was
cancelled. The upload task now yields on an empty chunk and stops when
the request is gone. - A cookie attribute value over 1024 octets is now ignored, as RFC 6265bis
requires, and the jar refuses cookies past the size limits from every
source, includingextend_fromand saved jars. - An HTTP/2 response that ended with trailers skipped the
Content-Lengthcheck, so a short body arrived as a success. It now
fails like a short body without trailers. - A
Body::streamwith a declared length that produced more or fewer
bytes sent a malformed request over HTTP/2 and HTTP/3. Every protocol
now fails the request with a body error and sends nothing past the
declared length. Jar::autosaveandDevice::autosavelost the last change when the
runtime stopped before the autosave task first ran. The final save now
runs in that case too.TestServer::next_requestreturned a request only after the
response'sdelay, just before the reply went out. It now returns the
request when it arrives, as documented.error_for_statusover HTTP/1.1 waited for a small error body with no
error-body timeout, so a stalled 500 with a shortContent-Lengthran to
thetotaltimeout and returnedKind::Timeoutwithout the status. The
error-body window now bounds that read, and the error keeps the status and
the bytes received.error_for_statusover HTTP/1.1 read error bodies up to 64 KiB regardless
ofmax_error_body. It now reads at mostmax_error_bodybytes with the
response, and never more thanmax_body_size. An error body that Leyline
does not decode, with compression turned off or an unknown coding, keeps
itsContent-Encodingheader.- A status error whose body read timed out or failed dropped the bytes
already decoded. It now keeps them.
Changed
- An HTTP/2 request whose header list size (RFC 7541 §4.1: name, value,
and 32 octets per field) is larger than the server's
SETTINGS_MAX_HEADER_LIST_SIZE, or than 256 KiB when the server sets
none, now fails before any of it is encoded or sent. Buffered
HTTP/2 uploads use DATA frames no larger than the free space in the
connection's 64 KiB output budget, even when the server allows larger
frames. - The timeout guide now states that
readandbodyapply to a
.stream()response and to a body the transport delivers in pieces; a
body the transport buffers is bounded byresponse_headerandtotal. Response::download_tocommits on a blocking thread after the body is
written and synced: a future dropped before the commit starts leaves the
target unchanged, and a commit that has started finishes. A
directory-sync failure after the rename says that the target was
already replaced.- Over HTTP/1.1, a
.stream()body dropped before its end closes the
connection. Leyline drains the rest only forerror_for_statuserror
bodies, up tomax_error_bodybytes.
v0.1.2
Security
leyline-bssl-syscarries BoringSSL's fix for CVE-2026-35189. A peer
certificate with manynameRelativeToCRLIssuerCRL distribution points
could make certificate verification allocate memory without bound. CRL
distribution points that use a relative name are now ignored.
Added
- A Firefox 157 profile,
Browser::Firefox157, from captures of Firefox 157.0
on macOS and Linux.
Changed
leyline-bssl-sysships pre-generated BoringSSL bindings for each
supported target, so a build no longer compiles or runsbindgen, and
links the BoringSSL source into the build directory instead of copying
it. Cold builds are faster. To generate the bindings at build time, for
example for another target, enable the newbindgenfeature. CMake uses
the Ninja generator whenninjais onPATHandCMAKE_GENERATORis
not set.
Fixed
leyline-httpbuilds again with default features off andhttp3left
out. 0.1.1 did not compile in that configuration.RequestBuilder::sendboxes its request state, so nesting a few requests
in one task no longer overflows a 2 MB thread stack.RetryPolicy::retry_unsent(true)no longer replays a non-idempotent
request after a redirect leg got a response. A connection failure on a
later leg used to resend the original request, for example a POST that
the server had already processed and answered with a 303. A 401 digest
challenge does not count as an answer, so a request whose authorized leg
could not connect is still retried.error_for_statusreads and decodes at mostmax_error_bodybytes of an
error response, within the error-body timeout, on HTTP/1.1, HTTP/2, and
HTTP/3. It used to read and decode the whole body first, so a large, slow,
or highly compressed error body ran to the body timeout or returned a body
error instead of the status error. A response below 400 is still read in
full, with its trailers and connection reuse. The error's headers no longer
list acontent-encodingthat its decoded body does not have.- Each
Browservariant keeps a fixed discriminant, listed in
profiles/browser_ids.toml, so adding a profile no longer changes the
value of the variants after it. - The
leyline-bssl-sysbuild script reruns when a patch is added to
patches/. Before, a new patch was not applied to an existing build
directory.
v0.1.1
Changed
Session::new()andSession::default()build a plain session.
Session::browser(browser)is the one-line browser session:
Session::browser(Browser::default())is the newest bundled Chrome on
Windows, whichSession::new()built before. A browser takes the first
platform its own profile covers, so a mobile browser gets its mobile
platform, with or without the builder.- A plain session never sends browser-only headers (
sec-*, client hints,
upgrade-insecure-requests,priority), for any preset. - On a browser session,
SessionBuilder::user_agentdrops thesec-ch-ua
client hint, so the user agent and the hint never disagree. - Firefox
accept-languageweights follow Firefox: equal steps from 1, as in
en,ja;q=0.5andde-DE,de;q=0.7,en;q=0.3. - A session with a browser and no
protocolcall takes its protocol policy
from the profile: it races HTTP/3 when thehttp3feature is on and the
profile's[h3]table setsrace = true. - Sessions derived with
with_identityshare the parent's connection pool,
HSTS store, andAlt-Svcknowledge, in a partition per profile, instead of
opening a new pool. - Converting an
io::Errorthat wraps aleyline::Errorreturns the wrapped
error with its own kind instead ofKind::Io.
Added
Sessions and requests
leyline::get(url)sends one GET from a plain session.SessionBuilder::base_url,bearer_auth,user_agent, andlanguages.
With abase_url, the session token goes only to the base URL's origin.Session::with_base_urlderives a session with another base.RequestBuilder::error_for_status()turns a4xxor5xxinto an error
after the retries, and keeps the headers and the start of the body.
CompressionConfig::max_error_bodycaps the kept body (64 KiB by default)
forerror_for_statusanddownload, andTimeoutConfig::error_body
bounds its read (10 s by default, and never past thetotaltimeout).
Decompression stops at the cap. The kept body is cut at the smaller of
max_error_bodyandmax_body_size, with or without content-encoding.RequestBuilder::download(path, limit)andResponse::download_towrite
the decoded body through a temporary file, so the path holds the whole
body or nothing; a declaredContent-Lengthover the cap is refused before
reading.Response::into_decoded_streamandcopy_decoded_tostream the
decoded body under a limit.RequestBuilder::pages()followsLink: rel="next", and counts the first
URL as fetched, so a redirect does not repeat the first page.
Pages::limit(max)stops aftermaxpages; there is no default limit;
Response::link(rel)andlinks()parseLinkheaders.RequestBuilder::cookie_jar,initiator, andtag.Response::attempts()
andResponse::proxy().TimeoutConfig::body(duration)bounds the time to read a whole body,
counted from the first read of the body.Session::shutdown()stops every clone of a session, including bodies
being streamed;Error::is_shut_down()names that error.relay_headers(headers, RelayBody)andResponse::relay_headersreturn the
headers to forward from a proxy or a service.proxy-authenticateand
proxy-authorizationare dropped as hop-by-hop.RelayBodyis
#[non_exhaustive]. WithRelayBody::Decoded,
Response::relay_headerskeepscontent-encodingwhen the session did not
decode the body.RequestBuilder::downloadandResponse::download_tokeep the permission
bits of a target file that already exists, never setuid, setgid, or sticky
bits.multipart::Part::file(path)reads a file part that takes a MIME type.
Errors and retries
Error::category()returns oneErrorCategoryper failure, with
as_str()andgateway_status().Error::is_connect()agrees with it.Error::is_dns(),is_proxy(), and
is_body_limit()refine it.Error::find()reaches a leyline error inside
any error chain, such as a towerBoxError.leyline::Errorconverts from
url::ParseError.- A status error keeps
headers(),header(name),body(),body_text(),
attempts(), andproxy(), and carries the retry policy's view of its last
response:retry_after()andretries_exhausted(). RetryPolicy::retry_if(predicate),wait_header(name, WaitFormat),
retry_unsent(true),skip_blocks(rules),backoff(attempt), and
rotate_proxies(list). Await_headerwins overRetry-After.
retry_unsent(true)retries any method after an error the server did not
process: DNS, connect, TLS, proxy, a connect timeout, an HTTP/2
REFUSED_STREAMreset, or an HTTP/3 request the server reports as not
processed. An HTTP/3 request the server rejects twice is retried on the
same connection.RetryPolicy::transient()caps a server-requested wait at 60 s
(max_retry_after).Error::retries_exhausted()istruewhen the policy
wanted another try and did not make it: no retries left, a server wait above
max_retry_after, or a wait or backoff longer than the time left.- A streaming request body that was never read, for example after a connect
error, is retried.
Proxies, crawling, and blocks
SessionBuilder::host_limits(HostLimits): requests in flight and per second
per origin, overrides per host, a total cap, andpause_on(statuses)to
stop new requests to an origin for the wait the server asks for, read from
the retry policy's wait headers orRetry-After.pause_forsets the pause
when the server asks for none (60 s), andmax_pausecaps a requested
pause (24 h). A streamed request holds its slot until its body ends; a
streamed body ends at its first error and releases the slot. A failed
attempt releases its slot before the retry wait.
Session::host_stats()reports requests in flight and waiting per origin.- Host limits admit each redirect hop against its own origin.
pause_onand
proxy-pool strikes apply to the origin that sent the response. A request
waits for its origin slot, then for any pause and its next rate slot, then
the total slot. No request holds a total slot while it waits for a pause or the rate spacing. The spacing is
measured at admission, and a pause is checked again just before admission.
A request that loses its turn while it waits for the total slot waits
again. A host name with a trailing dot
is the same host. Each session built from aHostLimitsvalue has its own
counters, shared by its clones and derived sessions. ProxyPool: sticky proxies, bans after repeated failures, rotation on block
rules, andProxyPool::identifiedto pin a browser identity to each proxy,
each proxy with its own cookie jar, also when two proxies share an
identity.ProxyPool::identifiedon a session without a browser fails at
build()withKind::Config.ProxyPool::stats()reports health.
Identity sessions share one connection pool, partitioned by profile.Response::block()reports a bot-protection challenge from vendors'
response headers.BlockRules::from_toml,statuses([..]),extend, and
checkdefine and apply your own rules; oneBlockRulesvalue drives
skip_blocks,rotate_on_block, andcheck.PoolStats::busyandPoolStats::idle.
Browser sessions, tabs, and devices
HeaderAnchor::BeforeCchUaplaces a header beforesec-ch-ua, where
Chrome sends the client hints a site asked for.Session::tab()returns aTabthat keeps the current page:open,
follow,submit,submit_form,fetch,xhr,post_json, and
subresourcesend the page as the initiator. A tab with no page refuses
script requests.Jar::save_to,load_from, andautosavekeep a jar on disk.
Jar and device files are versioned,{ "version": 1, .. }, and are written
with mode0600on Unix (default ACLs on Windows).autosave(path, interval)saves at most once perintervalafter a change, and saves
again when the runtime shuts down.Deviceholds a browser device (identity, frozen profile,profile_id,
user agent, proxy andproxy_password_env, languages, jar, browser state,
current page, and caller data inapp) and reopens it as the same session.
Device::checkrefuses a session that differs, andstrictrefuses a
device without a pinned profile or a proxy.Device::autosavekeeps the
device and its jar on disk. Serializing aDevicewith
proxy_password_envset never writes the proxy password;DeviceAutosave::updateandtrackchange it.
It takes aDurationorDeviceAutosaveOptions, whosestate_interval
(300 s by default) saves the connection state on a period.
SessionBuilder::expect_profile_idandError::is_profile_changed()catch
a changed profile.Session::state()returnsSessionState: TLS session tickets, HTTP/3
Alt-Svcknowledge, and the HSTS store. Sessions keep an HSTS store, and
learn no HSTS entry when certificate verification is off. Saved TLS session
keys for an authenticated proxy hold the proxy user name, never the
password or a value derived from it.Browser,Family,Platform,ChromiumBrand,Identity,ProxyUrl,
andleyline::Urlimplement serde; the profile enums have stable string
ids.SessionIdentity::to_identity()andprofile_id().- The
htmlfeature (on by default, and it enablesmultipart):
html::forms,Form::find,html::meta, andhtml::links. Forms follow
the browser rules:form="id"controls (the first element with that id),
disabled fieldsets (only the first<legend>that is a direct child is
exempt), disabled<optgroup>options left out, a list-box<select size>
with no selected option left out, one checked radio button per name, line
breaks in names and values converted to CRLF, and every HTML named
character reference.
Form::setreplaces every value of a name.Tab::submit_formencodes the
fields as the form's `FormEnc...