Repository navigation
0.2 replaces most of the 0.1 API with fewer types: configuration moves onto SessionBuilder setters, and each job has one name. The upgrade guide maps the changed 0.1.6 names. The sections below list new features, behavior changes, removals without a replacement, and fixes.
Breaking changes
Upgrading from 0.1.6
See UPGRADE.md for the migration steps and complete old-to-new API map.
Added
SessionBuilder::run_per_core, which runs one single-threaded runtime per CPU core for maximum request rate; the cores share cookies, cache, tokens and HSTS and keep their own connections.RequestBuilder::textandRequestBuilder::bytessend the request and read the decoded body in one.await.- Resumable
RequestBuilder::download_to, strong ETag checks, SHA-256 verification, and download progress callbacks. A dropped transfer is retried under the retry policy and resumes when the server sends a strong ETag. - An opt-in, bounded HTTP cache with conditional revalidation,
Varymatching, and mutation invalidation. RequestBuilder::events,Events, andEventfor parsed SSE withLast-Event-IDrecovery. The retry policy bounds reconnects in a row; each delivered event resets the count.SessionBuilder::refresh_tokenshares an asynchronous bearer-token refresh after concurrent 401 responses, scoped to the base URL origin.Crawl::saveandCrawl::loadpreserve pending and buffered pages, duplicate history, crawl policies, and retry waits.HostLimits,Page,Respect,Scope, andSkipReasonimplement serdeSerializeandDeserialize.- A Chrome 157 profile,
Browser::Chrome157, from Chrome 157.0.8103.38 on Linux and 157.0.8103.39 on macOS and Windows, with TCP and HTTP/3 captures for each desktop. - A Brave (Chromium 157) profile,
Browser::Brave157, from captures of Brave 1.99.55 (Chromium 157) on macOS, Linux, and Windows. - Edge and Opera are
Browservariants:Browser::Edge145toBrowser::Edge157, andBrowser::Opera129toBrowser::Opera139, on the Chrome captures.Browser::all()lists them. - A Firefox 159 profile,
Browser::Firefox159, from captures of Firefox 159.0 on macOS, Linux, and Windows. Browserparses from its id withFromStr, as in"chrome-154".parse().Browser::chromium_major()gives the Chromium base of a Chromium browser, andBrowser::next_hello()gives the next distinct ClientHello in the same product.session.save(path)writes the browser, the profile id, the proxy URL without its password, the cookies, and the TLS tickets, Alt-Svc entries, and HSTS store as one JSON file.Session::load(path)returns aSessionBuilder;build()fails withis_profile_changed()when the saved profile id differs, then restores the connection state.session.autosave(path, every)saves on a cookie or state change and onshutdown().SessionimplementsSerializeandSessionBuilderimplementsDeserialize, so your own data goes beside the session in your own struct. Version 1 device files load.session.to_builder()returns aSessionBuilderwith the session's settings and cookie jar. The new session shares the pool, the TLS session cache, the host limits, and the shutdown state with its parent. A new browser gets its own partition of the pool, and a change to a connection setting gives it a new pool.SessionBuilder::ja3,ja4_r,akamai, andheader_orderlayer a fingerprint on the browser. The session reports the layered profile inprofile()andprofile_id().Retry::delay(attempt)gives the backoff wait for a retry loop outside a request.Fingerprint::from_json(text)parses an echo service's JSON report, andsent.diff(&observed)lists the fields that differ.RequestBuilder::websocket()opens a WebSocket with the request's headers.Sessionimplementstower::Servicewith thetowerfeature.Redirectconverts from ausizelimit, so.redirect(3)follows up to 3 redirects.- A reference page on docs.rs for each core type:
Session,SessionBuilder,RequestBuilder,Response,Error,Browser,Platform,Proxy,Tab,Retry,Timeout,Redirect,cookie::Jar,Crawl, andtrace::Trace. The crate root is an overview with the job map. AGENTS.md, for coding agents that write code with Leyline.docs/guide/bundled-profiles.mdlists every bundled profile and brand overlay, generated from the profile files.Crawl::agent(name)obeys the robots.txt group and meta robots name of a named crawler, andCrawl::max_body_size(n)reads pages larger than the session's cap without a new session.blocking::Sessionruns requests on the calling thread for code without an async runtime:blocking::Session::new(builder), thensession.block_on(future), with everySessionmethod throughDeref. Its direct HTTP/1.1 connections read and write blocking sockets, and every timeout still applies. A blocking session isSendbut notSync; build one per thread.leyline-bssl-tokio:RecordStream::try_map_iomoves a TLS 1.3 record stream onto another transport for the same connection.Retry::budgetlimits retries to a share of requests, so retries cannot multiply the load on an overloaded upstream or on the client itself.HostLimits::max_pendingandHostLimits::max_total_pendinglimit the requests that wait for a slot. Without one, a request queue onmax_total_in_flightpast origin capacity lets every request reach the origin as its deadline ends, and goodput falls to zero.pool_max_lifetimestops reusing a connection a set time after it opened, so traffic moves after a DNS or load balancer change. Without it, a busy pool stays on the old address.max_pending_per_hostlimits the HTTP/1.1 requests that wait for a connection to one host. A request over the limit fails at once with the newKind::Overloaded, whichgateway_statusmaps to 503 andretrydoes not resend. The default is 1024, as in Envoy, so one slow host cannot hold an unbounded queue;max_pending_per_host(None)removes the limit.- New connections to a host start on the next address of each family in turn, so a pool spreads its connections across every address DNS returns instead of opening them all on the first.
- An address that keeps failing to connect moves to the back of the dial order for 30 seconds, longer each time it fails again, so new connections stop waiting on a dead address.
TcpProfilehas one getter per value:ttl(),mss(),window_size(),df(),window_scale(),no_delay(), andoptions().
Changed
- A GET or other idempotent request with a replayable body, on an HTTP/2 connection idle past
http2_ping_after_idle, is sent right after its liveness PING and does not wait a round trip for the ACK. A missing ACK fails the connection athttp2_ping_timeout, and the request is resent on a new connection. Other requests still wait for the ACK. - BoringSSL is pinned to
5b83e5d, the revision in Chromium's DEPS at tag157.0.8103.38, with six carried patches. The CRL security backport is removed because the fix and its cleanup are included upstream. - A browser profile id no longer includes
profiles/bare.toml, so an edit to the bare profile does not change browser profile ids. A saved session from before this change reports a changed profile id. - The bare profile advertises a 64 KiB HTTP/2 maximum frame size.
BodyisSend + Syncand is the one body type for requests and responses:bytes_stream()and thetowerservice both return it.- A request uses less CPU: it parses its URL once, keeps its method, URL, and headers in typed values that the response shares, and TLS reads the socket in 16 KiB batches. BENCHMARKS.md compares leyline with other HTTP clients over HTTP/2 and HTTP/1.1.
- HTTP/2 writes the requests that callers queue together in one write, and while requests wait for a stream slot it reads up to four frames before it writes.
- A session without a browser profile opens another HTTP/2 connection to a host when every open one is at the server's stream limit, up to
max_connections_per_host. Before the server announces its limit, a connection takes 100 requests, as Chrome and Go assume. A browser profile keeps one connection per host unlessmax_connections_per_hostis set, andmax_connections_per_host(1)keeps any session on one. - After a TLS 1.3 handshake, leyline seals and opens the connection's records itself and BoringSSL handles only the handshake and session tickets. A profile that sets a record size limit keeps BoringSSL's record layer, as does TLS 1.2.
- Sessions without a browser profile, client certificate, or custom CA files share one TLS context per trust setting. Each session keeps its own TLS session cache.
- Tracing has one
session.executespan per request, andpool.hitis recorded on it. Thetransport.*andpool.send_request*spans are gone. - An HTTP/1.1 or HTTP/2 response header value that is not valid UTF-8 reaches
Response::headers()with its bytes unchanged. It was replaced with U+FFFD before. Session::shutdown()returns a future. It still cancels at once;.awaitit to wait until every autosave writer has made its last save. A call without.awaitwarns.- Brave, Firefox, Safari, and Safari on iOS 18 and later race HTTP/3 by default, as Chrome does, from their captured HTTP/3 hellos. Their profile ids change.
- Every session applies the built-in block rules. A Cloudflare or AWS WAF challenge counts against its proxy in a proxy list, and
Attempt::is_block()sees it, without ablock_rulescall. Error::kind()is the one error classification.KindgainsBodyLimit,Dns, andProxyTarget, andKind::gateway_status(). A connect timeout isKind::Timeout, andis_connect()stays true.DisplayandDebugof anErrorprint the kind. The variants are in a new order, so aKindcast to an integer gives a different value; match on the variant instead.- Every way to read a body decodes it:
bytes(),text(),bytes_stream(),write_to(writer), anddownload_to(path).RequestBuilder::decode(false)gives every sink the bytes as sent and keeps the sameaccept-encodingon the wire. - Every sink stops at
max_body_size, raw streams included.RequestBuilder::max_body_size(n)replaces the session's value for one request,.stream()responses included. A body over the cap isKind::BodyLimit, and the message namesmax_body_size. - Each retry waits between half of and all of the computed backoff (equal jitter). A request that never reached the server, one whose TLS handshake timed out included, is retried for any method.
- Each attempt takes the request's own proxy, else the session's proxy or proxy list. In 0.1, a rotation list replaced a request's own proxy on retries.
build()checks the settings, so a bad proxy URL fails there and not at the first send.- Each
SessionBuilder::tracecall adds a hook beside the earlier ones, including a hook that the parent session carries intoto_builder(). error_for_status()borrows the response, so the body stays readable after a status error.query,form, andTab::submituse the standardBorrow<(K, V)>bound instead of a custom pair-conversion trait. Pass owned(key, value)tuples or references to them. Explicit generic arguments change from<I, P>to<I, K, V>; inferred calls need no changes.RequestBuilder::header_ordertakes any iterator of strings, asSessionBuilder::header_orderdoes.Response::text_with_charsetneeds thecharsetfeature. Without it, the method ignored its argument.text()stays in every build.leyline::get(url)returns aRequestBuilder, so request settings,.text(), and.bytes()chain before.await.multipart::Part::fileandForm::filereturnleyline::Result. A file that cannot be opened isKind::Io.TcpProfilefields are private; read them through its getters.TlsContext::new(browser, min_version)replacesTlsContext::from_profile, whose profile argument no caller outside the crate could build. A build with only--cfg leyline_unstable_bsslcompiles again.
Removed
leyline-bsslandleyline-bssl-tokiodrop the functions and constants thatleyline-httpandleyline-quichenever call, about 5,900 lines. Code that calls these crates directly may need the BoringSSL calls fromleyline-bssl-sysinstead.
Fixed
pool_idle_timeoutnow closes idle connections on a session with no traffic. Before, they stayed open until the next request.- A response with several
Transfer-Encodinglines is now read as one list, andchunkedmust end it exactly once. Before, only the first line counted, sochunkedthengzipwas framed as chunked, a response desync risk. - A session that talks to more hosts than
max_connectionsno longer collapses. Eviction scanned every entry under one write lock and closed sockets inside it; at 20,000 hosts and 300,000 req/s on the default limit this held the client to about 190,000 req/s with latency of many seconds. The pool now has up to 64 shards, and sockets close outside the lock. - An HTTP/2 connection that uses its last stream id now retires, and later requests move to a new connection. Before, the connection stayed in the pool and every later request to that host failed.
retry(n)now reaches the origin at mostn + 2times for an idempotent request: one resend on a stale pooled connection for the first attempt only. Before, every attempt could resend, up to2 * (n + 1)times.- HTTP/2 requests no longer wait until their deadline when more than 1,024 requests queue on a connection and the queue drains in a batch. A sender that waited now wakes the next one, and every waiting sender fails at once when the connection ends.
- Dropping a session that uses
Proxy::identifiedfrees the per-proxy identity sessions and their TLS contexts; before, each derived session kept the pool alive through a reference cycle. - A connection failure no longer evicts the healthy HTTP/2 or HTTP/3 connection that replaced the failed one in the pool.
- A retry that reuses a pooled connection is treated as a pooled attempt: it is replayed after a failure and reported as reused.
- HTTP/1 rejects malformed status-code widths and version tokens, discards the failed connection, and does not replay a malformed response.
- The HTTP/2 reader builds with the minimum supported
tokio-utilversion. - Reject malformed HTTP/2 response fields and invalid or repeated Content-Length fields instead of accepting incomplete metadata. Apply the shared field validation to HTTP/3 responses and trailers.
- Reject HTTP/2 connections whose server preface does not start with SETTINGS, and server push after acknowledgement of disabled push.
- An HTTP/2 server that sends
SETTINGS_ENABLE_PUSHwith a value other than 0 fails the connection with PROTOCOL_ERROR, as RFC 9113 section 6.5.2 requires. - A malformed PRIORITY frame or a zero WINDOW_UPDATE increment on a stream resets only that stream. These errors closed the whole HTTP/2 connection.
- Concurrent connections to one host and port share a single DNS lookup, including the HTTP/2 and HTTP/3 legs of
Protocol::Race. download_tosyncs its resume checkpoint at most once a second, writes every chunk that is already received in one blocking call, and hashes a saved prefix while the resume request is in flight. A 256 MiB download over loopback on an M-series Mac went from about 54 MiB/s to about 1,250 MiB/s.download_tofinishes a saved prefix of unknown length when the server answers the resume with 416 andContent-Range: bytes */Nthat equals the prefix. It failed with 416 on every retry.- An HTTP/2 RST_STREAM or WINDOW_UPDATE overflow that arrives after END_STREAM no longer discards a streamed body that is still queued for the reader.
- Benchmark clients use rustls 0.23.46, which fixes certificate decompression, session debug, key-exchange hint, and ECH rejection advisories. Leyline's TLS backend remains BoringSSL.
Session::stats()counts one HTTP/2 or HTTP/3 checkout per request. It counted again when a request opened a connection.- The
Debugoutput of a response fingerprint printed theauthorizationandcookievalues it sent.Fingerprintprints them as***. - A zstd body whose frame declares a window over 8 MiB fails to decode, the limit RFC 9659 sets for the
zstdcoding. - Cookie
Expiresdates with an hour over 23, a minute or second over 59, or a year before 1601 are ignored, as RFC 6265 requires. - HTTP/2 and HTTP/3 fail a response after 16 informational (1xx) responses, as HTTP/1.1 already did.
- An HTTP/1.1 chunk-size line follows RFC 9112 section 7.1: a sign, non-hex characters, whitespace before the size, or anything other than spaces, tabs or
;after it is a protocol error. - A padded HTTP/2 DATA, HEADERS or PUSH_PROMISE frame too short for its fixed fields is a FRAME_SIZE_ERROR, as RFC 9113 section 4.2 requires; padding longer than the rest of the frame is a PROTOCOL_ERROR.
Response::bytes()on a streamed response decodes each chunk as it arrives, so a corrupt compressed body fails at its first bad chunk.