Repository navigation
Releases: jose-pr/syshttp
Release list
v0.1.0
Added
syshttp.client: an HTTP client on the standard library, so that no feature
of syshttp needsrequests(which onlysyshttp.autopatchpatches, in a
program that imports it).HTTPClient(transport=None, *, headers, timeout, deadline, user_agent, env, proxymap, ssl_context, proxy_ssl_context, verify, cookies, tokens, auth_provider, retries, backoff, status_forcelist, cookie_hosts)sends through a transport and follows redirects with one rule
for credentials (Authorization,CookieandProxy-Authorizationgo to the
origin they were given for and to no other scheme, host or port). The defaultHTTPTransport(onhttp.client,
with a per-thread connection pool, framed bodies,gzip/deflate
decoding and a bounded reply size) routes every request and redirect hop by
proxy.proxy_map(env=env)(orproxymap=): the first entry that accepts a
connection, anhttporhttpsproxy (CONNECT; anhttpsproxy over TLS
with the origin's TLS inside it), a URL-prefix gateway, a proxy's
credential to the proxy alone; and verifies with the caller'sssl_context,
else theCA_BUNDLEbundle orverify=<path>, else the system's context
(proxy_ssl_contextfor anhttpsproxy or gateway). Cookies (RFC 6265,
per origin, on every hop, in a jar by host withcookies=), tokens,_PY
hooks,auth_providerand retries (Retry-Afterhonoured, a status that
outlasts them returned) work as they did on the session.HTTPResponse
reads its body whole ascontent,text()andjson().syshttp.fetch()(defined insyshttp.client, re-exported from the root
without importing it):fetch(url, method="GET", *, headers, data, json, timeout, client, transport, reauth, **client_options)returns the
HTTPResponsewith its body read, whatever the status; a givenclientis
left open and a new one is closed;reauthruns once after a 401 or 403.- The transport protocol is public:
Transport(send()andclose()),
PreparedRequestandprepare_request()give another transport (on
requests,httpx, or a fake) the checks and body framing of syshttp's own.
SYSHTTP_DEFAULT_TRANSPORT(module:attror<file>.py:attr, read under the
prefixes) names the transport of a client given none; a value that cannot be
used isConfigurationError.hooks.load_object()is the loader it shares
with the_PYhooks. syshttp.exceptions:SysHTTPError(the base of what goes wrong at run
time),ConfigurationError(aValueError),HookError(aRuntimeError)
andResponseError(aValueError), exported from the root. A malformed
setting, a hook that cannot be loaded, a failing hook and a body that is not
JSON raise them; every one is still theValueErrororRuntimeErrorit
was, so existing handlers keep working. A caller's own mistake stays a plain
ValueError/TypeError.- Every public module declares
__all__(proxyalso lists its setting-name
constants,PROXYandDEFAULT_AUTH_HEADERamong them). syshttp.env.HTTPEnv(prefixes=None, environ=None): every setting is a
variable with a full name -- syshttp's own carrySYSHTTP_
(SYSHTTP_PROXY,SYSHTTP_HOME,SYSHTTP_HOOKS_<KEY>,
SYSHTTP_IMPORTPATCH), conventional ones keep theirs (HTTPS_PROXY,
NO_PROXY,HTTP_PROXY_AUTH,CA_BUNDLE) -- read as<PREFIX><NAME>for
each prefix ofSYSHTTP_PREFIXin turn, a:-separated list used
literally (default: one empty prefix, the names as written). The first
prefix that sets a variable wins, each variable on its own:
SYSHTTP_PREFIX=MYTOOL_reads onlyMYTOOL_<NAME>,MYTOOL_:falls back
to the unprefixed names.find()returns the variable found with its
value, for messages. Every function that reads a setting takesenv=,
new_session()and thefetch*_autohelpers included.CA_BUNDLE(under the prefixes) names the CA bundle a session's
verify=Trueuses; sessions no longer read requests' own
REQUESTS_CA_BUNDLE/CURL_CA_BUNDLE.- The system's proxy configuration is the last proxy source, decided for the
URL being requested: WinHTTP on Windows, CFNetwork on macOS, libproxy or the
desktop's settings elsewhere, PAC scripts and WPAD included (through
proxylib, a new dependency). It comes afterSYSHTTP_PROXYand the
*_PROXYvariables, after theNO_PROXYcheck, and is always spoken to as
connect.SYSHTTP_SYSTEM_PROXY=0turns it off.proxy.plan(),
proxy_url(),resolve()andproxy_authorization()take the request URL
(url=/target=).proxy.configured_proxy_url()gives the
environment's answer alone. Thepacextra adds a JavaScript engine for PAC
scripts proxylib evaluates itself (Linux without libproxy). syshttp.autopatch:SYSHTTP_IMPORTPATCH=1(or:-separated names:
requests,urllib,httpx,aiohttp; read under the prefixes like
every setting) patches those libraries as they are imported,
in any program of the environment, so requests without a proxy or TLS
setting of the caller's route by syshttp and verify with the system's
context. Through importpatch, a new dependency, and its entry points;
patch()/unpatch()do it from code.- The system's answer is a list (
PROXY a; PROXY b; DIRECT): the first proxy
that accepts a connection is used (proxylib'sfirst_working_proxy), and a
DIRECT reached first is direct. No reachable proxy and no DIRECT raises
proxy.NoReachableProxyError(aConnectionError, exported from the
root); a session raisesrequests.exceptions.ProxyErrorfor it. new_session(ssl_context=..., proxy_ssl_context=...)(and the same on the
fetch*_autohelpers): a caller'sssl.SSLContextis used as given for
every https origin, or for anhttps://proxy. That covers mutual TLS,
ciphers and protocol versions. It cannot be combined with averifyother
thanTrue.hooks.wrapper_environment(): theSYSHTTP_HOOK_*variables a_CURL
wrapper run receives.jar.default_root(): the per-user state directory used when
SYSHTTP_HOMEis unset.syshttp.__version__.- Hosts that share cookies:
SYSHTTP_COOKIE_HOSTS(hosts split by,,
groups by;) ornew_session(cookie_hosts=...)declares names that reach
one service. In a group, a cookie a member sets for itself or another
member belongs to the first host, every member's requests carry the
group's cookies, and any member can delete them; aDomainoutside the
group is still judged on the host that sent it.cookies.host_aliases()
builds the mapping, whichset_cookie_specs,merge_set_cookie_headers
andhost_only_policytake asaliases=. - The shipped header and the settings page list every variable syshttp
reads under its own name (the proxy family,REQUEST_METHOD, the CA bundle
variables, the state-directory variables, importpatch's and proxylib's).
Changed
-
syshttp.fetchis a function, not a module:fetch()replaces the helpers
below._PYhooks are called ashook(client, method, url, kwargs)with the
keyword arguments ofHTTPClient.request(), and may return an
HTTPResponse;auth.AuthProvider.ensure(client, url)takes the client.
CA_BUNDLEis the bundle a client'sverify=Trueuses.cookies.pykeeps
the Netscape rows,set_cookie_specs,cookie_appliesandhost_aliases
only; the client keeps its cookies as those rows. -
Routing is proxylib 2's (
proxylib>=2.0.2,<3; 2.0.1 breaks pip's cached
requests).proxy.proxy_map(proxy, noproxy, proxy_user, *, env)is the settings as one proxylib proxy map,
read at each lookup;proxy.configured_proxy()isSYSHTTP_PROXYas a
proxylibProxy;proxy.system_map()is the system's configuration
(proxylib.auto_proxy). The*_PROXYvariables are read per scheme
(lowercase first, httpoxy-safe),NO_PROXYtakes CIDR entries, and curl's
-x ""is direct.SYSHTTP_PROXY_AUTHgoes toSYSHTTP_PROXYonly and
HTTP_PROXY_AUTHto the machine's proxies only. -
A list from the system (
PROXY a; PROXY b; DIRECT) is tried in order by the
session, httpx and aiohttp (proxylib's failover), replacing the probe of
the entry above; urllib takes the first reachable entry. No proxy
connecting and no DIRECT is the client's own connection error;
NoReachableProxyErroris proxylib's. -
The session's
ProxyAdapteris proxylib'sProxyMapAdapterwith the
system's TLS;adapter.route(url)returns the entries it will try.
proxies=given tonew_sessionare requests' own, with no syshttp
credential added. -
syshttp.autopatchroutes throughproxylib.patch()and patches TLS only;
requests' TLS is a pool hook on everyHTTPAdapter. A caller's own
proxies andtrust_env = Falsestill win over syshttp (proxylib 2.0.2's
rule). aiohttp is not offered
URL-prefix gateways. New targetpip(pip._vendor.requests, proxylib
2.0.1'spiptarget): pip's adapters route through the map unless pip is
given--proxyor--no-proxy-env; pip keeps its own TLS context. -
The jar constructors (
FileCookieJar,FileTokenJar) no longer create
their directory; it is created at the first write. -
A session returns the last response when a status in
status_forcelist
outlasts its retries, as plain requests does; it used to raise
requests.exceptions.RetryError, which the fetch helpers (-> (status, text)) passed on. The*_autohelpers takenew_session's options as
**session_options(every keyword they had still works) and their
timeoutmay be a(connect, read)pair. -
Every dependency's floor is its newest release that supports Python 3.9:
thesessionextra requiresrequests>=2.32.5andurllib3>=2.6.3(was
requests>=2.25), andsslcontext>=2.1.0. Below requests 2.32.3 a caller's
ssl_contextwas silently ignored, urllib3 1.26 raises a bareValueError
for an https IP-literal host, and urllib3 up to 2.2 leaks th...