Repository navigation
HTTPSConnection: Prefer proxy verify policy for the proxy connection - #5093
Conversation
3d8be4b to
afdd21c
Compare
| with proxy_from_url( | ||
| proxy_url, | ||
| ca_certs=proxy.ca_certs, | ||
| ca_certs=server.ca_certs, |
There was a problem hiding this comment.
I think these test fixes make sense but I'm not 100% sure, please double check?
Before we were overriding proxy certs with server target certs anyway, so it passed. Now we don't do that anymore, so we pass proxy policy via proxy_ctx and ca_certs becomes the target server cert policy.
Feels like this was a footgun that we're fixing, but might be breaking in some edge cases? Let me know if I should expand about this in the changelog.
| ssl_context = self.proxy_config.ssl_context | ||
| self.sock = sock = self._connect_tls_proxy(self.host, sock) | ||
| sock_and_verified = _WrappedAndVerifiedSocket( | ||
| socket=sock, | ||
| is_verified=self.proxy_is_verified is True, | ||
| ) |
There was a problem hiding this comment.
Should we make the private self._connect_tls_proxy return sock_and_verified instead of sock_and_verified.socket to avoid creating _WrappedAndVerifiedSocket again here?
Alternatively, we can make sock_and_verified unneeded in this method by saving local variables and using them instead of sock_and_verified.is_verified and sock_and_verified.socket.
There was a problem hiding this comment.
Tried both, the former version felt like it was warping the intent of _WrappedAndVerifiedSocket too much, so went with the latter.
| ``HTTPSConnection._connect_tls_proxy()`` now uses | ||
| ``ProxyConfig.ssl_context``, ``ProxyConfig.assert_hostname``, and | ||
| ``ProxyConfig.assert_fingerprint`` for the proxy handshake, while retaining the | ||
| existing fallback to ``HTTPSConnection.ssl_context``, ``cert_reqs``, ``ca_*``, | ||
| and ``ssl_*`` when ``ProxyConfig.ssl_context`` is ``None``. |
There was a problem hiding this comment.
Let's avoid documenting details of the private method?
There was a problem hiding this comment.
Yeaaa I was too tunnelvisioned about documenting literal changes, removed that one.
I looked at several changelog examples to try and find the appropriate tone we want there, but maybe I should rephrase to be more consumer-centric?
|
Drive-by: the First one — in ssl_context = self.ssl_context if self.proxy_is_forwarding else None
+ ssl_context: ssl.SSLContext | None
cert_reqs: int | str | None
if proxy_ssl_context is not None:Second one — assert proxy.proxy_ssl_context is ssl_context
+ assert proxy.proxy_config is not None
assert proxy.proxy_config.ssl_context is NoneWith both applied on your branch ( Two things I checked so this is not misleading:
Happy to open this as a PR against your branch if that is easier than applying it by hand. |
f1ea8b1 to
2486c54
Compare
… policy Route forwarding HTTPS proxy handshakes through _connect_tls_proxy so ProxyConfig controls the proxy context and assertions without applying target SNI or client credentials.
1113753 to
b5f9bba
Compare
2.25.900 (2026-09-22) ===================== - Added cohabitation wheels with a ``+isolation`` version suffix, distributed through the GitHub Pages isolation index with SLSA attestations. These install only ``urllib3_future`` and omit the ``.pth`` startup hook. - Fixed gzip responses with trailing garbage raising an error on Python versions with stricter ``zlib.Decompress.flush()`` error handling, in both sync and async. See urllib3#5268 - Added opt-in response body caching for retry-delay hooks with ``Retry(cache_response_body=True)``, plus ``Retry.async_get_retry_after()`` for async body inspection. Caching defaults to off in both sync and async. (#346) - Backported HTTPS proxy TLS configuration isolation from upstream urllib3#3786 and urllib3#5093 An explicit ``proxy_ssl_context`` now retains its own trust and certificate policy for forwarding and CONNECT proxies, in both sync and async. Destination SNI, identity assertions, and client-certificate parameters no longer configure forwarding-proxy TLS. (GHSA-8988-9cw3-xx77) - Backported Deflate end-of-stream handling from upstream GHSA-gh4c-6fx4-qh6g Finished Deflate streams with trailing bytes no longer raise a spurious ``DecodeError`` during sync or async streaming. - Added concurrent WebSocket reads and writes over HTTP/1.1, in both sync and async. A waiting ``next_payload()`` releases the connection while waiting for data, allowing another thread or task to call ``send_payload()`` or ``ping()``. Concurrent readers are serialized and read timeouts remain in effect. An already-entered blocking TLS receive can still delay a concurrent writer. (#400) - Added an alternative WebSocket backend based on the ``websockets`` Sans-I/O engine. Install ``urllib3-future[ws-fast]`` and select ``ws+fast://`` or ``wss+fast://`` for sync and async WebSocket over HTTP/1.1. Requires Python 3.9+ and ``websockets>=15.0.1,<18``. Plain ``ws://`` and ``wss://`` prefer ``wsproto`` when installed and otherwise use this backend. - Reduced webextension read/write overhead and avoided redundant buffering of single protocol chunks in ``DirectStreamAccess`` and its async counterpart. - Fixed connection allocation and pool eviction races under thread and task contention, including missed wakeups and incorrect ownership transfers. Available capacity can now be used while other connections are held, within the configured pool limits. - Fixed short HTTP/3 upgrade timeouts being applied to UDP socket creation and resolver work instead of preserving the configured connect timeout, in both sync and async. (#416) - Reset Linux ``UDP_SEGMENT`` after probing GSO support, preventing unintended segmentation when falling back to ordinary datagram sends. - Backported stricter URL host validation and percent normalization from upstream urllib3#5095 Rejects raw spaces and control characters, encoded controls, malformed host escapes, and trailing newlines after ports. Validation also covers WebSocket, SOCKS, and resolver schemes. - Backported removal of fragments from absolute request targets from upstream urllib3#5079 URL fragments are no longer sent to servers or forwarding proxies, in both sync and async. - Backported clearing request body framing and rewind state on HTTP 303 redirects from upstream urllib3#5161 Applies to sync and async, including deferred responses in multiplexed mode. - Backported draining unread response bodies in 64 KiB chunks from upstream urllib3#5019 Reduces peak memory use in sync and async ``drain_conn()`` while preserving upgraded webextension streams. - Backported raising ``UnrewindableBodyError`` when a request body has no ``seek()`` method from upstream urllib3#3780 Applies to both synchronous and asynchronous body rewinding. - Backported raising ``SSLError`` for non-hexadecimal certificate fingerprints from upstream urllib3#5212 - Backported percent-decoding SOCKS proxy credentials from upstream urllib3#5031 Applies to sync and async ``python-socks`` and the ``PySocks`` fallback, preserving explicit credential overrides and decoding each username and password only once. - Backported faster URL percent encoding from upstream urllib3#5221 Adds a fast path for valid ASCII components and reduces allocations in the general encoder. - Backported ``Url.auth_decoded`` and ``Url.auth_decoded_joined`` from upstream urllib3#5055 These expose UTF-8-decoded credentials while keeping ``Url.auth`` percent-encoded. - Backported ``basic_auth_encoding`` and ``proxy_basic_auth_encoding`` options in ``make_headers()`` from upstream urllib3#5092 Origin and proxy credentials can use independent encodings; both retain the Latin-1 default. - Fixed an async TLS connection cleanup race that could cause new connections to fail with "File descriptor ... is used by transport" when a socket descriptor was immediately reused. (#417)
|
Fix included in release 5.1.0-0.nightly-2026-10-23-010359 |
…3.9 line) (#9) * Fix CVE-2026-97688 (GHSA-gh4c-6fx4-qh6g): stop an infinite loop decoding Deflate responses DeflateDecoder kept feeding data that arrived after the end of the Deflate stream back into the decompressor, so a chunked Deflate response with trailing data could make HTTPResponse.stream() loop forever. Data received after EOF now yields no output, and the unconsumed-tail check stops at EOF. Upstream squash commit (urllib3 2.8.0): ea2ad7b "Merge commit from fork" (Prevent an infinite loop when decoding Deflate responses). Applied unchanged; it uses nothing newer than Python 3.7. Co-Authored-By: Claude <noreply@anthropic.com> * Fix CVE-2026-97689 (GHSA-vxq7-64xx-v4gw): bound chunk-size line reads The streaming chunked path (HTTPResponse.stream() / read_chunked()) read each chunk-size line with an unbounded readline(), so a malicious server could send a long run of bytes with no newline and force the client to buffer all of it. The chunk-size line and the trailing CRLF line are now capped at 65536 bytes (http.client._MAXLINE) and ProtocolError is raised when the cap is exceeded. Chunk bodies are unaffected. Upstream squash commit (urllib3 2.8.0): cd770b0 "Merge commit from fork" (Bound chunk-size line reads to prevent memory exhaustion DoS). The only conflict was upstream's unrelated _READ_CHUNK_SIZE constant, which 2.6.3 does not have and which was dropped. Co-Authored-By: Claude <noreply@anthropic.com> * Fix CVE-2026-97687 (GHSA-8988-9cw3-xx77), part 1: use proxy_ssl_context for HTTPS forwarding proxies With use_forwarding_for_https=True, the TLS handshake with an HTTPS proxy used the target's ssl_context and ignored proxy_ssl_context. The proxy's context is now used. Passing only ssl_context in this mode still works as a fallback but emits a FutureWarning. Upstream commit (urllib3 2.7.0): b644729 "Fix proxy_ssl_context ignored for HTTPS forwarding proxies (urllib3#3786)". GHSA-8988-9cw3-xx77 lists this as the first half of the fix. Applied unchanged. Co-Authored-By: Claude <noreply@anthropic.com> * Fix CVE-2026-97687 (GHSA-8988-9cw3-xx77), part 2: isolate HTTPS proxy TLS settings from target TLS The target's cert_reqs, CA settings, SNI, hostname/fingerprint assertions and client certificate were applied to the TLS connection to an HTTPS proxy. cert_reqs="CERT_NONE" for the target could even overwrite verify_mode on the proxy's ssl_context in place, disabling proxy verification for later connections. The proxy handshake now uses only the proxy's policy (proxy_ssl_context, or the pool's CA policy without target identity settings when none is given). Upstream commit (urllib3 2.8.0): 07408ce "Isolate HTTPS proxy TLS settings from target TLS (urllib3#5093)". Applied unchanged on top of part 1. Co-Authored-By: Claude <noreply@anthropic.com> * Release 2.6.3.post5: backport the urllib3 2.8.0 security fixes CVE-2026-97687, CVE-2026-97688 and CVE-2026-97689. The upstream changelog fragments are folded into CHANGES.rst, as earlier ActiveState releases did. Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
…3.7 line) (#10) * Fix CVE-2026-97688 (GHSA-gh4c-6fx4-qh6g): stop an infinite loop decoding Deflate responses DeflateDecoder kept feeding data that arrived after the end of the Deflate stream back into the decompressor, so a chunked Deflate response with trailing data could make HTTPResponse.stream() loop forever. Data received after EOF now yields no output, and the unconsumed-tail check stops at EOF. Upstream squash commit (urllib3 2.8.0): ea2ad7b "Merge commit from fork" (Prevent an infinite loop when decoding Deflate responses). Applied unchanged; it uses nothing newer than Python 3.7. Co-Authored-By: Claude <noreply@anthropic.com> * Fix CVE-2026-97689 (GHSA-vxq7-64xx-v4gw): bound chunk-size line reads The streaming chunked path (HTTPResponse.stream() / read_chunked()) read each chunk-size line with an unbounded readline(), so a malicious server could send a long run of bytes with no newline and force the client to buffer all of it. The chunk-size line and the trailing CRLF line are now capped at 65536 bytes (http.client._MAXLINE) and ProtocolError is raised when the cap is exceeded. Chunk bodies are unaffected. Upstream squash commit (urllib3 2.8.0): cd770b0 "Merge commit from fork" (Bound chunk-size line reads to prevent memory exhaustion DoS). The only conflict was upstream's unrelated _READ_CHUNK_SIZE constant, which 2.6.3 does not have and which was dropped. Co-Authored-By: Claude <noreply@anthropic.com> * Fix CVE-2026-97687 (GHSA-8988-9cw3-xx77), part 1: use proxy_ssl_context for HTTPS forwarding proxies With use_forwarding_for_https=True, the TLS handshake with an HTTPS proxy used the target's ssl_context and ignored proxy_ssl_context. The proxy's context is now used. Passing only ssl_context in this mode still works as a fallback but emits a FutureWarning. Upstream commit (urllib3 2.7.0): b644729 "Fix proxy_ssl_context ignored for HTTPS forwarding proxies (urllib3#3786)". GHSA-8988-9cw3-xx77 lists this as the first half of the fix. Applied unchanged. Co-Authored-By: Claude <noreply@anthropic.com> * Fix CVE-2026-97687 (GHSA-8988-9cw3-xx77), part 2: isolate HTTPS proxy TLS settings from target TLS The target's cert_reqs, CA settings, SNI, hostname/fingerprint assertions and client certificate were applied to the TLS connection to an HTTPS proxy. cert_reqs="CERT_NONE" for the target could even overwrite verify_mode on the proxy's ssl_context in place, disabling proxy verification for later connections. The proxy handshake now uses only the proxy's policy (proxy_ssl_context, or the pool's CA policy without target identity settings when none is given). Upstream commit (urllib3 2.8.0): 07408ce "Isolate HTTPS proxy TLS settings from target TLS (urllib3#5093)". Applied unchanged on top of part 1. Co-Authored-By: Claude <noreply@anthropic.com> * Release 2.6.3.post6: backport the urllib3 2.8.0 security fixes CVE-2026-97687, CVE-2026-97688 and CVE-2026-97689. The upstream changelog fragments are folded into CHANGES.rst, as earlier ActiveState releases did. Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
|
Fix included in release 5.0.0-0.nightly-2026-11-09-045611 |
Fixed
HTTPSConnectionoverriding theproxy_config's cert policy with the target request's SSL context.Including a
proxy_configwill now properly inheritverify_modefor the proxy connection and values forca_*ssl_*are ignored.