urllib3-transport update for httpcore v1.x.x

[psymbio]

The documentation mentions a public gist to use create a transport object with urllib3. But, this gist is no longer the most updated version as there has since been a restructuring of the httpcore library. Most of the interfaces are no longer available, etc.

Current version of httpx relies on "httpcore==1.*" (https://github.com/encode/httpx/blob/master/pyproject.toml) and a new update for the gist is required too.

My issue stems from pyodide/pyodide#4292 (comment).

[tomchristie]

Wonderful, yes. 😃

I'd very much like to see this updated.
I've commented against the gist on how to get started.

[tomchristie]

Related (note to self)... A pycurl transport would also be interesting.

[tomchristie]

I'd suggest we drop this section from the docs until we've got a working urllib3 transport available again. #3057

[psymbio]

Currently, Pyodide doesn't support httpx, so this custom transport layer would help with this issue and hopefully support other libraries on Pyodide that use httpx. Currently, joemarshall has come up with a way to support urllib3 in Pyodide (urllib3/urllib3#3195) which this solution relies on.

Testing environment: https://jupyter.org/try-jupyter/lab/

httpx fails on Pyodide:

import micropip

await micropip.install("ssl")
import ssl

await micropip.install("httpx", keep_going=True)
import httpx
client = httpx.Client()
client.get("https://swapi.dev/api/people/1")

Results in: ConnectError: [Errno 23] Host is unreachable

httpx working on Pyodide:

import micropip
await micropip.install("ssl")
import ssl
await micropip.install("httpx", keep_going=True)
import httpx
await micropip.install('https://raw.githubusercontent.com/psymbio/pyodide_wheels/main/urllib3/urllib3-2.1.0-py3-none-any.whl', keep_going=True)
import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
import json

class URLLib3Transport(httpx.BaseTransport):
    def __init__(self):
        self.pool = urllib3.PoolManager()

    def handle_request(self, request: httpx.Request):
        urllib3_response = self.pool.request('GET', str(request.url))  # Convert httpx.URL to string
        content = json.loads(urllib3_response.data.decode('utf-8'))  # Decode the data and load as JSON
        stream = httpx.ByteStream(json.dumps(content).encode("utf-8"))  # Convert back to JSON and encode
        headers = [(b"content-type", b"application/json")]
        return httpx.Response(200, headers=headers, stream=stream)

client = httpx.Client(transport=URLLib3Transport())
response = client.get("https://swapi.dev/api/people/1")
print(response.text)
assert response.status_code == 200

This results in: {"name": "Luke Skywalker", "height": "172", ...

Let me know if any changes are needed. Thanks for the enthusiasm and direction!