Drop HSTS Preloading

[florimondmanca]

Maybe? See the rationale in #1102

Fixes #1102, closes #896

Essentially:

>>> import httpx
>>> r = httpx.get('http://www.paypal.com')  # -> Now kept over HTTP rather than forced to HTTPS.

I suppose if we want to move forward with this we'd want it in 0.14, rather than 1.0, since it might be a small breaking change?

Since this was an always-on feature not controlled by any options, I can't think of a smooth deprecation path, but any ideas welcome!

[StephenBrown2]

Perhaps hstspreloading could be kept, but not enforced by default, and a TRACE or DEBUG message emitted when a host is found on the list and doesn't match the preferred scheme?

[tomchristie]

So, it's a wooly issue, but I think yes let's get this in and treat HSTS as appropriate as a browser feature, but not necessarily desirable in client library.

[StephenBrown2]

I would hate for the great work in the hstspreload module to go unused. Would it be good to include a brief mention in the docs for those that do want that functionality?

[florimondmanca]

I'm trying to think of how people could enable this without us having to support it in core with a flag…

I don't think a custom transport would be the way to go, since transports don't deal with client logic like "optionally modify the request URL".

Probably more like a client subclass, then…?

import httpx
import hstspreload

class HSTSPreloadMixin:
    def build_request(self, *args, **kwargs):
        request = super().build_request(*args, **kwargs)

        url = request.url

        if (
            url.scheme == "http"
            and hstspreload.in_hsts_preload(url.host)
            and len(url.host.split(".")) > 1
        ):
            port = None if url.port == 80 else url.port
            request.url = url.copy_with(scheme="https", port=port)

        return request

class AsyncClient(HSTSPreloadMixin, httpx.AsyncClient):
    pass

class Client(HSTSPreloadMixin, httpx.Client):
    pass

Incidentally I think this could also fit in a "middleware" kind of concept (#345, also mentioned in #984), but that's definitely not something we'll have 1.0.

[StephenBrown2]

Possibly, but I was thinking of and even simpler approach/demonstration:

util.py

from typing import Union

from httpx import URL
from hstspreload import in_hsts_preload

def check_hsts(url: Union[str, URL]):
    if isinstance(url, str):
        url = URL(url)
    if in_hsts_preload(url.host):
        return url.copy_with(scheme="https")
    return url

...
main.py

import httpx

from util import check_hsts

httpx.get(check_hsts(the_url))

or with base_url:

with httpx.Client(base_url=check_hsts(the_url)) as client:
    client.get(path)