Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Event hooks #1215

Closed
wants to merge 7 commits into from
Closed

Event hooks #1215

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
f07bfc5
Add EventHooks internal datastructure
tomchristie Aug 24, 2020
3767797
Add support for 'request' and 'response' event hooks
tomchristie Aug 24, 2020
4450ea2
Support Client.event_hooks property
tomchristie Aug 24, 2020
d3e0472
Handle exceptions raised by response event hooks
tomchristie Aug 24, 2020
7247802
Docs for event hooks
tomchristie Aug 24, 2020
e9ca1c1
Only support 'request' and 'response' event hooks
tomchristie Aug 24, 2020
b61972c
Add event_hooks to top-level API
tomchristie Aug 24, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 48 additions & 0 deletions docs/advanced.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -221,6 +221,54 @@ with httpx.Client(headers=headers) as client:
...
```

## Event Hooks

HTTPX allows you to register "event hooks" with the client, that are called
every time a particular type of event takes place.

There are currently two event hooks:

* `request` - Called once a request is about to be sent. Passed the `request` instance.
* `response` - Called once the response has been returned. Passed the `response` instance.

These allow you to install client-wide functionality such as logging and monitoring.

```python
def log_request(request):
print(request)

def log_response(response):
print(response)

client = httpx.Client(event_hooks={'request': log_request, 'response': log_response})
```

You can also use these hooks to install response processing code, such as this
example, which creates a client instance that always raises `httpx.HTTPStatusError`
on 4xx and 5xx responses.

```python
def raise_on_4xx_5xx(response):
response.raise_for_status()

client = httpx.Client(event_hooks={'response': raise_on_4xx_5xx})
```

You can also register multiple event hooks for each type of event, and can modify
the event hooks either on client instantiation, or modify and inspect the
event hooks using the `client.event_hooks` property.

```python
client = httpx.Client()
client.event_hooks['request'] = [log_request]
client.event_hooks['response'] = [log_response, raise_for_status]
```

!!! note
If you are using HTTPX's async support, then you need to be aware that
hooks registered with `httpx.AsyncClient` MUST be async functions,
rather than plain functions.

## .netrc Support

HTTPX supports .netrc file. In `trust_env=True` cases, if auth parameter is
Expand Down
31 changes: 29 additions & 2 deletions httpx/_api.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ def request(
verify: VerifyTypes = True,
cert: CertTypes = None,
trust_env: bool = True,
event_hooks: dict = None,
Copy link
Member

@florimondmanca florimondmanca Aug 29, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'd like us to prefer a stricter typing here, and a more straightforward style that requires using a list of callables:

client = httpx.Client(event_hooks={"request": [on_request], "response": [on_response])

It would make it clearer that client.event_hooks is really just always a dict of lists, rather than "a dict of sometimes a callable, sometimes a list of callables, depending on what you've set" (not even sure that's what this PR intends, actually?).

Hence this suggestion (which would need to be applied across _client.py as well):

Suggested change
event_hooks: dict = None,
event_hooks: Dict[str, List[Callable]] = None,

) -> Response:
"""
Sends an HTTP request.
Expand Down Expand Up @@ -84,7 +85,12 @@ def request(
```
"""
with Client(
proxies=proxies, cert=cert, verify=verify, timeout=timeout, trust_env=trust_env,
proxies=proxies,
cert=cert,
verify=verify,
timeout=timeout,
trust_env=trust_env,
event_hooks=event_hooks,
) as client:
return client.request(
method=method,
Expand Down Expand Up @@ -117,6 +123,7 @@ def stream(
verify: VerifyTypes = True,
cert: CertTypes = None,
trust_env: bool = True,
event_hooks: dict = None,
) -> StreamContextManager:
"""
Alternative to `httpx.request()` that streams the response body
Expand All @@ -128,7 +135,13 @@ def stream(

[0]: /quickstart#streaming-responses
"""
client = Client(proxies=proxies, cert=cert, verify=verify, trust_env=trust_env)
client = Client(
proxies=proxies,
cert=cert,
verify=verify,
trust_env=trust_env,
event_hooks=event_hooks,
)
request = Request(
method=method,
url=url,
Expand Down Expand Up @@ -162,6 +175,7 @@ def get(
verify: VerifyTypes = True,
timeout: TimeoutTypes = DEFAULT_TIMEOUT_CONFIG,
trust_env: bool = True,
event_hooks: dict = None,
) -> Response:
"""
Sends a `GET` request.
Expand All @@ -184,6 +198,7 @@ def get(
verify=verify,
timeout=timeout,
trust_env=trust_env,
event_hooks=event_hooks,
)


Expand All @@ -200,6 +215,7 @@ def options(
verify: VerifyTypes = True,
timeout: TimeoutTypes = DEFAULT_TIMEOUT_CONFIG,
trust_env: bool = True,
event_hooks: dict = None,
) -> Response:
"""
Sends an `OPTIONS` request.
Expand All @@ -222,6 +238,7 @@ def options(
verify=verify,
timeout=timeout,
trust_env=trust_env,
event_hooks=event_hooks,
)


Expand All @@ -238,6 +255,7 @@ def head(
verify: VerifyTypes = True,
timeout: TimeoutTypes = DEFAULT_TIMEOUT_CONFIG,
trust_env: bool = True,
event_hooks: dict = None,
) -> Response:
"""
Sends a `HEAD` request.
Expand All @@ -260,6 +278,7 @@ def head(
verify=verify,
timeout=timeout,
trust_env=trust_env,
event_hooks=event_hooks,
)


Expand All @@ -279,6 +298,7 @@ def post(
verify: VerifyTypes = True,
timeout: TimeoutTypes = DEFAULT_TIMEOUT_CONFIG,
trust_env: bool = True,
event_hooks: dict = None,
) -> Response:
"""
Sends a `POST` request.
Expand All @@ -301,6 +321,7 @@ def post(
verify=verify,
timeout=timeout,
trust_env=trust_env,
event_hooks=event_hooks,
)


Expand All @@ -320,6 +341,7 @@ def put(
verify: VerifyTypes = True,
timeout: TimeoutTypes = DEFAULT_TIMEOUT_CONFIG,
trust_env: bool = True,
event_hooks: dict = None,
) -> Response:
"""
Sends a `PUT` request.
Expand All @@ -342,6 +364,7 @@ def put(
verify=verify,
timeout=timeout,
trust_env=trust_env,
event_hooks=event_hooks,
)


Expand All @@ -361,6 +384,7 @@ def patch(
verify: VerifyTypes = True,
timeout: TimeoutTypes = DEFAULT_TIMEOUT_CONFIG,
trust_env: bool = True,
event_hooks: dict = None,
) -> Response:
"""
Sends a `PATCH` request.
Expand All @@ -383,6 +407,7 @@ def patch(
verify=verify,
timeout=timeout,
trust_env=trust_env,
event_hooks=event_hooks,
)


Expand All @@ -399,6 +424,7 @@ def delete(
verify: VerifyTypes = True,
timeout: TimeoutTypes = DEFAULT_TIMEOUT_CONFIG,
trust_env: bool = True,
event_hooks: dict = None,
) -> Response:
"""
Sends a `DELETE` request.
Expand All @@ -421,4 +447,5 @@ def delete(
verify=verify,
timeout=timeout,
trust_env=trust_env,
event_hooks=event_hooks,
)
37 changes: 37 additions & 0 deletions httpx/_client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
DEFAULT_MAX_REDIRECTS,
DEFAULT_TIMEOUT_CONFIG,
UNSET,
EventHooks,
Limits,
Proxy,
Timeout,
Expand Down Expand Up @@ -66,6 +67,7 @@ def __init__(
cookies: CookieTypes = None,
timeout: TimeoutTypes = DEFAULT_TIMEOUT_CONFIG,
max_redirects: int = DEFAULT_MAX_REDIRECTS,
event_hooks: dict = None,
base_url: URLTypes = "",
trust_env: bool = True,
):
Expand All @@ -77,6 +79,7 @@ def __init__(
self._cookies = Cookies(cookies)
self._timeout = Timeout(timeout)
self.max_redirects = max_redirects
self._event_hooks = EventHooks(event_hooks or {})
self._trust_env = trust_env
self._netrc = NetRCInfo()

Expand Down Expand Up @@ -117,6 +120,14 @@ def timeout(self) -> Timeout:
def timeout(self, timeout: TimeoutTypes) -> None:
self._timeout = Timeout(timeout)

@property
def event_hooks(self) -> EventHooks:
return self._event_hooks

@event_hooks.setter
def event_hooks(self, event_hooks: dict) -> None:
self._event_hooks = EventHooks(event_hooks)

@property
def auth(self) -> typing.Optional[Auth]:
"""
Expand Down Expand Up @@ -509,6 +520,7 @@ def __init__(
limits: Limits = DEFAULT_LIMITS,
pool_limits: Limits = None,
max_redirects: int = DEFAULT_MAX_REDIRECTS,
event_hooks: dict = None,
base_url: URLTypes = "",
transport: httpcore.SyncHTTPTransport = None,
app: typing.Callable = None,
Expand All @@ -521,6 +533,7 @@ def __init__(
cookies=cookies,
timeout=timeout,
max_redirects=max_redirects,
event_hooks=event_hooks,
base_url=base_url,
trust_env=trust_env,
)
Expand Down Expand Up @@ -710,6 +723,13 @@ def send(
finally:
response.close()

try:
for hook in self._event_hooks["response"]:
hook(response)
except Exception:
response.close()
raise

return response

def _send_handling_redirects(
Expand Down Expand Up @@ -765,6 +785,10 @@ def _send_handling_auth(

auth_flow = auth.auth_flow(request)
request = next(auth_flow)

for hook in self._event_hooks["request"]:
hook(request)

while True:
response = self._send_single_request(request, timeout)
if auth.requires_response_body:
Expand Down Expand Up @@ -1109,6 +1133,7 @@ def __init__(
limits: Limits = DEFAULT_LIMITS,
pool_limits: Limits = None,
max_redirects: int = DEFAULT_MAX_REDIRECTS,
event_hooks: dict = None,
base_url: URLTypes = "",
transport: httpcore.AsyncHTTPTransport = None,
app: typing.Callable = None,
Expand All @@ -1121,6 +1146,7 @@ def __init__(
cookies=cookies,
timeout=timeout,
max_redirects=max_redirects,
event_hooks=event_hooks,
base_url=base_url,
trust_env=trust_env,
)
Expand Down Expand Up @@ -1312,6 +1338,13 @@ async def send(
finally:
await response.aclose()

try:
for hook in self._event_hooks["response"]:
await hook(response)
except Exception:
await response.aclose()
raise

return response

async def _send_handling_redirects(
Expand Down Expand Up @@ -1367,6 +1400,10 @@ async def _send_handling_auth(

auth_flow = auth.auth_flow(request)
request = next(auth_flow)

for hook in self._event_hooks["request"]:
await hook(request)

while True:
response = await self._send_single_request(request, timeout)
if auth.requires_response_body:
Expand Down
43 changes: 43 additions & 0 deletions httpx/_config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
import typing
import warnings
from base64 import b64encode
from collections.abc import MutableMapping
from pathlib import Path

import certifi
Expand Down Expand Up @@ -410,6 +411,48 @@ def __repr__(self) -> str:
)


class EventHooks(MutableMapping):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think if we apply my suggestion of using a plain Dict[str, List[Callable]] always, we can mostly drop this custom data structure, correct?

I'm also not sure we need to enforce that the only possible keys be request and response. Currently we do this with a bit of a surprising behavior ("don't actually store items for unknown keys"), while I think making it so that event_hooks behaves just like a normal dict would be beneficial.

So perhaps we could replace this with a simple factory helper…

def create_event_hooks(initial: Dict[str, List[Callable]] = None) -> Dict[str, List[Callable]]):
    event_hooks = {"request": [], "response": []}
    event_hooks.update(initial or {})
    return event_hooks

def __init__(self, *args: typing.Any, **kwargs: typing.Any) -> None:
value = dict(*args, **kwargs)
self._dict = {
"request": self._as_list(value.get("request", [])),
"response": self._as_list(value.get("response", [])),
}

def _as_list(self, value: typing.Any) -> list:
if not isinstance(value, (list, tuple)):
return [value]
return list(value)

def __getitem__(self, key: typing.Any) -> typing.List[typing.Callable]:
return self._dict[key]

def __setitem__(
self,
key: str,
value: typing.Union[typing.Callable, typing.List[typing.Callable]],
) -> None:
if key in self._dict:
self._dict[key] = self._as_list(value)

def __delitem__(self, key: str) -> None:
if key in self._dict:
self._dict[key] = []

def __iter__(self) -> typing.Iterator[str]:
return iter(self._dict.keys())

def __len__(self) -> int:
return len(self._dict)

def __str__(self) -> str:
return str(self._dict)

def __repr__(self) -> str:
class_name = self.__class__.__name__
return f"{class_name}({self._dict!r})"


DEFAULT_TIMEOUT_CONFIG = Timeout(timeout=5.0)
DEFAULT_LIMITS = Limits(max_connections=100, max_keepalive_connections=20)
DEFAULT_MAX_REDIRECTS = 20
Loading