Populate "endpoint" key in ASGI scope

[simonw]

This is a trick used by Starlette so that other layers of ASGI middleware can see which route was selected.

They added it here: encode/starlette@34d0097

If Datasette supports it as well we can benefit from it if we integrate this sentry_asgi middleware (probably as a datasette-sentry plugin): https://github.com/encode/sentry-asgi/blob/c6a42d44d31f85885b79e4ee898683ecf8104971/sentry_asgi/middleware.py#L34-L35

[SteadBytes]

It looks as if the datasette.utils.AsgiRouter.__call__ is the place to add this

datasette/datasette/utils/asgi.py

Line 101 in 90d4f49

return await view(new_scope, receive, send)

.

The sentry_asgi middleware uses the __qualname__ or __name__ attributes of the endpoint https://github.com/encode/sentry-asgi/blob/c6a42d44d31f85885b79e4ee898683ecf8104971/sentry_asgi/middleware.py#L84

Looking at the Starlette implementation endpoint is a Callable encode/starlette@34d0097#diff-34fba745b50527bfb4245d02afd59246R100 which as far as I can tell is analogous to the view function which is matched here

datasette/datasette/utils/asgi.py

Line 96 in 90d4f49

for regex, view in self.routes:

.

A slight issue is that __qualname__ is matched first in the sentry_asgi middleware, and __name__ is used if that doesn't exist. I think (please correct me if I am wrong) that for datasette, the __name__ is what should be used. For example, when using the development fixtures and hitting http://127.0.0.1:8001/fixtures/compound_three_primary_keys the view function that is matched gives:

>>> view.__qualname__
'AsgiView.as_asgi.<locals>.view'
>>> view.__name__
'TableView'

Would TableView be the desired value here? Or am I looking in entirely the wrong place? 😄

[simonw]

Yes, TableView is desirable here I think. Maybe datasette.views.table.TableView if we can get that somehow. Thanks for taking a look!

[SteadBytes]

Ok great, getting the __qualname__ to be TableView and adding endpoint to the scope in AsgiRouter is simple enough (already done). However, (unless I'm missing a plugin hook or something) the suggestion of utilising it within a datasette-sentry plugin may not work. The only hook that would have access to the scope is the asgi_wrapper hook. But as this wraps the existing asgi app, the endpoint won't yet have been added to the scope received by the hook https://github.com/SteadBytes/datasette/blob/107d47567dedd472eebec7f35bc34f5b58285ba8/datasette/app.py#L672 . However, I'm not sure where else the endpoint could be added to the asgi scope 🤔

[simonw]

Yeah that's a good call: the Datasette plugin mechanism where middleware is wrapped around the outside doesn't appear to be compatible with the Sentry mechanism of expecting that scope has been populated before it gets to their error handler.

@tomchristie is this something you've thought about?

[simonw]

Asked about this on Twitter: https://twitter.com/simonw/status/1152238730259791877

[tomchristie]

The middleware implementation there works okay with a router nested inside if the scope is mutated. (Ie. "endpoint" doesn't need to exist at the point that the middleware starts running, but if it has been made available by the time an exception is thrown, then it can be used.)

Starlette's usage of "endpoint" there is unilateral, rather than something I've discussed against the ASGI spec - certainly it's important for any monitoring ASGI middleware to be able to have some kind of visibility onto some limited subset of routing information, and "endpoint" in the scope referencing some routed-to callable seemed general enough to be useful.

[simonw]

Huh, interesting. I'd got it into my head that scope should not be mutated under any circumstances - if that's not true and it's mutable there's all kinds of useful things we could do with it.

[simonw]

It strikes me that if scope is indeed meant to stay immutable the alternative way of solving this would be to add an outbound custom request header with the endpoint - X-Endpoint: datasette.views.table.TableView for example - and teach the Sentry plugin to optionally read that.

[SteadBytes]

The asgi spec doesn't explicitly specify (at least as far as I can tell) whether the scope is immutable/mutable https://asgi.readthedocs.io/en/latest/specs/lifespan.html#scope . @simonw using a header for this would be a nice approach. It would also potentially increase the portability of any middleware/plugins/clients across different applications/frameworks as it's not tied directly to an asgi implementation

[tomchristie]

Right now the spec does say “copy the scope, rather than mutate it” https://asgi.readthedocs.io/en/latest/specs/main.html#middleware

I wouldn’t be surprised if that there’s room for discussion on evolving the exact language there.

There’s obvs a nice element to the strictness there, tho practically I’m not sure it’s something that implementations will follow, and its not something that Starlette chooses to abide by.

[SteadBytes]

Oh yes well spotted thank you 😁

I agree that the strictness would be nice as it could help to avoid different middleware altering the scope in incompatible ways. However I do also agree that it's likely for not all implementations to follow 🤔

[SteadBytes]

@simonw do you think it is still worth populating the endpoint key in the scope as originally intended by this issue, or should we hold off until a decision about possibly using an X-Endpoint header instead? 😄