ASGI spec: Connections, exceptions, and task management.

[lovelydinosaur]

Some questions that may or may not necessarily need be part of the ASGI (http) spec, but that might need implementor guidelines...

Trying to be fairly complete here in thinking about potential edge cases around uvicorn's handling:

• Should an ASGI server be responsible for detecting & handling applications that raise an exception?
• Should an ASGI server be responsible for detecting & handling applications that complete without sending a response?
• Should an ASGI server be responsible for handling applications that do not complete, after sending a response? (Within some given timeframe.)
• Should an ASGI server be responsible for timing out slow-returning responses?
• How should an ASGI server be expected to act if an application raises an exception once a response is partially sent?
• How should an ASGI server be expected to act if an application raises an exception once a response is completely sent?
• Should an ASGI application wrap and handle exceptions themselves as 500 errors, or just propagate them to the server?
• Should an ASGI application wrap and handle exceptions that occur once a response is partially sent, and if so how should it communicate it to the server?
• Should an ASGI application wrap and handle exceptions that occur once a response is already fully sent, and if so how should it communicate it to the server?
• How should send be expected to behave in the event of a closed or closing connection?
• How should receive be expected to behave in the event of a closed or closing connection?
• Should a connection becoming closed cancel any outstanding applications, or allow them to run to completion?

Not sure which of these do or don't need to be explicit, but think it's worth at least opening this up for discussion here.

[andrewgodwin]

My thoughts:

• The ASGI server should do last-ditch exception handling and log errors to console (and down the protocol if it can). Any exception that gets to it should close the connection.
• Completing without sending a response should just close the connection. If that's bad protocol handling, that's the application's choice.
• Applications should be able to hold a connection open basically forever (I recently removed the HTTP timeout from Daphne). It would be wise to have an option to kill them after so many minutes though if people want to turn that on. Same for slow-returning responses.
• Once a response is partly or completely sent, the server should log the error to an error log and hard-close the connection.
• ASGI servers should return a 500 response if an error gets to them and the response is not started, but it should be very basic (like the default WSGI errors) and applications are encouraged to have better 500s. Should not show the traceback in the response unless it's turned on for debugging/dev.
• If an application has an exception after response is started, it should log the error to wherever it logs errors and let an exception bubble up to close the connection harshly (or do that itself if it's capable)
• When a connection is closing or closed, send() should return an error
• receive() should return the disconnect/close event if it has not already, and then raise an exception
• A connection being closed (either expected or unexpectedly) should give the application coroutines a grace period to exit (10 seconds on Daphne by default) and if they're still alive, then kill them.

[lovelydinosaur]

Thanks, those all sound like reasonable behaviors. Pending one more point below I'll close this off, since I don't think they strictly need to be mentioned at this point, but I suppose some of these aspects might end up in as some kind of "implementation recommendations" further down the line.

One point of clarification:

Should an ASGI server be responsible for handling applications that do not complete, after sending a response?

Applications should be able to hold a connection open basically forever (I recently removed the HTTP timeout from Daphne). It would be wise to have an option to kill them after so many minutes though if people want to turn that on. Same for slow-returning responses.

The case I'm considering there is the "application has sent a response, and is now doing some deferred work". If the server allows those cases to continue indefinitely, then do we need to be concerned about the potential for leaking tasks. (Eg. Gradually ballooning memory because we've got an application that responses quickly to the requests initially, but then spends a long amount of time handling some post-response behavior.) The application doesn't actually need an open connection at that point.

It's also slightly at odds with:

A connection being closed (either expected or unexpectedly) should give the application coroutines a grace period to exit (10 seconds on Daphne by default) and if they're still alive, then kill them.

Which implies that any background processing being viable or not depends on if the connection happens to close after the response or not.

There's also a question around sensible behaviors for ordering there. In uvicorn pipelined requests are handled by the parser, and get queued up, but requests/responses against the connection are made strictly sequentially (we don't start task 2 until task 1 has sent it's response, even though we could if we made sure to pipeline the data back in the correct order.) The implementation could be even stricter, in not allowing task 2 to start until task 1 on the same connection has completed. (ie not only has the response been sent, but the task has exited) It's not obvious to me if that'd be desirable behavior or not. ( I guess not, and that there's an expectation that a response might be followed by a slow-running background task, while concurrently starting another request/response on the same connection)

[andrewgodwin]

The case I'm considering there is the "application has sent a response, and is now doing some deferred work". If the server allows those cases to continue indefinitely, then do we need to be concerned about the potential for leaking tasks.

Which implies that any background processing being viable or not depends on if the connection happens to close after the response or not.

I think the way I'm thinking about this is that, once you've finished sending a response, the webserver should "close the connection" anyway (if there's keepalive or HTTP2 it's not a closed connection really, but it is in terms of the ASGI scope abstraction).

Thus, I'm basically of the opinion that you get the request/response lifecycle plus a small grace period to do processing. If you want to do actual true background processing, then that needs to be managed separately from an ASGI-HTTP context IMO.

[lovelydinosaur]

Thus, I'm basically of the opinion that you get the request/response lifecycle plus a small grace period to do processing.

Indeed. At some point I might put a timeout on task-lifetimes-after-response. I don't much like the idea of allowing tasks to hang around indefinitely without ever collecting them, since it seems like a good route onto silently leaking resources.

I'll close this off for now. Thanks as ever for your time! 😃

[andrewgodwin]

No problem, thanks for the questions, it's nice to think about these things :)