Repository navigation
Native ASGI worker leaks active connection count and delays SIGTERM shutdown[Triage] #3659
frankxzw
started this conversation in
Issue Triage
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Type
Bug Report
Description
The native ASGI worker does not decrement
worker.nr_connswhen the server closes a transport.After at least one HTTP request has been handled, sending SIGTERM to the Gunicorn master causes the worker to wait for the entire
graceful_timeout, even though the client connection has already been closed and there are no in-flight requests.Steps to Reproduce (for bugs)
--asgi-loop auto)Minimal application
Configuration
Start Gunicorn: gunicorn app:app \ --worker-class asgi \ --workers 1 \ --bind 127.0.0.1:8000 \ --keep-alive 0 \ --graceful-timeout 3 \ --no-control-socket \ --pid /tmp/gunicorn-asgi.pid \ --access-logfile - Send one request and explicitly close the connection: ```bash curl --http1.1 -H 'Connection: close' http://127.0.0.1:8000/ kill -TERM "$(cat /tmp/gunicorn-asgi.pid)"Gunicorn Version
gunicorn 26.0.0
Python Version
Python 3.12.13
Worker Class
asgi (beta)
Operating System
Ubuntu 24.04.4 LTS
Additional Context
Expected behavior
Once the transport has closed and there are no in-flight requests,
worker.nr_connsshould return to zero and SIGTERM should not consume the entire graceful timeout.Suspected root cause
ASGIProtocol.connection_made()increments the connection count:https://github.com/benoitc/gunicorn/blob/26.0.0/gunicorn/asgi/protocol.py#L389-L392
_close_transport()closes the transport and immediately sets_closed = True:https://github.com/benoitc/gunicorn/blob/26.0.0/gunicorn/asgi/protocol.py#L1450-L1464
When asyncio subsequently invokes
connection_lost(), it returns early because_closedis already true. Therefore,worker.nr_conns -= 1is never executed:https://github.com/benoitc/gunicorn/blob/26.0.0/gunicorn/asgi/protocol.py#L672-L687
The ASGI worker then waits on the leaked count during graceful shutdown:
https://github.com/benoitc/gunicorn/blob/26.0.0/gunicorn/workers/gasgi.py#L239-L251
A direct state-level reproduction gives:
Possible fix
Connection accounting should use a state flag separate from
_closed, for example_connection_lost_handled, soconnection_lost()decrementsnr_connsexactly once even if_close_transport()has already marked the transport closed.A regression test could verify that:
connection_made()incrementsnr_conns._close_transport()followed byconnection_lost()decrements it to zero.graceful_timeout.Checklist
All reactions