Error while retrieving session from request in middleware in async environment

[jvdboog]

Hello!

I get the following error when I upgraded django-allauth from 55.2 to 60.1.

django  | 2024-01-22 12:51:24,637 ERROR    Internal Server Error: /cms/sprite-f0864303/
django  | Traceback (most recent call last):
django  |   File "/usr/local/lib/python3.11/site-packages/django/contrib/sessions/backends/base.py", line 187, in _get_session
django  |     return self._session_cache
django  |            ^^^^^^^^^^^^^^^^^^^
django  | AttributeError: 'SessionStore' object has no attribute '_session_cache'
django  | 
django  | During handling of the above exception, another exception occurred:
django  | 
django  | Traceback (most recent call last):
django  |   File "/usr/local/lib/python3.11/site-packages/asgiref/sync.py", line 534, in thread_handler
django  |     raise exc_info[1]
django  |   File "/usr/local/lib/python3.11/site-packages/django/core/handlers/exception.py", line 42, in inner
django  |     response = await get_response(request)
django  |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
django  |   File "/usr/local/lib/python3.11/site-packages/allauth/account/middleware.py", line 18, in middleware
django  |     _remove_dangling_login(request, response)
django  |   File "/usr/local/lib/python3.11/site-packages/allauth/account/middleware.py", line 52, in _remove_dangling_login
django  |     if "account_login" in request.session:
django  |        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
django  |   File "/usr/local/lib/python3.11/site-packages/django/contrib/sessions/backends/base.py", line 50, in __contains__
django  |     return key in self._session
django  |                   ^^^^^^^^^^^^^
django  |   File "/usr/local/lib/python3.11/site-packages/django/contrib/sessions/backends/base.py", line 192, in _get_session
django  |     self._session_cache = self.load()
django  |                           ^^^^^^^^^^^
django  |   File "/usr/local/lib/python3.11/site-packages/django/contrib/sessions/backends/db.py", line 42, in load
django  |     s = self._get_session_from_db()
django  |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
django  |   File "/usr/local/lib/python3.11/site-packages/django/contrib/sessions/backends/db.py", line 32, in _get_session_from_db
django  |     return self.model.objects.get(
django  |            ^^^^^^^^^^^^^^^^^^^^^^^
django  |   File "/usr/local/lib/python3.11/site-packages/django/db/models/manager.py", line 87, in manager_method
django  |     return getattr(self.get_queryset(), name)(*args, **kwargs)
django  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
django  |   File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 643, in get
django  |     num = len(clone)
django  |           ^^^^^^^^^^
django  |   File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 380, in __len__
django  |     self._fetch_all()
django  |   File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 1926, in _fetch_all
django  |     self._result_cache = list(self._iterable_class(self))
django  |                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
django  |   File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 91, in __iter__
django  |     results = compiler.execute_sql(
django  |               ^^^^^^^^^^^^^^^^^^^^^
django  |   File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1560, in execute_sql
django  |     cursor = self.connection.cursor()
django  |              ^^^^^^^^^^^^^^^^^^^^^^^^
django  |   File "/usr/local/lib/python3.11/site-packages/django/utils/asyncio.py", line 24, in inner
django  |     raise SynchronousOnlyOperation(message)
django  | django.core.exceptions.SynchronousOnlyOperation: You cannot call this from an async context - use a thread or sync_to_async.

After further investigation the cause of this is the following:

    if not getattr(request, "_account_login_accessed", False):
        if "account_login" in request.session:
            request.session.pop("account_login")

In the AccountMiddleware. Trying to access session is a (sync) database request. However, since I am running it from an async environment, Django throws an error. Trying to find a solution is hard, since I cannot remove AccountMiddleware, since Django won't start due to an Improperly configured error.

Is there anything I could do to solve this?

Thanks!

[pennersr]

I can confirm the issue. It occurs when the session has not been loaded yet, and the middleware is the first one to trigger the actual load. Possibly related:

https://code.djangoproject.com/ticket/34901

Trouble is, there is no way of triggering the load of a session asynchronously due to the ticket above, and using sync_to_async in the middleware does not sound like a plan as the middleware always kicks in, meaning all views would be hampered by this.

TBD what to do.

[pennersr]

Here -- https://github.com/django/django/pull/17372/files -- request.session.ahas_key() is being introduced, which is what the middleware should be using. Looks like this issue is blocked on missing Django functionality.

[pennersr]

Given that the Django pull request is using sync_to_async() under the hood as well, see #3614

Can you give this a try @jvdboog ?

[bkbirr]

I have been seeing the same error. The #3614 update fixed it. Thanks!

[pennersr]

Merged.

[pennersr]

@bkbirr @jvdboog I would appreciate it if you could comment over at #3546 what, according to you, is missing / needs to be done in order to improve interop with async. If all is simply working as is for you, then that is also valuable information to hear.