TypeError: 'NoneType' object is not iterable

[allen-munsch]

Thanks for sharing this library. Very interesting!

Any suggestions on why NoneType would be returned from async_all?

web              |   File "/venv/lib/python3.8/site-packages/fastapi/applications.py", line 199, in __call__
web              |     await super().__call__(scope, receive, send)
web              |   File "/venv/lib/python3.8/site-packages/starlette/applications.py", line 112, in __call__
web              |     await self.middleware_stack(scope, receive, send)
web              |   File "/venv/lib/python3.8/site-packages/starlette/middleware/errors.py", line 181, in __call__
web              |     raise exc from None
web              |   File "/venv/lib/python3.8/site-packages/starlette/middleware/errors.py", line 181, in __call__
web              |     raise exc from None
web              |   File "/venv/lib/python3.8/site-packages/starlette/exceptions.py", line 93, in __call__
web              |     await response(scope, receive, sender)
web              |   File "/venv/lib/python3.8/site-packages/starlette/exceptions.py", line 93, in __call__
web              |     await response(scope, receive, sender)
web              |   File "/venv/lib/python3.8/site-packages/starlette/routing.py", line 580, in __call__
web              |     await route.handle(scope, receive, send)
web              |   File "/venv/lib/python3.8/site-packages/starlette/routing.py", line 241, in handle
web              |     await self.app(scope, receive, send)
web              |   File "/venv/lib/python3.8/site-packages/starlette/routing.py", line 52, in app
web              |     response = await func(request)
web              |   File "/venv/lib/python3.8/site-packages/fastapi/routing.py", line 201, in app
web              |     raw_response = await run_endpoint_function(
web              |   File "/venv/lib/python3.8/site-packages/fastapi/routing.py", line 148, in run_endpoint_function
web              |     return await dependant.call(**values)
web              |   File "/app/api/generated_app/main.py", line 100, in get_buildings
web              |     data = await get_data()
web              |   File "/app/api/generated_app/main.py", line 88, in get_data
web              |     async for building in data:
web              |   File "/venv/lib/python3.8/site-packages/django_async_orm/query.py", line 71, in __aiter__
web              |     return AsyncIter(self._result_cache)
web              |   File "/venv/lib/python3.8/site-packages/django_async_orm/iter.py", line 6, in __init__
web              |     self._iter = iter(iterable)
web              | TypeError: 'NoneType' object is not iterable

py3.8.10
django3.2

async def get_data():
    data = await Building.objects.async_all()
    result = []
    async for building in data:
        print(data, dir(data))
        print(building.__dict__)
        result.append({"id": building.id, "number": building.number})
    return result

@app.get("/buildings", response_model=List[BuildingSummary])
async def get_buildings(limit: conint(le=500), skip: conint(le=500)) -> List[BuildingSummary]:
    """
    Building List
    """
    # this is a custom thing
    data = await get_data()
    return data

[rednaks]

Thank you for reporting the issue, the only reason I can see here is that the thread executor to fetch all data when the iterator is call didn't wait to create the iterator. does this happen all the time ?

I couldn't reproduce with my tests but looking at the call stack and the routing I don't see a typical django app. can you tell me more about your use case ? it seems you're using starlette for routing do you have any idea how is working with django and the async world ?

What I can do to protect the code, is to try to block the async call, because in this specific use case I'm using threads to create the iterator.

    def __aiter__(self):
        with concurrent.futures.ThreadPoolExecutor(max_workers=1) as executor:
            executor.submit(self._fetch_all)

        return AsyncIter(self._result_cache)

Also, if you try to print(data) before iterating, it will fetch data because it's already blocking.

[allen-munsch]

The code is using something like this:

• https://github.com/allen-munsch/benchmark-django-fastapi/blob/main/testdjango/wsgi_with_slow_api_mounted.py#L25

Not entirely sure why, but nesting the function, or creating a closure around it seemed to work.

@app.get("/buildings", response_model=List[BuildingSummary])
async def get_buildings(limit: conint(le=500), skip: conint(le=500)) -> List[BuildingSummary]:
    """
    Building List
    """
    # this is a custom thing
    async def get_data():
        data = await Building.objects.async_all()
        result = []
        for building in data:
            print(data, dir(data))
            print(building.__dict__)
            result.append({"id": building.id, "number": building.number})
         return result

    data = await get_data()
    return data

closing this issue, since something else is going on.

Cheers. Thanks for sharing the code.

[rednaks]

that's interesting, but still an issue I think would you try to patch the code locally and test ? I don't think people would make a closure as a workaround.

Also I would be interested to know the performance difference between the workaround and the blocking one.

Thanks for your help.

diff --git a/django_async_orm/query.py b/django_async_orm/query.py
index 5df0118..90f12d3 100644
--- a/django_async_orm/query.py
+++ b/django_async_orm/query.py
@@ -66,7 +66,8 @@ class QuerySetAsync(QuerySet):
 
     def __aiter__(self):
         with concurrent.futures.ThreadPoolExecutor(max_workers=1) as executor:
-            executor.submit(self._fetch_all)
+            f = executor.submit(self._fetch_all)
+            f.result()
 
         return AsyncIter(self._result_cache)