Async doesn't work - is sequential instead

[thomasfrederikhoeck]

This issue is for a: (mark with an x)

- [x] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

The app doesn't handle multiple request with async. If you add a asyncio.sleep(1) like this it will only be able to handle 1 request per second.

import azure.functions as func
from FastAPIApp import app  # Main API application
import nest_asyncio
import asyncio
nest_asyncio.apply()

@app.get("/sample")
async def index():
  await asyncio.sleep(1)
  return {
      "info": "Try /hello/Shivani for parameterized route.",}

async def main(req: func.HttpRequest, context: func.Context) -> func.HttpResponse:
    return func.AsgiMiddleware(app).handle(req, context)

Which is in contrast to following which can take a large amount of requests.

import azure.functions as func

async def main(req: func.HttpRequest, context: func.Context) -> func.HttpResponse:
  print("called 1")
  await asyncio.sleep(1)
  return "Try /hello/Shivani for parameterized route."

You can test it out with locust using the following:

1. pip install locust
2. create file locustfile.py

##  locustfile.py
from locust import HttpUser, task

class HelloWorldUser(HttpUser):
    @task
    def hello_world(self):
        self.client.get("/sample")

3. run locust

Any log messages given by the failure

Expected/desired behavior

That the async would work.

OS and Version?

Windows 10. Linux

Versions

fastapi==0.80.0
azure-functions== 1.11.2

Mention any other details that might be useful

---

Thanks! We'll be in touch soon.

[thomasfrederikhoeck]

@shreyabatra4 I don't know if you are the correct to tag but maybe you can't point me in the correct direction ? :-)

[tonybaloney]

@thomasfrederikhoeck we discussed this issue today and are working on a fix, we'll update the issue with some more details soon

[thomasfrederikhoeck]

@tonybaloney sounds good - looking forward to hearing more!

[thomasfrederikhoeck]

I saw that someone used some of the hidden functions to make it work

import azure.functions as func
from azure.functions._http_asgi import AsgiResponse, AsgiRequest
from FastAPIApp import app

async def handle_asgi_request(req: func.HttpRequest, context: func.Context) -> func.HttpResponse:
    asgi_request = AsgiRequest(req, context)
    scope = asgi_request.to_asgi_http_scope()
    asgi_response = await AsgiResponse.from_app(app, scope, req.get_body())
    return asgi_response.to_func_response()


async def main(req: func.HttpRequest, context: func.Context) -> func.HttpResponse:
    return await handle_asgi_request(req, context)

[tonybaloney]

Yes, we're going to implement something similar and add it to the AsgiMiddleware class. It'll have to be a new API for backward compatibility with the old one

[tonybaloney]

Issue has been patched in dev, this sample repo will be updated once the client-side package updates are rolled out.

Azure/azure-functions-python-library#143

We've also added a warning for use of the existing .handle() method, as this is unlikely to be what people need.

If you want to try it out, you can pip install the branch directly from GitHub in the requirements.txt for your project.

[thomasfrederikhoeck]

@tonybaloney nice! Thank you! As I see it also removes the need for nest_asyncio which is required by the old one: (see https://learn.microsoft.com/en-us/samples/azure-samples/fastapi-on-azure-functions/azure-functions-python-create-fastapi-app/). So much cleaner :-)

[gavin-aguiar]

This change is rolled out with azure functions version 1.12.0. Thanks @tonybaloney for the fix.

[tonybaloney]

@gavin-aguiar outstanding items

• Update template Add async wrapper #6
• Update docs (will do separately)

[tblatrille]

The change to fix this (Add async wrapper PR) generates an exception, when rolling back it does work. Does it work for you?

Result: Failure Exception: AttributeError: 'AsgiMiddleware' object has no attribute 'handle_async' Stack: File "/azure-functions-host/workers/python/3.9/LINUX/X64/azure_functions_worker/dispatcher.py", line 444, in _handle__invocation_request call_result = await self._run_async_func( File "/azure-functions-host/workers/python/3.9/LINUX/X64/azure_functions_worker/dispatcher.py", line 697, in _run_async_func return await ExtensionManager.get_async_invocation_wrapper( File "/azure-functions-host/workers/python/3.9/LINUX/X64/azure_functions_worker/extension.py", line 147, in get_async_invocation_wrapper result = await function(**args) File "/home/site/wwwroot/WrapperFunction/__init__.py", line 21, in main return await func.AsgiMiddleware(app).handle_async(req, context)

[gavin-aguiar]

@tblatrille Which azure functions version are you using? The handle_async() method is in azure functions version 1.12.0 and up.

[tblatrille]

@gavin-aguiar I actually have azure-functions==1.12.0 in the requirements.txt

[makalekseev]

@tblatrille try to update azure-functions-core-tools to latest version

[tonybaloney]

And set the environment variable PYTHON_ISOLATE_WORKER_DEPENDENCIES=1 if that doesn't work. There's a more detailed discussion on #7