How to know the size of the threadpool ? and why --workers makes no difference on concurrency

[yyfhust]

First Check

• I added a very descriptive title to this issue.
• I used the GitHub search to find a similar issue and didn't find it.
• I searched the FastAPI documentation, with the integrated search.
• I already searched in Google "How to X in FastAPI" and didn't find any information.
• I already read and followed all the tutorial in the docs and didn't find an answer.
• I already checked if it is not related to FastAPI but to Pydantic.
• I already checked if it is not related to FastAPI but to Swagger UI.
• I already checked if it is not related to FastAPI but to ReDoc.

Commit to Help

• I commit to help with one of those options 👆

Example Code

uvicorn run:app --host 0.0.0.0 --port 8000 --workers 2

Description

Hi,
How to know the size of the thread pool ? And what is the use of --workers in uvicorn

I have tested locally:
I used uvicorn to run fastAPI based backend locally on my Mac, and I can see that it can create up to 6 distinct threads ( the server gets blocked when I sent more request ).
I also tried to run the backend with different --workers, but there was no difference at all, the threads limit was always 6 no matter how many workers I specified.
Commend I have run:
uvicorn run:app --host 0.0.0.0 --port 8000 --workers 1
uvicorn run:app --host 0.0.0.0 --port 8000 --workers 2
uvicorn run:app --host 0.0.0.0 --port 8000 --workers 10

Operating System

macOS

Operating System Details

Macpro 2020

FastAPI Version

0.70.0

Python Version

Python 3.7.9

Additional Context

No response

[jgould22]

You will have to be more specific as to which thread pool you are referring to.

[raphaelauv]

uvicorn doc -> https://www.uvicorn.org/settings/#production

then if you don't understand how python and asyncio work -> #4265 (comment)

[edbird]

The above linked doc page doesn't give an answer to this question. If you write an endpoint using a synchronous def function rather than an asynchronous async def function, then FastAPI will run this synchronous function inside of a threadpool:

• How do you configure the size of that thread pool?
• What is the default size?
• How can you check the size at runtime?

None of these questions are answered there.

[YuriiMotov]

It defaults to 40. You can change it with RunVar("_default_thread_limiter").set(CapacityLimiter(THREADPOOL_SIZE)):

from contextlib import asynccontextmanager
import random
import time
from anyio.lowlevel import RunVar
from anyio import CapacityLimiter
from fastapi import FastAPI

THREADPOOL_SIZE = 2


@asynccontextmanager
async def lifespan(app: FastAPI):
    RunVar("_default_thread_limiter").set(CapacityLimiter(THREADPOOL_SIZE))
    yield

app = FastAPI(lifespan=lifespan)


@app.get("/")
def index():
    i = random.randint(0, 1000)
    print('start', i)
    time.sleep(10)
    print('finish', i)