startup event lock with multiple uvicorn workers

[gcdsss]

First Check

• I added a very descriptive title to this issue.
• I used the GitHub search to find a similar issue and didn't find it.
• I searched the FastAPI documentation, with the integrated search.
• I already searched in Google "How to X in FastAPI" and didn't find any information.
• I already read and followed all the tutorial in the docs and didn't find an answer.
• I already checked if it is not related to FastAPI but to Pydantic.
• I already checked if it is not related to FastAPI but to Swagger UI.
• I already checked if it is not related to FastAPI but to ReDoc.

Commit to Help

• I commit to help with one of those options 👆

Example Code

import uvicorn
import threading

from typing import Optional
from datetime import datetime

from passlib.context import CryptContext
from beanie import Document, init_beanie
from motor.motor_asyncio import AsyncIOMotorClient
from fastapi import FastAPI


lock = threading.Lock()


app = FastAPI()

pwd_context = CryptContext(schemes=["bcrypt"], deprecated="auto")


class User(Document):
    username: str
    full_name: Optional[str] = None
    email: Optional[str] = None

    class Settings:
        name = "users"

    def __repr__(self) -> str:
        return f"<User {self.username}>"

    def __str__(self) -> str:
        return self.username

    def __hash__(self) -> int:
        return hash(self.username)

    def __eq__(self, other: object) -> bool:
        if isinstance(other, User):
            return self.username == other.username
        return False

    @property
    def created(self) -> datetime:
        return self.id.generation_time

    @classmethod
    async def by_username(cls, username: str) -> "User":
        """Get a user by username"""
        return await cls.find_one(cls.username == username)


class UserPW(User):
    password: str


def hash_password(password: str) -> str:
    return pwd_context.hash(password.encode())


async def create_admin_user(
        username: str = "test",
        password: str = "test"
):
    # lock not working, still create 4 users
    with lock:
        found_user = await User.by_username(username)
        if found_user is not None:
            print("admin user already exists, pass!")
        else:
            user = UserPW(
                username=username,
                password=password
            )
            hashed = hash_password(user.password)
            user.password = hashed
            await user.create()
            return User(**user.dict())


async def init_db(app: FastAPI):
    app.db = AsyncIOMotorClient("mongodb://test:test@127.0.0.1:27017/pml_dev?authSource=admin")
    col = getattr(app.db, "pml_dev")
    await init_beanie(col, document_models=[
        User
    ])

@app.on_event("startup")
async def init():
    # init database
    await init_db(app)
    # create admin user if not exists
    await create_admin_user()


# create 4 duplicate users
# uvicorn main:app --host=0.0.0.0 --port 5000 --workers 4

# only create 1 user on single worker
# uvicorn main:app --host=0.0.0.0 --port 5000

Description

• I am current immigrate flask project to fastapi.
• on startup it will to setup mongodb database and create 1 admin user to provide security feature.
• some api router is sync due to compatibility, to improve performance so i run it on 4 workers.
• but due to multiple workers threads, it create 4 users instead of 1.
• I tried add threading lock or async lock and it's not working, how can we add lock on startup event to make sure only 1 event execute in multiple threading?

Operating System

macOS

Operating System Details

No response

FastAPI Version

0.86.0

Python Version

3.10

Additional Context

No response

[Kludex]

My recommendation is to not use startup for this. Create a separate script for it, that runs before uvicorn.

In case you want to continue with your idea... You can use a redis lock, or file system lock. 👀

[gcdsss]

yes, i will use typer cli to create setup data if this is not doable, i just read startup chapter in fastapi docs and thinking use this to create setup datas.
currently, the threading lock seems not working in uvicorn multiple workers

[NewSouthMjos]

On startup event never should do the migrations! As suggested above, use the bash script for starting the application like this:

#!/bin/bash
# alembic upgrade head
python3 migrations/migrate.py
WORKERS_COUNT="${WORKERS_COUNT:=4}"
exec uvicorn main:app --host=0.0.0.0 --port=80 --workers=$WORKERS_COUNT

If you using docker, set the entrypoint to this bash script.

[gcdsss]

ok, i will use click to do migrations, the startup event i just use for beaine init, close it.