What is the best practice for enabling runtime config from command line during fast-api app launch?

[napsternxg]

First Check

• I added a very descriptive title to this issue.
• I used the GitHub search to find a similar issue and didn't find it.
• I searched the FastAPI documentation, with the integrated search.
• I already searched in Google "How to X in FastAPI" and didn't find any information.
• I already read and followed all the tutorial in the docs and didn't find an answer.
• I already checked if it is not related to FastAPI but to Pydantic.
• I already checked if it is not related to FastAPI but to Swagger UI.
• I already checked if it is not related to FastAPI but to ReDoc.

Commit to Help

• I commit to help with one of those options 👆

Example Code

def main(args):
    model_path = args[1]
    model = load_model(model_path)
    app = FastAPI()
    
    @app.get("/{item_id}")
    def predict_item(item_id: int):
        return dict(item_id=item_id, prediction=model(item_id))
    
    uvicorn.run(app)

if __name__ == "__main__":
    args = sys.argv
    print(args)
    main(args)

Description

Hi
FastAPI team,

I just started using FastAPI and I am liking it.
I would like to know what is the best practice for enabling runtime config from command line during fast-api app launch.
I would like to have a library which I can share with my team mates so that they can serve their models via fast-api by just changing the model path.

However, right now I am creating the app = FastApi() inside a function e.g. main, which can read the command line args and then and defining all the app end points under this main function. This makes it harder to test the app endpoint functions defined inside main.

I am doing something like the code example above. But I think there must be a better way.

I looked at FastAPI docs and I saw from fastapi import Depends may help with this but I am not sure how to use it for the setup I have above.
Effectively I would like to know if there is a pattern which is cleaner, e.g. something like this:

app = FastAPI()
    
@app.get("/{item_id}", model = Depends(something))
def predict_item(item_id: int):
   # Somehow get this model properly defined outside of main
    return dict(item_id=item_id, prediction=model(item_id))



def main(args):
    model_path = args[1]
    model = load_model(model_path)

   # configure app to use model. 
   
    
    uvicorn.run(app)

if __name__ == "__main__":
    args = sys.argv
    print(args)
    main(args)

Also asked this question on Twitter at @FastAPI: https://twitter.com/FastAPI/status/1505619967311499264

Operating System

Linux

Operating System Details

No response

FastAPI Version

0.75.0

Python Version

3.7.11

Additional Context

No response

[kheealam]

You could try using environment variables for that. For example:

app = FastAPI()
    
@app.get("/{item_id}", model = Depends(something))
def predict_item(item_id: int):
   # Somehow get this model properly defined outside of main
    return dict(item_id=item_id, prediction=model(item_id))



def main(args):
    model_path = os.environ.get("MODEL_PATH")
    model = load_model(model_path)

   # configure app to use model. 
   
    
    uvicorn.run(app)

if __name__ == "__main__":
    args = sys.argv
    main(args)

[acidjunk]

We use a settings class that has defaults and can be overriden by ENV.

import secrets
from pydantic import BaseSettings


class AppSettings(BaseSettings):
    TESTING: bool = True
    SESSION_SECRET: str = "".join(secrets.choice(string.ascii_letters) for i in range(16))  # noqa: S311
    CORS_ORIGINS: str = "*"
    CORS_ALLOW_METHODS: List[str] = ["GET", "PUT", "PATCH", "POST", "DELETE", "OPTIONS", "HEAD"]
    CORS_ALLOW_HEADERS: List[str] = ["If-None-Match", "Authorization", "If-Match", "Content-Type"]
    CORS_EXPOSE_HEADERS: List[str] = [
        "Cache-Control",
        "Content-Language",
        "Content-Length",
        "Content-Type",
        "Expires",
        "Last-Modified",
        "Pragma",
        "Content-Range",
        "ETag",
    ]
    ENVIRONMENT: str = "local"
    DATABASE_URI: str = "postgresql://user:pas@localhost/orchestrator-core"
    CACHE_HOST: str = "127.0.0.1"
    CACHE_PORT: int = 6379
    SERVICE_NAME: str = "orchestrator-core"
    LOGGING_HOST: str = "localhost"
    LOG_LEVEL: str = "DEBUG"
    WEBSOCKET_BROADCASTER_URL: str = "memory://"
    ENABLE_WEBSOCKETS: bool = True
    
app_settings = AppSettings()   

in main.py

import app_settings from settings

if app_settings.ENABLE_WEBSOCKETS:
    print("Do something with WS")

Pydantic ensure that you get validation for you config + the mentioned ENV support. E.g. every parameter can be set via ENV; and will have sensible defaults for development.

[mcobzarenco]

We use a settings class that has defaults and can be overriden by ENV.

This forces you to have a constructor with no arguments or set all the fields to dummy variables. Which is kind of clunky, not type safe and goes against strongly typed APIs that FastAPI otherwise emphasises. E.g. imagine having required arguments for the cli.

[napsternxg]

Thanks @kheealam and @acidjunk yes I have tried the ENV approach but I was looking for something more configurable via command line args. My current setup allows me to read the command line args but I don't like encapsulating making the app inside a function, hence I was hoping if there was a better way.

I like the AppSettings approach. Is it possible to update AppSettings via command line args instead of ENV vars?

[acidjunk]

I think you could do something by inheriting from BaseSettings and implement CLI overridable defaults there. We use it in a k8s context were everything is configured via ENV vars.

[napsternxg]

Hi @acidjunk thanks that is very useful. let me give it a shot.

[srcortez45]

i did something similar related to config file, it load base on the environment

class AppConfig(BaseModel):
    
    DESCRIPTION: str = 'API'
    VERSION: float = 1.0
    PORT: int = 8000


class GlobalConfig(BaseSettings):

    APP_CONFIG: AppConfig = AppConfig()

    ENV_STATE: Optional[str] = Field(None,env="ENV_STATE")

    class Config:

        env_file: str = ".env"

class DevConfig(GlobalConfig):

    TITLE: str = 'DEV'

    HOST: str = "localhost"

    DEBUG:bool = True

    BASE_PATH:str = os.getcwd()

    origins: str = "*"

    openapi_url:Optional[str] = '/api/openapi'

    docs_url:Optional[str] = '/docs'
    
    redoc_url:Optional[str] = '/redoc'

    proxy_headers: bool = False

    forwarded_allow_ips: List[str] = ["*"]

    WHITELISTED_IPS: List[str] = ["*"]

    class Config:
        env_prefix: str = "dev."


class ProdConfig(GlobalConfig):
    
    TITLE: str = 'PROD'

    HOST: str = "localhost"

    DEBUG:bool = False

    BASE_PATH:str = 'path'

    origins:str = '*'

    openapi_url:Optional[str] = 'path'

    docs_url:Optional[str] = 'path'
    
    redoc_url:Optional[str] = None

    proxy_headers: bool = True

    forwarded_allow_ips: List[str] = ["*"]

    WHITELISTED_IPS: List[str] = ["*"]

    class Config:
        env_prefix: str = "prod."

class FactoryConfig:

    def __init__(self, env_state: Optional[str]):
        self.env_state = env_state

    def __call__(self):
        if self.env_state == "dev":
            return DevConfig()

        elif self.env_state == "prod":
            return ProdConfig()

cnf = FactoryConfig(GlobalConfig().ENV_STATE)()