Share a solution about adding single file logging handler instance for both app loggers and uvicorn loggers

[boholder]

boholder/python_web_server_template@006cdc6

Quite a trick but it works. I don't want to create two file logging handlers for one same file, although it works but I'm not sure if this solution is reliable under heavy workload.

How the trick works:

1. Add text style file handler config to uvicorn.config.LOGGING_CONFIG.

FILE_HANDLER_NAME = "file_handler"

FILE_HANDLER_CONFIG = {
    "formatter": "default",
    # ref: https://docs.python.org/3/library/logging.handlers.html#timedrotatingfilehandler
    "class": "logging.handlers.TimedRotatingFileHandler",
    "filename": _LOG_FILE_PATH,
    "when": "midnight",
    "backupCount": 7,
    "encoding": "utf-8",
}

LOGGING_CONFIG["handlers"][FILE_HANDLER_NAME] = FILE_HANDLER_CONFIG 

for logger in LOGGING_CONFIG["loggers"].values():
    # There are three loggers in uvicorn default logging config: "uvicorn", "uvicorn.access", "uvicorn.error".
    # We need to add "file_handler" to logger "uvicorn" and "uvicorn.access"
    # since logger "uvicorn.error" will propagate the log to the other two.
    if "handlers" in logger:
        logger["handlers"] += [FILE_HANDLER_NAME]

2. let the uvicorn starts and initializes the file handler instance.

if __name__ == "__main__": uvicorn.run("main:app", port=8000, reload=True)

3. Retrieve the file handler instance in fastapi's lifecycle function, set

log: logging.Logger

def configure_app_logging(file_handler: logging.Handler) -> None:
    """
    :param file_handler: Reuse the file handler instance created by uvicorn
    """

    console_handler = logging.StreamHandler()
    console_handler.addFilter(TRACE_ID_FILTER)
    logging.basicConfig(
        handlers=[console_handler, file_handler],
        level=LOG_LEVEL,
        format=LOG_FORMAT_PATTERN)


def find_log_file_handler() -> logging.Handler:
    # ref: https://stackoverflow.com/a/55400327/11397457
    l = logging.Logger.manager.loggerDict["uvicorn"]
    for h in l.handlers:
        if isinstance(h, logging.handlers.TimedRotatingFileHandler):
            return h
    raise Exception("Can not found file handler.")


@asynccontextmanager
async def lifespan(app: FastAPI):
    # We must configure app logging after uvicorn started, thus the file handler should exists for reusing.
    log_config.configure_app_logging(find_log_file_handler())
    global log
    log = logging.getLogger(__name__)
    yield

Hope it helps.

PS: Please DO separate fastapi app instance from main module (where calls uvicorn.run()) to ensure there is only one app instance is generated

boholder/python_web_server_template@393d573