Use custom logger with loguru that logs celery task id

[MarcBresson]

Hello,

I have a celery app and I do want to use loguru to handles all the logs (with multiple loggers, and multiples handlers). I'd like all my logs to start with the task id for easier identification (and to benefit from the grep command to have logs from a specific event only). My goal is to have my loguru logger logs that type of message:

celery_task_id_1 2024-02-28 15:46:21 = INFO = the dataset has X lines and Y columns
celery_task_id_1 2024-02-28 15:47:01 = CRITICAL = training of the model failed. Traceback:
...
celery_task_id_2 2024-02-29 09:12:51 = INFO = running analysis

I found this guide https://celery.school/customise-celery-logging but it tries to replace it using the default logging library, while I want to create an entirely new logger with extra attributes and async as allowed by loguru.

My current idea is to bind each and every task (with @app.task(bind=True)) to initialize my logger and storing the task id in it. However, it would force me to initialize my logger for every task which will be cumbersome (I cannot use decorators since @app.task is required to be the last one).

Is there a better way to do something like that?
Will my logs reach stout if I use a custom logger?

Thank you in advance for your time.

[MarcBresson]

Ok, I think I found the right way to do it, tho I have not tested yet. Making everyone of my task inherits a mother class with both a before_start and on_failure method allows me to initialize my logger with the celery task id.

class LoggingTask(Task):
    def before_start(self, task_id, args, kwargs):
        bind_task_id_to_logger(task_id)
        return super().before_start(task_id, args, kwargs)

    def on_failure(self, exc, task_id, args, kwargs, einfo):
        log_critical(exception=einfo)
        return super().on_failure(exc, task_id, args, kwargs, einfo)

def bind_task_id_to_logger(celery_task_id: str):
    """Bind celery task id to logger.

    The task id will be saved in the logger so that it can use it when
    generating logs.

    Parameters
    ----------
    celery_task_id : str
        the celery task id.
    """
    logger.bind(celery_task_id=celery_task_id)