Skip to content

Unable to change log timestamp formats using logging config class or log formatter class #62937

@jnstockley

Description

@jnstockley

Apache Airflow version

3.1.7

If "Other Airflow 3 version" selected, which one?

No response

What happened?

I am trying to change the timestamp of the logs, not for the access logs but specifically for the different Airflow components like the api-server, scheduler, etc.

I have used the AIRFLOW__LOGGING__LOG_FORMAT environment variable to change the format of the logs, and have used the --log-config argument to change the access log formats.

I have tried out AIRFLOW__LOGGING__LOGGING_CONFIG_CLASS and AIRFLOW__LOGGING__LOG_FORMATTER_CLASS to change the timestamp of the actual components, and this is where the issue lies. They don't seem to be respected.

Below is what I have set the environment variables too, and the contents of the python files.
AIRFLOW__LOGGING__LOGGING_CONFIG_CLASS=config.logging_config_class.LOGGING_CONFIG
AIRFLOW__LOGGING__LOG_FORMATTER_CLASS=config.log_formatter_class.TimestampConfig
logging_config_class.py
log_formatter_class.py

What you think should happen instead?

The Airflow component logs should respect the mentioned environment variables

How to reproduce

  1. Create the above files in the config directory, which gets copied during the docker build process
  2. Set the above environment vairables.
  3. Using docker look at the Airflow component logs, the timestamps are not being updated correctly.

Operating System

Debian

Versions of Apache Airflow Providers

apache-airflow-providers-oracle==4.5.0
apache-airflow-providers-google==20.0.0
apache-airflow-providers-amazon==9.21.0
apache-airflow-providers-postgres==6.6.0
apache-airflow-providers-microsoft-mssql==4.5.0
apache-airflow-providers-common-sql==1.32.0
apache-airflow-providers-sftp==5.7.0
apache-airflow-providers-openai==1.7.2
apache-airflow-providers-fab==3.4.0

Deployment

Other Docker-based deployment

Deployment details

Custom Dockerfile based off of the official slim docker images

Anything else?

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions