Container Log Groomer should be configurable with a log retention time <1d - OTEL Collector FileLogReceiver

[n-badtke-cg]

Description

Hi, thank you for the amazing ecosystem ❤️

We are using remote_logging and want to somehow use delete_local_logs, but with a delay (we are exporting the logs using OpenTelemetry Collector FileLog Receiver)

So in our current solution, we have set logging.delete_local_logs = "False" and the environment variable AIRFLOW__LOG_RETENTION_DAYS=1. The env is getting used in https://github.com/apache/airflow/blob/main/scripts/docker/clean-logs.sh

Now, even with only 1d retention, there are too many logs in our system, and the OTEL FileLog Receiver is unfortunately working very poorly with many files, I guess the receiver uses polling in some way.

The https://github.com/apache/airflow/blob/main/scripts/docker/clean-logs.sh script is using the command
find -type f -mtime +${RETENTION} where RETENTION is set to the AIRFLOW__LOG_RETENTION_DAYS env.

Why not just use (or additionally offer) AIRFLOW__LOG_RETENTION_MINUTES with find -type f -mmin +${RETENTION_MINUTES}? 🙂

Use case/motivation

More granular configuration option for log retention and thus better offering for largely scaled systems

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[shahar1]

Thanks for suggesting, feel free to contribute a fix!