Increase default worker_refresh_interval to 6000 seconds
#14970
Conversation
The default value for `[webserver] worker_refresh_interval` was `30` seconds for Airflow <=2.0.1. However, since Airflow 2.0 DAG Serialization is a hard requirement and the Webserver used the serialized DAGs, there is no need to kill an existing worker and create a new one as frequently as `30` seconds.
| ### Default `[webserver] worker_refresh_interval` is changed to `6000` seconds | ||
|
|
||
| The default value for `[webserver] worker_refresh_interval` was `30` seconds for | ||
| Airflow <=2.0.1. However, since Airflow 2.0 DAG Serialization is a hard requirement |
There was a problem hiding this comment.
Should this be in 2.0.2 (or 2.1) as it is not a bugfix but a good optimization?
This wouldn't affect the current deployment that have [webserver] worker_refresh_interval set in their airflow.cfg but affects new deployments and existing deployments that don't have it in airflow.cfg and rely on the default fallback
|
The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*. |
|
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest master at your convenience, or amend the last commit of the PR, and push it with --force-with-lease. |
github-actions
bot
added
the
full tests needed
The default value for
[webserver] worker_refresh_intervalwas30seconds forAirflow <=2.0.1. However, since Airflow 2.0 DAG Serialization is a hard requirement
and the Webserver used the serialized DAGs, there is no need to kill an existing
worker and create a new one as frequently as
30seconds.^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.