Skip to content

Commit

Permalink
Clarify lack of sync between workers and scheduler (#37913)
Browse files Browse the repository at this point in the history
* Clarify lack of sync between workers and scheduler

* Update docs/apache-airflow/core-concepts/overview.rst

---------

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
(cherry picked from commit 2867951)
  • Loading branch information
eladkal authored and ephraimbuddy committed Mar 6, 2024
1 parent a7b8882 commit c3ebe74
Showing 1 changed file with 8 additions and 0 deletions.
8 changes: 8 additions & 0 deletions docs/apache-airflow/core-concepts/overview.rst
Original file line number Diff line number Diff line change
Expand Up @@ -181,6 +181,14 @@ executed in the context of the scheduler.

.. image:: ../img/diagram_dag_processor_airflow_architecture.png

.. note::

When DAG file is changed there can be cases where the scheduler and the worker will see different
versions of the DAG until both components catch up. You can avoid the issue by making sure dag is
deactivated during deployment and reactivate once finished. If needed, the cadence of sync and scan
of DAG folder can be configured. Please make sure you really know what you are doing if you change
the configurations.

.. _overview:workloads:

Workloads
Expand Down

0 comments on commit c3ebe74

Please sign in to comment.