Skip to content

Commit

Permalink
Mention mapped operator in cluster policy doc (#28885)
Browse files Browse the repository at this point in the history
  • Loading branch information
uranusjr committed Jan 12, 2023
1 parent f1eb2f1 commit 49e178e
Show file tree
Hide file tree
Showing 2 changed files with 3 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ If you want to check or mutate DAGs or Tasks on a cluster-wide level, then a Clu
There are three types of cluster policy:

* ``dag_policy``: Takes a :class:`~airflow.models.dag.DAG` parameter called ``dag``. Runs at load time of the DAG from DagBag :class:`~airflow.models.dagbag.DagBag`.
* ``task_policy``: Takes a :class:`~airflow.models.baseoperator.BaseOperator` parameter called ``task``. The policy gets executed when the task is created during parsing of the task from DagBag at load time. This means that the whole task definition can be altered in the task policy. It does not relate to a specific task running in a DagRun. The ``task_policy`` defined is applied to all the task instances that will be executed in the future.
* ``task_policy``: Takes a parameter called ``task`` that is of type either :class:`~airflow.models.baseoperator.BaseOperator` or :class:`~airflow.models.mappedoperator.MappedOperator` (for `dynamically expanded tasks <dynamic-task-mapping>`_). The policy gets executed when the task is created during parsing of the task from DagBag at load time. This means that the whole task definition can be altered in the task policy. It does not relate to a specific task running in a DagRun. The ``task_policy`` defined is applied to all the task instances that will be executed in the future.
* ``task_instance_mutation_hook``: Takes a :class:`~airflow.models.taskinstance.TaskInstance` parameter called ``task_instance``. The ``task_instance_mutation`` applies not to a task but to the instance of a task that relates to a particular DagRun. It is executed in a "worker", not in the dag file processor, just before the task instance is executed. The policy is only applied to the currently executed run (i.e. instance) of that task.

The DAG and Task cluster policies can raise the :class:`~airflow.exceptions.AirflowClusterPolicyViolation` exception to indicate that the dag/task they were passed is not compliant and should not be loaded.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@
specific language governing permissions and limitations
under the License.
.. _dynamic-task-mapping:

====================
Dynamic Task Mapping
====================
Expand Down

0 comments on commit 49e178e

Please sign in to comment.