Add DAG level cluster policy

[turbaszek]

Description

Currently we have only task level cluster policy:

airflow/airflow/models/dagbag.py

Lines 386 to 387 in 7dfb398

	for task in dag.tasks:
	settings.policy(task)

This is an amazing tool that allows users to validate their tasks by either skipping DAG or setting default values (queues, owners, emails etc). I would like to proposes same mechanism for DAGs.

Some may argue that users already can implement it by accessing task.dag attribute in the policy, however this has the following drawbacks:

• it's not "explicit over implicit" see Default default_args #12086 (comment)
• the "DAG policy" is executed for every task (if there's 1000 tasks in DAG, the DAG policy will be checked 1000 times)

Use case / motivation

Create explicit mechanism for creating "DAG level" cluster policy that will be optimised for this task (check only once per dag not for every task) and will not required users to implement this custom optimisation logic (for example by monkey patching or changes in their fork).

Related Issues

#10282

[turbaszek]

CC: @mik-laj @jaketf @swalkowski @JavierLopezT @kaxil

[potiuk]

Good idea !

[mik-laj]

+1

[JavierLopezT]

Nice feature. Thanks for informing me