Skip to content

Support for Task Groups
Compare
Choose a tag to compare
@chriscardillo chriscardillo released this 31 Dec 19:29
v0.3.0
d3a4c05

gusty 0.3.0 provides full support for Airflow 2.x task groups through the use of subfolders in a DAG directory, as illustrated in the updated breakfast DAG.

gusty 0.3.0 task group support includes:

  • task_group_defaults - pass a default task group configuration as a dictionary to the create_dag function, which will take any parameter from Airflow's TaskGroup Class

  • METADATA.yml support - Task groups can also be configured with a METADATA.yml file in their folder, just like DAGs

  • prefix_group_id off by default - Explicitly set when you want a task group's id prefixed to each task group's task.

  • suffix_group_id - A gusty exclusive! Explicitly set when you want a task group's id suffixed to each task group's task.

  • smart dependencies (dependency levels) - A task can depend on any other task, regardless of task group. Task groups can depend on only other tasks / task groups that are their siblings.

  • smart dependencies (prefixes/suffixes) - If tasks depend on other tasks in their task group, where the task group specifies a prefix or a suffix, the prefixed/suffixed dependency id will be added to the list of task group dependencies automatically

The full gusty 0.3.0 release notes can be found here.

Assets 2