Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not all DAGs have DAG_DEFAULT_ARGS applied #3803

Closed
AetherUnbound opened this issue Feb 16, 2024 · 0 comments · Fixed by #3928
Closed

Not all DAGs have DAG_DEFAULT_ARGS applied #3803

AetherUnbound opened this issue Feb 16, 2024 · 0 comments · Fixed by #3928
Assignees
Labels
💻 aspect: code Concerns the software code in the repository 🛠 goal: fix Bug fix 🟨 priority: medium Not blocking but should be addressed soon 🧱 stack: catalog Related to the catalog and Airflow DAGs

Comments

@AetherUnbound
Copy link
Contributor

Description

When running the Elasticsearch health check DAG recently, I noticed that it didn't have the DAG_DEFAULT_ARGS applied (because a failure wasn't automatically alerted into Slack when it occurred). In addition to adding the default args to the following DAGs, we should also add a check or test which ensures that DAGs have an on_failure_callback set up (or some default args set up).

[nav] In [10]: from airflow.models import DagBag
          ...: d = DagBag()
          ...: for dag_id, dag in d.dags.items():
          ...:     on_failure_callback = dag.default_args.get('on_failure_callback')
          ...:     if on_failure_callback is None:
          ...:         print(dag_id)
[2024-02-16T20:42:53.422+0000] {dagbag.py:538} INFO - Filling up the DagBag from /opt/airflow/catalog/dags
rotate_db_snapshots
production_elasticsearch_cluster_healthcheck
staging_elasticsearch_cluster_healthcheck

Additional context

@AetherUnbound AetherUnbound added 💻 aspect: code Concerns the software code in the repository 🛠 goal: fix Bug fix 🟨 priority: medium Not blocking but should be addressed soon 🧱 stack: catalog Related to the catalog and Airflow DAGs labels Feb 16, 2024
@AetherUnbound AetherUnbound self-assigned this Mar 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
💻 aspect: code Concerns the software code in the repository 🛠 goal: fix Bug fix 🟨 priority: medium Not blocking but should be addressed soon 🧱 stack: catalog Related to the catalog and Airflow DAGs
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

1 participant