Not all default arguments are being passed to the DbtBuildOperator

[marcusintrohive]

Hey!

I noticed that the on_failure_callback from the default argument is not being passed to the DBTBuildOperator as wall as the sla argument.

I can see from the "Instance Details" these arguments are being passed to my other tasks but not to the airflow_dbt_python operators I bring in. Will try and come up with a PR for the fix, but if you have any guidance on where to start that would be great!

Thanks

[tomasfarias]

Hey @marcusintrohive. Thanks for opening an issue.

I'm not able to reproduce your problem: we have multiple operators running in production and we have on_failure_callbacks setup for all of them. We use the following recipe (only for illustration purposes, I've omitted/changed some things):

def notify():
    """Our failure callback.""""
    pass

DEFAULT_ARGS = {
    "on_failure_callback": notify,
}

with DAG(
    "dag_name",
    default_args=DEFAULT_ARGS,
    schedule_interval="0 * * * *",
) as dag:
    dbt_test = DbtTestOperator(
        task_id="dbt_test",
        project_dir=PROJECT,
        profiles_dir=PROFILES,
        target="prod",
        retries=1,
    )

Just to be sure, I re-checked our Airflow environment and all airflow_dbt_python operators appear to have our callbacks properly set when checking the Instance Details.

In order to help you I'll need more information:

• What airflow_dbt_version are you running?
• What Airflow version are you running? We are using a small hack to support Airflow v1, so maybe this is causing issues.
• Could you share some sample DAG code? Ideally something that highlights your callbacks and operators.

Thank you!

[tomasfarias]

For the record, we are running Airflow 2.2.2, and the latest airflow_dbt_python.

[marcusintrohive]

Thanks for the reply!

I gave you snippet above a little test, and what I notice is not all the default args are being passed for the dags which are using the @dag decorator and the TaskFlow API, but it does work when I test what you have as above with from airflow import DAG !

Airflow 2.0.2 and airflow-dbt-python 0.12.0

[tomasfarias]

Interesting, we don't really have any tests that cover the @dag decorator or the TaskFlow API, so it could be something related to that specific syntax. I'll write a separate issue to include some tests that cover these APIs to see if we can reproduce the issue.

In the meantime, I'd suggest upgrading to airflow-dbt-python 0.13.2, and/or helping by providing a sample DAG for me to reproduce the issue.

Thank you!

[tomasfarias]

This issue hasn't been active in a while and as we have added further test coverage for the @dag TaskFlow API, I assume this issue is good to close.

If any new information crops up a new issue can be opened, and we will take a look again.